北京雁栖湖应用数学研究院 北京雁栖湖应用数学研究院

  • 关于我们
    • 院长致辞
    • 理事会
    • 协作机构
    • 参观来访
  • 人员
    • 管理层
    • 科研人员
    • 博士后
    • 来访学者
    • 行政团队
  • 学术研究
    • 研究团队
    • 公开课
    • 讨论班
  • 招生招聘
    • 教研人员
    • 博士后
    • 学生
  • 会议
    • 学术会议
    • 工作坊
    • 论坛
  • 学院生活
    • 住宿
    • 交通
    • 配套设施
    • 周边旅游
  • 新闻
    • 新闻动态
    • 通知公告
    • 资料下载
关于我们
院长致辞
理事会
协作机构
参观来访
人员
管理层
科研人员
博士后
来访学者
行政团队
学术研究
研究团队
公开课
讨论班
招生招聘
教研人员
博士后
学生
会议
学术会议
工作坊
论坛
学院生活
住宿
交通
配套设施
周边旅游
新闻
新闻动态
通知公告
资料下载
清华大学 "求真书院"
清华大学丘成桐数学科学中心
清华三亚国际数学论坛
上海数学与交叉学科研究院
BIMSA > Advances in Artificial Intelligence Generative Simulation for Enabling Sim2Real Skill Discovery in Embodied AI
Generative Simulation for Enabling Sim2Real Skill Discovery in Embodied AI
组织者
孙明明 , 王雅晴
演讲者
Guiliang Liu
时间
2024年10月25日 14:00 至 16:00
地点
A3-2-301
线上
Zoom 293 812 9202 (BIMSA)
摘要
Embodied Artificial Intelligence refers to AI systems that are integrated with physical entities, enabling them to interact with the world in human-like ways. As a cross-disciplinary research field, Embodied AI encompasses topics such as perceptual understanding, interactive decision-making, and automatic policy adaptation, focusing on developing AI agents that consistently solve real-world tasks. Recent advancements in Large Language Models (LLMs) have significantly impacted the developmental trajectory of Embodied AI systems. These advancements facilitate the evolution from task-driven AI agents to Vision Language Agents (VLAs), which are generalist AI agents capable of solving multiple tasks based on various commands. Training such large multi-modality models typically requires extensive datasets. Moreover, unlike LLM datasets which are readily available online, the datasets for agent supervision include control signals conditioned on diverse environmental dynamics. Previous studies typically collect these datasets through manual teleoperation with real robots or by extracting motion features from other entities such as humans or animals. For training large models, the Real-to-Real pipeline is not computationally feasible. In this presentation, I introduce an alternative approach that utilizes a generative simulation to synthesize training data. The environment for this simulator is designed using a generative model capable of producing a wide range of objects, layouts, and contexts for robot training. By setting various goals and objectives through an LLM, we employ motion planning and reinforcement learning algorithms to identify skills represented by problem-solving trajectories. We demonstrate that the learned skills can address realistic problems through experiments conducted with real robots. Moving forward, we are exploring the possibility of extending this simulation approach across multiple embodiments, including robot arms and humanoids, to further enhance the versatility and applicability of Embodied AI systems.
北京雁栖湖应用数学研究院
CONTACT

No. 544, Hefangkou Village Huaibei Town, Huairou District Beijing 101408

北京市怀柔区 河防口村544号
北京雁栖湖应用数学研究院 101408

Tel. 010-60661855
Email. administration@bimsa.cn

版权所有 © 北京雁栖湖应用数学研究院

京ICP备2022029550号-1

京公网安备11011602001060 京公网安备11011602001060