Beijing Institute of Mathematical Sciences and Applications Beijing Institute of Mathematical Sciences and Applications

  • About
    • President
    • Governance
    • Partner Institutions
    • Visit
  • People
    • Management
    • Faculty
    • Postdocs
    • Visiting Scholars
    • Staff
  • Research
    • Research Groups
    • Courses
    • Seminars
  • Join Us
    • Faculty
    • Postdocs
    • Students
  • Events
    • Conferences
    • Workshops
    • Forum
  • Life @ BIMSA
    • Accommodation
    • Transportation
    • Facilities
    • Tour
  • News
    • News
    • Announcement
    • Downloads
About
President
Governance
Partner Institutions
Visit
People
Management
Faculty
Postdocs
Visiting Scholars
Staff
Research
Research Groups
Courses
Seminars
Join Us
Faculty
Postdocs
Students
Events
Conferences
Workshops
Forum
Life @ BIMSA
Accommodation
Transportation
Facilities
Tour
News
News
Announcement
Downloads
Qiuzhen College, Tsinghua University
Yau Mathematical Sciences Center, Tsinghua University (YMSC)
Tsinghua Sanya International  Mathematics Forum (TSIMF)
Shanghai Institute for Mathematics and  Interdisciplinary Sciences (SIMIS)
BIMSA > Advances in Artificial Intelligence Generative Simulation for Enabling Sim2Real Skill Discovery in Embodied AI
Generative Simulation for Enabling Sim2Real Skill Discovery in Embodied AI
Organizers
Ming Ming Sun , Ya Qing Wang
Speaker
Guiliang Liu
Time
Friday, October 25, 2024 2:00 PM - 4:00 PM
Venue
A3-2-301
Online
Zoom 293 812 9202 (BIMSA)
Abstract
Embodied Artificial Intelligence refers to AI systems that are integrated with physical entities, enabling them to interact with the world in human-like ways. As a cross-disciplinary research field, Embodied AI encompasses topics such as perceptual understanding, interactive decision-making, and automatic policy adaptation, focusing on developing AI agents that consistently solve real-world tasks. Recent advancements in Large Language Models (LLMs) have significantly impacted the developmental trajectory of Embodied AI systems. These advancements facilitate the evolution from task-driven AI agents to Vision Language Agents (VLAs), which are generalist AI agents capable of solving multiple tasks based on various commands. Training such large multi-modality models typically requires extensive datasets. Moreover, unlike LLM datasets which are readily available online, the datasets for agent supervision include control signals conditioned on diverse environmental dynamics. Previous studies typically collect these datasets through manual teleoperation with real robots or by extracting motion features from other entities such as humans or animals. For training large models, the Real-to-Real pipeline is not computationally feasible. In this presentation, I introduce an alternative approach that utilizes a generative simulation to synthesize training data. The environment for this simulator is designed using a generative model capable of producing a wide range of objects, layouts, and contexts for robot training. By setting various goals and objectives through an LLM, we employ motion planning and reinforcement learning algorithms to identify skills represented by problem-solving trajectories. We demonstrate that the learned skills can address realistic problems through experiments conducted with real robots. Moving forward, we are exploring the possibility of extending this simulation approach across multiple embodiments, including robot arms and humanoids, to further enhance the versatility and applicability of Embodied AI systems.
Beijing Institute of Mathematical Sciences and Applications
CONTACT

No. 544, Hefangkou Village Huaibei Town, Huairou District Beijing 101408

北京市怀柔区 河防口村544号
北京雁栖湖应用数学研究院 101408

Tel. 010-60661855
Email. administration@bimsa.cn

Copyright © Beijing Institute of Mathematical Sciences and Applications

京ICP备2022029550号-1

京公网安备11011602001060 京公网安备11011602001060