Generalist Agent AI (GAA) is a family of systems that generate effective actions in a given environment based on the understanding of multimodal sensory input. With the advent of large foundation models, numerous GAA systems have been proposed in fields ranging from basic research to applications. While these research areas are growing rapidly by integrating with the traditional technologies of each domain, they share common interests such as data collection, benchmarking, and ethical perspectives. In this tutorial, we focus on the some representative research areas of Embodied GAA, namely embodied-multimodality, robotics, gaming (VR/AR/MR), and healthcare, etc., and we aim to provide comprehensive knowledge on the common concerns discussed in these fields. As a result we expect the participants to learn the fundamentals of GAA and gain insights to further advance their research. Specific learning outcomes include:
Led by esteemed experts from academia and industry, we expect that the tutorial will be an interactive and enriching experience, complete with lectures, case studies, Q&A sessions, and panel discussion ensuring a comprehensive and engaging learning experience for all participants.
Download links will be available
Time Slot | Talk Scheduling | Areas |
---|---|---|
08:30 - 08:40 | Jianfeng Gao | Opening Remarks |
08:40 - 09:20 | Talk1: Juan Carlos Niebles | LLM tool-based agents |
09:20 - 10:00 | Talk2: Katsushi Ikeuchi | Agent Robotics |
10:00 - 10:40 | Talk3: Yong Jae Lee | TBD |
10:40 - 11:00 | Coffee Break | |
11:00 - 11:50 | Katsushi Ikeuchi, Yong Jae Lee, Hoi Vo, Jianfeng Gao |
Panels and Q&A |
11:50 - 12:00 | Naoki Wake and Qiuyuan Huang | Ending Remarks |