Open-Ended Robot Learning through Imagination

The capability for imagination is valuable in robotics as it empowers robots to be more intelligent and responsive to new challenges. From a human cognition standpoint, imagination is often considered a foundational ability of planning and is defined as the ability of the brain to form images or sensations without any immediate sensory input. In robotics, imagination refers to the ability of robots to comprehend information across multiple input modalities (e.g., visual, auditory, and textual) to generate a representation of the current situation and then anticipate multiple possible future outcomes. This capability enables robots to understand and respond to a wide range of information, enhancing their overall abilities and making them more adaptable in various environments. This workshop explores the pivotal role of imagination in robotics and how imagination could facilitate continual learning, spanning various levels of perception, action, and long-horizon planning. Our discussion will mainly revolve around how imagination enhances robotic capabilities, including generative and diffusion models in perception and action and visual task planning. Beyond enhancing the capabilities of robots, we explore how these imagination models serve as versatile tools, offering insights into explaining complex concepts to non-expert users and generating invaluable training data.

The workshop is expected to attract researchers and professionals from academia and industry, who work in generative modeling, computer vision, manipulation, planning, and interaction. Additionally, end-users and researchers from high-tech companies are also expected to be interested in attending the workshop.