Interactive Robot Perception & Learning

SIREN proposes a unique systemic view of robot learning with a holistic representation of robot and environment as an integrated system. Robot and environment are NOT separate entities. 

We posit that robot and environment are not separate entities. They co-exist under the same constrained world and physical laws while exchanging information.

SIREN aims to uncover this underlying structure and information flow that govern the robot-environment interaction. Our goal is to unveil key properties of the action-perception cycle for developing embodied intelligence. We will, thus, study the intertwined flow of information and energy within the components of our proposed holistic robot-environment system.

We propose a framework that pioneers information-driven and physics-aware objectives. It aims to learn from embodied multisensorial streams a unified representation of the robot-environment system and its dynamics. Within this framework, we will study modular uncertainty estimation to promote robustness.

A key component is to ground high-level semantic information, e.g., from foundation models and affordances, onto robot-environment properties that allow for scaling and generalization of robotic behaviors within highly dynamic and unpredictable human-like environments.Eventually, we will investigate how to train uncertainty-aware, composable skills to adapt to new tasks.

SIREN’s breakthroughs will enable robots, particularly humanoid mobile manipulators, to merge in unstructured, human-like settings and perform challenging tasks that require smooth and efficient perception-action coordination, balancing generalization and robustness in the face of inevitable real-world uncertainties.

Our paradigm shift opens avenues for future groundbreaking research rooted in SIREN’s impacts toward continuous robot learning systems that are integrated and evolve with their environment.