Monday 31 March 2025
Scientists have long sought to create realistic simulations of humans interacting with objects, but achieving this has been a daunting task. The complexity of human movements and the variability of object geometries have made it challenging to develop a system that can accurately mimic real-world interactions.
Recently, researchers have made significant progress in this area by introducing a framework called InterMimic, which enables a single policy to robustly learn from hours of imperfect motion capture (MoCap) data covering diverse full-body interactions with dynamic and varied objects. The key innovation lies in the use of a curriculum strategy, where the teacher policy is first trained on the MoCap data to mimic, retarget, and refine the reference frames.
The InterMimic framework consists of two main components: the teacher policy and the student policy. The teacher policy is responsible for mimicking the human movements from the MoCap data, while the student policy learns to imitate these movements by interacting with virtual objects in a simulation environment. The teacher policy acts as an online expert, providing direct supervision and high-quality references for the student policy.
One of the significant advantages of InterMimic is its ability to correct errors in reference interactions. This is achieved through a process called reference state initialization (RSI), which involves selecting initialization states from a buffer that stores successful imitation sequences. This approach allows the student policy to learn from a curated dataset, reducing the impact of MoCap data errors.
InterMimic has been tested on various datasets, including OMOMO, which provides a range of human-object interaction scenarios. The results show that InterMimic can effectively imitate these interactions, even when the reference frames contain significant errors. Furthermore, the framework demonstrates good scalability by training on hours of MoCap data and generalizing to unseen skills and object geometries.
The potential applications of InterMimic are vast. For instance, it could be used to generate realistic human-object interaction sequences for use in video games, movies, or virtual reality experiences. Additionally, the framework could be adapted for use in robotics, enabling humanoid robots to learn complex interactions with their environment.
While InterMimic is a significant step forward in the field of human-computer interaction, it is not without its limitations. For example, the system struggles with MoCap data that contains significant errors or artifacts. However, this limitation is mitigated by the teacher policy’s ability to correct these errors and provide high-quality references for the student policy.
Cite this article: “InterMimic: A Framework for Realistic Human-Object Interactions”, The Science Archive, 2025.
Human-Computer Interaction, Intermimic, Motion Capture, Full-Body Interactions, Dynamic Objects, Curriculum Strategy, Teacher Policy, Student Policy, Reference State Initialization, Robotics
