HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object Interaction

This White Paper gives robotics researchers and engineers an overview of a new large-scale motion capture dataset built to close the data gap limiting humanoid robot learning. It also shows how policies trained on the dataset transfer to a real humanoid robot.

What you will learn about:

• Why humanoid robot learning, a central problem in embodied AI and Physical AI, needs data that internet video and existing motion capture datasets cannot provide.
• How FrameNet, a linguistic framework for human action, can guide motion capture collection to systematically cover a broad range of whole-body motion.
• Why synchronized object trajectories and meshes make human-object interaction data useful for teaching robots real-world tasks such as carrying, pushing, and pulling.
• How reinforcement learning policies trained on this motion capture data improve with scale, and how sim-to-real transfer carries them onto a physical humanoid robot.