HiPHI contains 617.5 hours of whole-body human motion recorded with optical motion capture at sub-millimeter accuracy. The collection is a humanoid motion capture dataset built for robot learning, and 245.7 hours of it covers people interacting with objects. Each of those interaction recordings comes with synchronized object trajectories and object meshes.
The release also includes a benchmark suite and results from policies trained on the data and run on a physical Unitree G1 humanoid robot.
The data gap behind humanoid learning
Humanoid robots have to balance, move and handle objects across a very wide range of situations. The data available to train them has limits at both ends. Internet video shows diverse behavior, but it cannot capture precise physical states such as exact joint positions or contact timing. Laboratory motion capture records movement accurately, but those systems usually cover only a narrow set of actions.
HiPHI is built to sit between those two. It keeps the accuracy of optical capture and aims for much broader coverage of what people actually do. The optical system tracks to sub-millimeter precision, which matters when the recordings are used as targets for a robot controller rather than as loose visual reference.
How coverage is organized
The collection is structured around FrameNet, a linguistic framework that describes human action. Instead of recording whatever is convenient, the approach uses the framework to guide what gets captured, so that a broad range of whole-body motion is covered in a systematic way. This is meant to address a common weakness in motion datasets, where large hours-of-footage totals can hide heavy repetition of a few action types.
The benchmark suite measures this directly. It evaluates motion diversity and interaction grounding, giving researchers a way to compare datasets on how much ground they cover and how well the motion is tied to the objects involved, not only on how many hours they contain.
Object interaction as a training signal
Of the 617.5 total hours, 245.7 are human-object interaction. That leaves 371.8 hours of other whole-body motion. The interaction portion pairs the human movement with the object's trajectory and mesh, recorded in sync.
That pairing is what makes the data usable for practical tasks. A robot learning to carry, push or pull something needs to know what the object did while the body moved, and where its surfaces were. Motion capture of a person alone does not supply that. Recordings that include the object let a training pipeline reward the correct outcome for the object as well as the correct body pose.
Training results and the real robot
Reinforcement learning policies trained on the data improve as the amount of data grows. The work also covers sim-to-real transfer, the step of moving a policy trained in simulation onto hardware. The policies were deployed on a Unitree G1 humanoid robot.
The source does not give task success rates, tracking error figures, the number of subjects, the list of objects, or the specific policy architectures. Those details determine how much weight the scaling and transfer claims can carry.
What this means, and what is still open
The main contribution is the combination of properties. Precise capture, broad and deliberately planned coverage, and synchronized object data are each available separately in other places. Internet video offers breadth without physical precision. Laboratory capture offers precision without breadth. A humanoid motion capture dataset that targets all three at a scale of hundreds of hours addresses the gap that the source identifies as limiting humanoid learning.
The inclusion of a benchmark may prove as useful as the data. Datasets are often compared by hours or by the number of clips. A suite that scores diversity and interaction grounding gives the field a more specific basis for comparison, provided other groups adopt it and apply it to their own collections.
Several questions remain unproven. The reported improvement with scale shows a trend, but the source does not say where it levels off or how large the gains are. Deployment on one robot, the Unitree G1, shows transfer to one body. Whether the same data and policies carry over to humanoids with different proportions, actuators or hands is not established. Carrying, pushing and pulling are also a limited slice of manipulation, and the source does not describe fine-grained tasks such as dexterous handling.
There is also the retargeting question. Human motion recorded at sub-millimeter accuracy still has to be mapped onto a robot with different limb lengths, joint limits and balance constraints. The accuracy of the recording does not by itself guarantee that the mapped motion is physically feasible for the robot. The source describes the transfer result but not how much of the work was done by the data and how much by the training and simulation setup.
For now, HiPHI is best read as a well-specified resource and a proposed yardstick. Its value will be settled by independent use: whether other labs train on it, whether the benchmark scores correlate with real-robot performance, and whether the scaling behavior holds on more varied hardware and tasks.