A Unified Benchmark
for Hand-Centric World Models
- Authors
- Affiliations
-
- 1 Zhongguancun Academy
- 2 Tsinghua University
- 3 Tianjin University
- 4 Manifold AI
- 5 Zhejiang University
- 6 University of Science and Technology of China
- Corresponding authors
- Project Leader
- Paper
- arXiv:2611.24731 (PDF) ↗
- Code
- HandWorldBench.code ↗Evaluation framework and data processing pipeline
- Datasets
-
- HandWorldBench ↗500 evaluation cases with inference inputs and scoring annotations
- HandWorldBench-Corpus ↗About 20,000 curated clips with unified control annotations
- Leaderboard
- View results ↗
Abstract
HandWorldBench evaluates hand-centric world models across human hands, simulated dexterous hands, and real dexterous hands. From over 600 hours of footage, we curate 20,500 clips and select 500 evaluation references spanning seven task families and 22 task categories. Seventeen metrics assess six dimensions, from visual quality and hand fidelity to physical consistency and temporal coherence.
Evaluating eleven models reveals a gap between controlling hand motion and predicting coherent, persistent changes in the world.
Three embodiments.
One benchmark.
Human hands, simulated dexterous hands, and real dexterous hands share a common language of interaction.
Explore the interactions.
From folding and assembling to grasping, pouring, and stirring.
High-quality interaction data
20,000 curated clips across human, simulated, and real dexterous hands.
Clip lengths
Unified Control
Explore three ways to represent the same hand motion, alongside the reference, model output, and scene conditions.
Six dimensions.
A fuller picture.
Evaluate what a world model sees, how it moves, and how the world changes over time.
Benchmark results.
Eleven models. Three embodiments. A shared evaluation protocol.
| Rank | Model |
|---|
Beyond hand motion.
Object-state evolution, persistence, and memory reveal complementary strengths across world models.
