HandWorldBench
Tsinghua University Zhongguancun Academy Manifold AI

HandWorldBench

A Unified Benchmark
for Hand-Centric World Models

Authors
  • Zhaolu Wang1,*
  • Jiacheng Xu2,*
  • Lihong Wang1,*
  • Shuqing Wang3,*
  • Jianjie Fang2,‡
  • Ziyou Wang2
  • Xvyuan Liu2
  • Rongze Tang1
  • Yiding Ma2
  • Xin Zhang4
  • Jiawei Chen5
  • Wei Wu4
  • Chen Gao2,†
  • Yong Li2,†
  • Zhibo Chen6

* Equal contribution† Corresponding authors‡ Project Leader

Affiliations
  • 1 Zhongguancun Academy
  • 2 Tsinghua University
  • 3 Tianjin University
  • 4 Manifold AI
  • 5 Zhejiang University
  • 6 University of Science and Technology of China
Corresponding authors
Project Leader
Paper
arXiv:2611.24731 (PDF) ↗
Code
HandWorldBench.code ↗Evaluation framework and data processing pipeline
Datasets
Leaderboard
View results ↗

Abstract

HandWorldBench evaluates hand-centric world models across human hands, simulated dexterous hands, and real dexterous hands. From over 600 hours of footage, we curate 20,500 clips and select 500 evaluation references spanning seven task families and 22 task categories. Seventeen metrics assess six dimensions, from visual quality and hand fidelity to physical consistency and temporal coherence.

Evaluating eleven models reveals a gap between controlling hand motion and predicting coherent, persistent changes in the world.

Three embodiments.
One benchmark.

Human hands, simulated dexterous hands, and real dexterous hands share a common language of interaction.

20,500Curated clips
500Evaluation cases
7Task families
22Task categories

Explore the interactions.

From folding and assembling to grasping, pouring, and stirring.

High-quality interaction data

20,000 curated clips across human, simulated, and real dexterous hands.

Operation diversity

Clip lengths

Unified Control

Explore three ways to represent the same hand motion, alongside the reference, model output, and scene conditions.

Six dimensions.
A fuller picture.

Evaluate what a world model sees, how it moves, and how the world changes over time.

Benchmark results.

Eleven models. Three embodiments. A shared evaluation protocol.

RankModel

Beyond hand motion.

Object-state evolution, persistence, and memory reveal complementary strengths across world models.