WOB_WorldOracle robot world model benchmark

di Polly Spring: AgiBot

A model that outputs a clean, successful grasp can still be describing something the hardware never managed. That mismatch is the thing WOB_WorldOracle exists to surface. Picture a gripper going for a piece of fruit. The real attempt misses. Feed that same starting image and the same sequence of actions into an action-conditioned world model (ACWM), and the rollout comes back as a textbook pick with nothing visibly off. The robot on the table still failed. AGIBOT and NVIDIA put the benchmark together with Princeton, UT Austin and the University of Toronto. What it checks is whether a behavior model forecasts what actually goes down once real actions run, and the method is comparison: each prediction gets lined up next to its paired real execution. Visual realism isn't the yardstick. The stakes are practical. Skewed forecasts mess with robot training, policy evaluation and closed-loop learning, so a rollout that looks right while getting the outcome wrong is a genuine problem. The question it keeps returning to: does a model track the actions it's given, failure included? Part 2/3 digs into the mechanics #AgiBot