Video Reasoning · Topic Summary

Topic overview

Video reasoning gives a video generation model a first frame and a text instruction, such as a maze with "move the red dot to the green goal", and asks it to draw the solution. Unlike most video generation tasks, the result can be checked objectively. This summary covers 26 papers from 2025-09 to 2026-09, read in publication order.

Why now?

In 2025-09, Veo3-ZS claimed that Veo 3 solves such tasks zero-shot. Within three months a dozen benchmarks tested that claim, often reaching opposite conclusions in the same month. In 2026 the first million-scale training set (VBVR) appeared, followed by mechanism studies, VLM-assisted methods and, by 2026-09, stricter benchmarks and controlled scaling studies.

Background

Chain-of-frames vs chain-of-steps: does the answer emerge across frames in time, or across denoising steps of the whole clip?
Outcome vs process evaluation: checking only the final frame, or requiring every intermediate step to obey the rules.
Prompt rewriters: closed video APIs often expand the user's prompt with a language model before generation, which can write the answer into the prompt.
VLM judges vs rule-based checkers: who decides whether a generated video is correct.

Method landscape

2025-09 to 2025-11, claims and first cracks (papers 1–8): zero-shot demonstrations, benchmarks, and the discovery that reaching the goal is different from taking a legal path.
2025-12, opposite conclusions (papers 9–12): VMEvalKit and MMGR disagree in the same month; VIPER introduces process-aware judging.
2026-02 to 2026-04, data and mechanism (papers 13–17): VBVR's training set, chain-of-steps (Demystifying), early plan commitment (ChEaP), multi-turn failures (CLVG-Bench).
2026-05 to 2026-07, methods (papers 18–22): VLM planning (CollabVR), VLM rewards (VLM-as-Teacher), reasoning tokens (OpenCoF), hierarchical denoising (HDR), realistic input images (VIPE).
2026-08 to 2026-09, recalibration (papers 23–26): realistic strict tasks (VGI-Bench), rule-based judging (VBVR-Pro), object permanence (WROP), controlled scaling (Rubik-Scaling).

Idea map

The deck closes with four questions and the current evidence for each:

Who does the reasoning? Many gains come from language models. Turning off Wan 2.5's rewriter drops GSM8K from 78.4 to 0 (Thinking with Video); VLM planning raises VBVR-Bench from 0.671 to 0.757 (CollabVR); a VLM reward adds 0.115 over the 0.666 baseline (VLM-as-Teacher).
What counts as correct? Outcome-only success is 2.0–4.3 times process success (VIPER); VLM judges change 54.6–92.8% of scores when re-run at temperature 0 (VBVR-Pro); the best strict zero-shot scores are 5.6% (VR-Bench), 23.3% (VIPER) and 14% (VGI-Bench).
Where does reasoning happen? On denoising steps. Noising all frames at one step drops the score from 0.685 to below 0.3 (Demystifying); by step 5 the predicted trajectory already agrees 93% with the final one on 4×4 mazes (ChEaP); wrong-to-right switches mid-denoising happen in 2 of 702 cases (VGI-Bench).
Can it be taught? Synthetic data helps but stops improving after about 200K samples (VBVR), and helps less the further a task is from training (+40.5 vs +6.2, VGI-Bench). A realistic first frame raises Veo 3.1 from 41.3% to 59.3% (VIPE). Looking right is learned long before tracking the right state (Rubik-Scaling).

The source for the graph version is idea-map.md in this entry.

Core technical ideas

Three ideas recur. First, separate the outcome from the process, and score the process with rules where possible. Second, the plan forms early in denoising, so selection or verification can act on early predictions instead of finished videos (ChEaP's seed screening and relay raise long-maze success from 7% to 67%). Third, keep the hidden state explicit: hierarchical coarse-to-fine denoising lets a streaming model match a bidirectional one (HDR, 60.3% vs 60.0%), and feeding the cube state back to the video model raises accuracy from 31.1% to 67.3% (Rubik-Scaling).

Failure cases

Paths that cut through walls but stop at the goal; zero-shot video models reach the maze goal up to 83% of the time but follow the exact shortest path at most 5.6% (VR-Bench).
Multi-turn generation: under 10% success in the first round and complete failure from the second round on (CLVG-Bench).
State drift at the end of long sequences: walls disappear and colours merge in the final frames (HDR).
Objects reappear from nowhere, the wrong cup is lifted, or a ball passes through a hole half its size (WROP).
The model with the best visual quality reasons worst (MME-CoF-Pro, Kling).

Connections to our research

CollabVR's plan-render-check loop is close to storyboard planning in multi-shot generation. HDR's streaming results connect to autoregressive and long video generation. WROP and VIPE connect to physics-consistent generation. In WROP, edit models occupy most of the middle of the leaderboard, which matters for video editing. The evaluation questions continue the earlier discussion From Video Scores to World Consistency.

Research-quality prompts

Central claim: Much of the measured progress comes from language channels (rewriters, planners, reward questions) and from lenient outcome-only scoring; under strict process checks, the best zero-shot video models stay between about 5% and 25%.

Hidden assumption: Most papers assume the score measures the video backbone. Closed APIs rewrite prompts, VLM planners and judges inject or decide answers, and 13 of the 26 papers used closed APIs without mentioning the built-in rewriter.

Failure boundary: Methods fail when the task requires tracking hidden state over many steps (long mazes, multi-turn edits, occlusion, cube faces out of view), and when inputs move away from the synthetic training distribution.

Connection: The same hidden-state question appears in long video, multi-shot and world-model work: does the model keep the state, or only keep the appearance?

Concrete experiment: On one task set with rule-based scoring, compare three systems with the same language model: the language model answers directly; it writes a program that a deterministic renderer draws; and the video-plus-language system. No paper reports these two baselines.

Discussion seed: Is "video reasoning" a property of the video model, or of the whole system around it?

Open questions

Does video plus language beat language alone on any task family, under the same checker?
Which reporting rules should be mandatory: rewriter state, exact prompt, judge model and temperature?
If the plan is fixed within the first few denoising steps, where should search, verification and training signals act?
Why do synthetic-data gains saturate around 200K samples: limited task diversity, or limited model capacity?
Can loss, FVD or other training signals predict whether the hidden state is correct?
Why does a realistic first frame help so much, and does it transfer beyond physics-style tasks?

Possible projects

A small diagnostic benchmark that keeps the task fixed and varies only the channel that carries the answer (rewriter on or off, plan given or not, state given or not), scored by rules. A second direction: early-step verification for long-horizon generation, building on the finding that plans commit early.