Deep Reading Notes: Fast-WAM

1. Basic information

• Full title: Fast-WAM: Do World Action Models Need Test-time Future Imagination?
• Authors / affiliations: Tianyuan Yuan, Zibin Dong, Yicheng Liu, Hang Zhao; Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University; Galaxea AI
• Publication: arXiv 2026 preprint; these notes analyze v2 (2026-03-23), without adding unverified acceptance information to metadata.
• Primary sources: arXiv abstract, v2 full text, official project, official code
• Reading mode: deep; paper experiments are recorded separately from repository updates checked on 2026-09-12.

2. Positioning in one sentence

Fast-WAM separates two potential WAM benefits: future-video prediction can train current-observation representations without being an inference step before every action generation.

The paper offers a more economical policy parameterization and supporting experiments, not a theorem that future imagination is always useless.

Method inputs and outputs

Item	Content (compiled for this site)
Inference input	Current multi-camera RGB and language; cache current-observation conditions, then initialize the action expert with random action noise. Default inference neither receives real futures nor samples future-video latents.
Final output	A continuous action chunk of length 32 for robot execution / replanning, not object pose, segmentation, or future RGB.
Training input	Current images, language, future video, and corresponding demonstration actions. Current tokens stay clean; future video / actions receive noise. Attention prevents actions from reading future video.
Training objective	Video / action flow-matching fields optimized jointly as $L_{act}+\lambda L_{vid}$. Futures improve representations through supervision, not as deployment prerequisites.
Output details	Intermediate action fields become 32-step actions through 10 default denoising steps. Dimensions, joint / end-effector frames, units, and normalization depend on the benchmark; no universal specification makes them fixed 6D object transforms.
Requirements	Video-pretrained weights and robot demonstrations, without explicit CAD / depth / masks / intrinsics. Proprioceptive inputs require implementation checks; multiple cameras do not establish geometric triangulation.

2.1 Guide to the paper’s figures and tables

Figure / table	Evidence role, compiled by this site	Misreading to avoid
Figure 1	Defines Joint, IDM, and Fast-WAM dependencies	Skipping RGB decoding does not skip future-latent denoising
Figure 2(a)	Video / action expert architecture	Shared context does not make all tokens mutually visible
Figure 2(b)	Training / deployment visibility masks	Current frames must not read futures and leak them indirectly to actions
Table 1	RoboTwin Clean / Rand. results and ablations	Missing Rand. entries cannot represent complete two-setting averages
Table 2	Four LIBERO suites and ablations	High averages hide Spatial / Long gaps
Figure 3	Real dual-arm towel-folding platform	Not broad evaluation across many real manipulation tasks
Figure 4	Success, completion time, GPU latency	Faster does not imply every real-task quality metric improves

Figures are corresponding originals from the author project page. All Markdown tables are site compilations, not screenshots of paper tables.

3. Problem setting and task boundaries

3.1 Output is a policy, not object pose

Let $o$ be current vision, $l$ the language instruction, and $a_{1:H}$ an action chunk of length $H$. A conventional visuomotor policy learns:

$$p(a_{1:H}\mid o,l).$$

With future video $v_{1:T}$, an imagine-then-act policy is written as original Eq. (2):

$$p(a_{1:H}\mid o,l) =\int p(v_{1:T}\mid o,l)\, p(a_{1:H}\mid o,l,v_{1:T})\,dv_{1:T}.$$

Future video is an intermediate variable; this does not claim exact evaluation of the integral. Actual methods use generated samples or joint denoising. Controlled comparisons focus on one action chunk, omitting some previous methods’ outer autoregressive rollouts.

Fast-WAM instead uses:

$$p_\theta(a_{1:H}\mid o,l) =p_\theta(a_{1:H}\mid z(o,l)),$$

Here $z(o,l)$ encodes current context once, rather than a sampled future video. This policy-modeling form does not imply exact equivalence of a finite-capacity model to the true action distribution.

3.2 Training versus inference resources

Item	Training	Inference
Current cameras and language	Input conditions	Input conditions
Future video	Noised demonstration supervision	Default Fast-WAM generates none
Action chunk	Targets and noised targets	Recover continuous actions from noise
Generic video priors	Wan2.2-5B, video VAE, T5	Encoding and conditioning remain
Additional large-scale embodied pretraining	Not used by this model	Does not imply zero-shot policy
CAD / depth / object masks / intrinsics	Not listed as required model inputs	No explicit geometry-solving pipeline needed

Multi-camera images are concatenated before VAE encoding. No object canonical frame, output $R,t$, size recovery, symmetry classes, PnP, or ICP is defined. Multiple cameras should not imply calibrated triangulation. Action frames, dimensions, normalization, and proprioception are not specified uniformly enough in the text; use benchmark configurations for reproduction.

4. Research context

The following summarizes the paper’s Related Work, rather than asserting priority across all WAM literature.

Approach	Representatives discussed	Relationship
Direct VLA policies	OpenVLA, $\pi_0$, $\pi_{0.5}$	Similar deployment interfaces, different representations / objectives
Joint video–action generation	Motus, Unified World Models, etc.	Fast-WAM-Joint models tightly coupled generation
Future video before actions	LingBot-VA, Vidar, etc.	Fast-WAM-IDM conditions actions on generated futures
Reduced test-time video computation	VPP, UVA	Fast-WAM is not the first to bypass full video synthesis

VPP uses predictive visual representations from video diffusion; UVA jointly models video / actions and skips test-time video decoding. Fast-WAM emphasizes comparing training-video supervision with test-time future generation within one framework. Distinguish visible-video VAE decoding from multistep future-latent generation cost.

5. Research motivation

Seeing a future might help action selection, but leaves two engineering questions:

1. Has video-prediction training already encoded useful temporal information into current-observation features?
2. Does generating futures before every action provide enough control benefit for its cost?

The authors retain video supervision but prevent actions from reading future tokens from the start; inference removes that branch to compare quality and latency. This removes a deployment dependency, not video pretraining, demonstrations, or action denoising.

6. Method details

6.1 Architecture

A Mixture-of-Transformer (MoT) retains separate video DiT and action-expert branches interacting through shared attention. This is not sparse MoE routing that selects a few experts per token.

Module	Input	Output / role
T5 text encoder	Task instructions	Language conditions for cross-attention in all token groups
Pretrained video VAE	Concatenated multi-camera images / video	Current / future latent tokens
Wan2.2-5B video DiT	Clean current frames, noised future frames in training	Video training; current-context encoding at deployment
1B action-expert DiT	Action noise, current features, language	Action-field prediction and continuous chunks
Structured attention mask	Three token groups	Restricts dependencies to exclude futures from action conditions

The action expert follows video DiT structure with hidden size reduced to 1024. Core scale is 5B video + 1B actions, approximately 6B total; this does not account for all auxiliary encoders, caches, and optimizer memory.

6.2 Attention masks and prevention of future leakage

Let clean current-frame tokens be $C$, future-video tokens $F$, and action tokens $A$. Compiled from Figure 2(b), rows are queries and columns readable keys/values; visibility refers to the corresponding attention block.

Query / Key-Value	Current $C$	Future video $F$	Actions $A$
Current $C$	Visible	Hidden	Hidden
Future video $F$	Visible	Bidirectional within group	Hidden
Actions $A$	Visible	Hidden	Bidirectional within group

All groups additionally read language. Two safeguards matter:

• $A$ cannot directly read $F$.
• $C$ cannot read $F$ either, preventing indirect leakage through $F\rightarrow C\rightarrow A$.

Remove $F$ rows / columns for inference, preserving $C/A$ dependencies. Since $C$ does not depend on action noise, layer features / KVs can be cached after one pass rather than rerunning video DiT per action-denoising step.

Why can video supervision help without reading futures? My analysis: future prediction and current encoding share video-backbone parameters. Future losses alter those parameters and context representations. Learning through gradients differs from reading a future sample during a test forward pass; the mask blocks the latter, not parameter learning.

6.3 Joint flow matching

For action or future-latent target $y$, sample Gaussian noise $\epsilon\sim\mathcal N(0,I)$ and time $t\in(0,1)$, following original Eqs. (5)–(9):

$$y_t=(1-t)y+t\epsilon,$$ $$\mathcal L_{\mathrm{FM}}(y) =\mathbb E_{y,\epsilon,t}\left[ \left\|f_\theta(y_t,t,o,l)-(\epsilon-y)\right\|_2^2 \right],$$ $$\mathcal L_{\mathrm{act}}=\mathcal L_{\mathrm{FM}}(a_{1:H}), \qquad \mathcal L_{\mathrm{vid}}=\mathcal L_{\mathrm{FM}}(z_{1:T}), \qquad \mathcal L=\mathcal L_{\mathrm{act}}+\lambda\mathcal L_{\mathrm{vid}}.$$

Here $z_{1:T}$ denotes future-video target latents, unlike current-context $z(o,l)$ above. Under this interpolation, $t=0$ is data and $t=1$ noise, with target velocity $\epsilon-y$. Generation solves from noise toward data; reversing time conventions requires changing signs consistently.

The joint objective minimizes demonstrated action / video field errors, not task success directly; evaluation measures closed-loop behavior. The paper introduces $\lambda$ without listing its value in the implementation settings. Check weights and normalization in a fixed code version.

6.4 Controlled variants

Variant	Video objective	Training / inference dependence	Question
Fast-WAM	Retained	Actions read current context only; no generated test futures	Default design
Fast-WAM-Joint	Retained	Video / action attention enabled; joint test denoising	Is joint future imagination necessary?
Fast-WAM-IDM	Retained	Actions depend on futures; generate test video first	Is imagine-then-act more effective?
w.o. video co-train	Removed	Default architecture and inference retained	Does video-prediction training matter?

IDM adds noise to real training-video tokens with probability $p=0.5$ to reduce real-versus-generated future mismatch. Variants alter attention and some training processing, so this is not a strict factorial experiment changing only one switch. It is nevertheless more informative than comparisons across differently sized / trained external WAMs.

The no-video-co-training variant still starts from pretrained Wan. It tests additional downstream video supervision, not whether video pretraining itself helps.

6.5 Implementation and inference cost

Item	Paper setting, compiled by this site
Action horizon	32
Video sampling	Temporal downsampling by 4; 9 frames per chunk
Action-expert hidden size	1024
Noise schedule	Logit-normal time distribution
Inference denoising	10 steps; CFG scale 1.0
Optimizer	AdamW, learning rate $10^{-4}$, weight decay 0.01
Training stability	Cosine schedule, mixed precision, gradient clipping 1.0
Timing hardware	One NVIDIA RTX 5090D V2 32GB

Deployment encodes language and current images, runs video DiT once to cache context, initializes action noise, performs 10 action-expert denoising steps, outputs a chunk, and delegates execution / replanning to the robot interface.

A single-pass world encoder is not a single-step policy. No generated future does not mean no video model. Reported latency is hardware-specific model inference, without a full sensor, communication, safety-control, and execution-scheduling breakdown.

7. Experimental analysis

7.1 Data and protocols

Environment	Training data / steps	Evaluation protocol
LIBERO	Four suites, 10 tasks and 500 demonstrations each; 20k training steps	2000 evaluations across 40 tasks; per-suite success
RoboTwin 2.0	2500 Clean + 25000 randomized demonstrations; 30k steps	100 trials per task in Clean / Rand.; appendix lists 50 tasks
Real towel folding	60 hours of R1 Lite teleoperation; 30k steps	Success and mean completion time; inference latency separately

The RoboTwin text says over 50 tasks while the appendix lists 50; these notes invent no extra tasks. Training steps refer to this paper’s setup, not every external baseline’s historical training.

7.2 RoboTwin: averages and missing entries

Compiled by this site from Table 1, in %. Embodied PT means additional robot pretraining, excluding generic Wan video pretraining. Averages retain displayed paper precision.

Method	Embodied PT	Clean	Rand.	Average
$\pi_0$	Yes	65.92	58.40	62.2
$\pi_{0.5}$	Yes	82.74	76.76	79.8
Motus	Yes	88.66	87.02	87.8
Motus from Wan2.2	No	77.56	77.00	77.3
LingBot-VA	Yes	92.90	91.50	92.2
LingBot-VA from Wan2.2	No	80.60	Not reported	80.6
Fast-WAM	No	91.88	91.78	91.8
Fast-WAM-Joint	No	90.84	90.32	90.6
Fast-WAM-IDM	No	91.16	91.34	91.3
Fast-WAM w.o. video co-train	No	82.76	84.80	83.8

LingBot-VA from Wan2.2 has no Rand. value; its Average of 80.6 reflects only reported Clean performance. It cannot be compared as an equivalent two-setting mean. Clean values 91.88 and 80.60 can be compared, as can averages for methods with complete Rand. data.

Fast-WAM’s mean exceeds Joint / IDM by 1.2 / 0.5 percentage points, while removing video co-training loses 8.0 points. This scale comparison better addresses the central question than external-model rankings.

7.3 LIBERO: explicit futures can still help slightly

Compiled by this site from Table 2, in %, retaining paper average precision.

Method	Embodied PT	Spatial	Object	Goal	Long	Average
OpenVLA	Yes	84.7	88.4	79.2	53.7	76.5
$\pi_0$	Yes	96.8	98.8	95.8	85.2	94.1
$\pi_{0.5}$	Yes	98.8	98.2	98.0	92.4	96.9
LingBot-VA	Yes	98.5	99.6	97.2	98.5	98.5
Motus	Yes	96.8	99.8	96.6	97.6	97.7
Fast-WAM	No	98.2	100.0	97.0	95.2	97.6
Fast-WAM-Joint	No	99.6	99.4	98.2	96.8	98.5
Fast-WAM-IDM	No	98.8	97.8	97.8	97.6	98.0
Fast-WAM w.o. video co-train	No	89.2	99.2	95.4	90.0	93.5

Fast-WAM is 0.9 / 0.4 percentage points below Joint / IDM, so omitting test-time futures is not strictly lossless. Removing video training loses 4.1 mean points, with Spatial 98.2→89.2 and Long 95.2→90.0, consistent with RoboTwin’s broader direction.

My analysis: this supports the relative importance of training-video supervision. No independent-training variance or significance tests accompany these differences; sub-point gaps should not be treated as stable rankings, and a perfect Object-suite score does not establish open-object generalization.

7.4 Real experiment: lower latency with quality tradeoffs

The following is compiled by this site from Figure 4. Success / completion time are approximate scatter-coordinate readings; latencies are explicit bar labels. The text also explicitly reports 10% success without video training.

Method	Approx. success	Approx. mean completion time, s	Inference latency, ms
Pretrained $\pi_{0.5}$	100%	120	180
$\pi_{0.5}$ w.o. pretrain	40%	206	Not separately reported
Fast-WAM	75%	152	190
Fast-WAM-Joint	70%	227	580
Fast-WAM-IDM	90%	177	810
Fast-WAM w.o. video co-train	10%	241	190

The figure conveys more than a fourfold speedup:

• 810 / 190 is approximately 4.26 for IDM; 580 / 190 is approximately 3.05 for Joint.
• Fast-WAM and no-video-co-training have identical latency but sharply different quality, supporting retention of the training target.
• IDM still has higher real success, while Fast-WAM has shorter completion time and model latency.
• Pretrained $\pi_{0.5}$ leads this task’s success, completion time, and inference latency; Fast-WAM cannot be advertised as uniformly better than VLA.

Success measures eventual completion within sufficient time, while completion time measures efficiency. The text does not fully specify how failures enter average time, trial counts, or confidence intervals, limiting interpretation of the gaps.

7.5 Claims and evidence

Author claim	Evidence	My assessment
Strong policies without test-time futures	Fast-WAM / Joint / IDM on two simulations and one real task	Broadly supported, not lossless per task
Video co-training matters more than test-time generation	Removing video objectives hurts all three experiments	Strong within this recipe, not automatically every model scale
More suitable inference for real time	190 ms, faster than Joint / IDM	Supports lower model latency, not a complete real-time control system
Better world representations	Downstream success and ablations	Plausible, without direct representation / physical-prediction diagnosis
Competitive without embodied pretraining	Shared tables with pretrained baselines	Does not establish equal data, compute, or pretrained knowledge

7.6 Robustness and failure boundaries

RoboTwin Rand. tests scene randomization and real folding tests deformable manipulation. There is no complete breakdown by occlusion, lighting, textureless objects, camera movement, or novel-object difficulty, nor systematic failure-trajectory analysis.

Symmetry-equivalent object poses, depth noise, and mask errors are not directly evaluated. Not using these inputs does not prove robustness to their errors. Sufficiency of current frames in partial observability also remains underexplored.

8. Critical analysis

8.1 Design strengths

Separating training needs from deployment needs is reflected in actual caching through structured masks, linking the conceptual test to engineering savings. Two benchmarks plus a real task broaden evidence beyond one simulated suite.

8.2 Conclusions requiring restraint

First, explicit planning is not shown to be useless. Branch search, counterfactual action comparison, and multi-hypothesis assessment in risky scenes are not systematically studied. Generating one future video is not synonymous with world-model planning.

Second, video-prior sources are not fully separated. Removing downstream video training still retains pretrained Wan; no full comparison crosses video-pretraining presence with downstream-video-objective presence.

Third, data efficiency lacks full scaling curves. Without extra embodied pretraining, RoboTwin still uses 27500 demonstrations and real folding 60 hours. The recipe works but is not shown optimal for arbitrary few-shot budgets.

Fourth, benchmark scores do not establish reliable deployment. LIBERO approaches saturation and real tasks are narrow, without long-term failures, safe contact, or cross-robot transfer. A 32GB card’s 190 ms does not establish edge-hardware feasibility.

9. Reproduction and repository versions

9.1 Released resources

The source notes’ 2026-09-12 check found training, LIBERO / RoboTwin evaluation, and configuration in the official repository, with weights, LIBERO data, and RoboTwin data. This site verified resource entry points but did not train the model or reproduce benchmarks.

Preparation should follow this order rather than loading only a policy checkpoint:

1. Fix commits, model / simulator versions, and task configurations; record precision, denoising steps, and timing scope.
2. Prepare Wan components and the action DiT backbone per README, then task checkpoints and matching dataset statistics.
3. Match LIBERO data with MuJoCo versions; RoboTwin also requires simulator environments, scene assets, and policy interfaces.
4. Check camera ordering, action normalization, and success termination on a few tasks before full evaluation.
5. Keep data, training budgets, and rollouts consistent in controlled comparisons and report repeated-training variance.

9.2 Repository updates versus original paper

The README checked in the source notes added compiled inference, LeRobot 3.0, and Optional IDM. The latter lets a newer model choose imagination or direct current-frame execution; it is not the same checkpoint set as the paper’s four-row ablation.

The repository notes specify EVALUATION.sigma_shift=5.0 for original release weights, while newer default action-scheduler shift is 1.0. Latest defaults need not reproduce original checkpoint numbers.

The repository also notes unseen RoboTwin instructions differing from the compared LingBot-VA protocol. Unseen language and unseen objects are distinct splits; the former does not establish novel-object pose generalization. Strict comparison requires matched protocols. These are official README engineering supplements, separate from v2 main results.

9.3 Remaining checks

• Exact commits, hyperparameters, and loss weights for original results;
• Benchmark action representations, normalization, proprioception, and replanning execution steps;
• Real trial counts, initialization distributions, failure timeouts, and completion-time rules;
• Inclusion of text / VAE encoding, preprocessing, transfer, first compilation, and warmup in latency;
• Peak memory, training GPU hours, and seed variance, which cannot be inferred from parameter counts or one timing value.

10. Implications for 6D perception and embodied research

The following is my analysis and research advice, not paper findings.

• Geometry as supervision: test whether future pose, contact, or affordance prediction similarly shapes low-latency policies without requiring those outputs at deployment.
• Adaptive computation: default to current-frame execution and generate futures only under occlusion, ambiguity, or increased failure risk; evaluate trigger quality and total task cost.
• Representation diagnosis: probe pose / depth / motion on the same backbone to separate geometric dynamics from general multitask regularization.
• Fair controls: comparisons with Pose-VLA should match demonstrations, action heads, budgets, and splits. Video-temporal and explicit spatial supervision differ; existing cross-paper averages cannot isolate causes.

11. Summary

Fast-WAM’s useful conclusion is that within the tested tasks and recipe, retaining future-video training matters more than actually generating futures at test time. Structured attention turns this into one-pass visual encoding, while actions still require multistep generation and real success has tradeoffs. Complex planning and open environments need further evidence.