Object Pose Estimation · 2025 · ICCV · CCF A

RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimation

Template-based pose estimation is formulated as object-centered camera-ray alignment. A diffusion Transformer conditioned on multiview templates with known poses denoises structured rotation / translation maps for coarse-to-fine inference.

Paper

Research area

Object Pose Estimation

Task level

Instance-level

Object setting

Unseen objects

Input modalities

RGBposed template views

Requirements

camera intrinsicstemplate images with known posesobject crop/detection

Full paper metadata Authors, affiliations, research area, datasets, metrics, method tags, and reading status Expand

Authors: Junwen Huang, Shishir Reddy Vutukur, Peter KT Yu, Nassir Navab, Slobodan Ilic, Benjamin Busam
Affiliations: Technical University of Munich / Munich Center for Machine Learning / XYZ Robotics
Conference / journal: Conference · ICCV · CCF A
Research area: Object Pose Estimation
Code availability: Unavailable / not yet recorded
Reading status: Overview:Available Deep Dive:Available
Task setting: Instance-level · Unseen objects
Datasets: LINEMOD Occlusion LINEMOD YCB-Video T-LESS
Evaluation metrics: ADD(S) / ADD(-S) ADD ADD-S BOP AR VSD MSSD MSPD Rotation Error Translation Error
Method tags: Novel-object Pose Estimation Template Matching & Render Retrieval Synthetic Data & Foundation Models Relative Pose Estimation
Requirements: camera intrinsicstemplate images with known posesobject crop/detection

Reading Notes

Overview / Deep Dive

Available src/paper-notes/raypose-iccv2025/coarse.md

Paper overview notes

Quickly assess the paper's scope, problem setting, core method, experimental findings, and whether to read it in depth.

Overview Notes: RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimation

1. Basic information

Year / venue: 2025 / ICCV; reading mode: coarse.
Task: estimate an unseen query object’s 6D pose from multiple template views with known poses.
Source: ICCV Open Access.

2. Positioning in one sentence

RayPose represents pose as object-centered camera-ray bundles and translation maps. A diffusion transformer denoises query poses conditioned on multiview posed templates, reducing dependence on nearest-template retrieval.

2.1 Guide to the paper’s figures and tables

Figure / table	What to inspect
Figure 2	Information flow through query and template encoders, the multiview fuser, and diffusion decoder.

RayPose Figure 2: diffusion transformer pipeline — Original Figure 2: RayPose ray-bundling diffusion pipeline. Source: ICCV Open Access. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

3. Method overview

Rotation uses camera rays from the object center; translation uses dense offset maps, avoiding conditional diffusion directly over continuous $SE(3)$ variables.
Query images and posed templates are encoded separately. Self-attention fuses template features, then a DiT decoder predicts clean pose maps.
Coarse training randomly samples templates; fine training restricts sampling near the query view. The architecture stays the same while the training distribution changes.

4. Experimental overview

Multiple instance-level unseen-object benchmarks evaluate pose accuracy under different template / reference conditions.
Authors’ claim: competitive unseen 6D pose estimation, especially reducing catastrophic failures from retrieving the wrong nearest template.
My analysis: rays provide geometrically interpretable denoising targets. Diffusion sampling, template pose accuracy, and multimodal solutions for reflective / symmetric objects still warrant separate evaluation within real BOP pipelines.

5. Strengths, weaknesses, and takeaways

Strength: combines generative modeling with template geometry rather than image-only regression.
Risk: posed template images remain necessary; speed, sampling steps, and symmetry handling require supplementary material / code checks.
Takeaway: treat a template set as a conditioning distribution rather than requiring selection of one candidate image.

Available src/paper-notes/raypose-iccv2025/deep.md

In-depth paper notes

Support reproduction, comparison, related-work writing, and identifying opportunities for further research.

Deep Reading Notes: RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimation

1. Basic information

Authors / affiliations: Junwen Huang et al.; TUM, MCML, XYZ Robotics.
Year / venue: 2025 / ICCV; reading mode: deep.
Source: ICCV Open Access.

2. Positioning in one sentence

RayPose formulates template-based unseen-object pose estimation as ray-bundle diffusion from multiview posed templates to the query. Geometrically constrained structured pose maps replace hard dependence on a single nearest template.

2.1 Guide to the paper’s figures and tables

RayPose Figure 2: query/template diffusion transformer — Original Figure 2: query / template encoders, multiview fuser, and conditional DiT decoder. Source: ICCV Open Access. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

3. Problem setting and representation

Input: an unposed query object crop and RGB templates with known object poses; output: the query’s absolute 6D pose.
Rotation: object-center-to-camera ray bundles geometrically connect object rotation with camera viewing direction.
Translation: dense translation offsets extend a scale-invariant translation formulation.
This constrains diffusion outputs beyond arbitrary six-dimensional vectors, but does not automatically resolve multiple solutions from rotational symmetry; check canonicalization / evaluation protocols.

4. Method details

Stage	Input	Output	Main purpose
Query encoder	Query RGB, view condition	Query embedding	Extract visual conditions for the target view
Template encoder	Multiple RGB views + known poses	Per-view features	Supply template geometry as a conditioning prior
Multiview fuser	Template features	Fused template token	Aggregate views through self-attention
Diffusion transformer	Noisy pose maps + conditioning tokens	Clean rotation / translation maps	Recover pose from multimodal candidates
Coarse-to-fine sampling	Different template sampling distributions	Coarse / fine predictors	Cover broad viewpoints before restricting to the query neighborhood

Training denoises pose maps rather than merely generating images; the same architecture learns coarse / fine prediction with two template sampling ranges.

5. Experimental analysis

Multiple instance-level pose benchmarks compare unseen-pose baselines, separating multiview settings, fine prediction, and relative-pose / translation design effects.
Authors’ claim: competitive template-based unseen-object results with fewer catastrophic initializations caused by closest-template retrieval failures.
My analysis: verify equal template budgets, template pose accuracy, and whether target crops are GT. Otherwise frontend or template database quality may amplify apparent generative-model gains.

6. Critical analysis

Strength: diffusion’s multimodal capacity operates on geometrically interpretable ray maps; template information is not reduced to one retrieved image.
Risk: diffusion steps, memory use, and template count may jointly affect deployment; ambiguous / symmetric objects can still yield multiple plausible ray solutions.
Reproduction challenges: ray / translation map construction, coordinate transforms from relative to absolute pose, and template sampling schedules.
Future directions: confidence-calibrated diffusion samples for multiple hypotheses; combine with MixRI point correspondences or BoxDreamer global corners.

7. Research takeaways

Template-driven pose estimation can move from retrieval to conditional probabilistic modeling, motivating uncertainty and multiple hypotheses with sparse references.

8. Additional original experimental figures

RayPose main BOP results and template sampling analysis — Experimental page excerpt: unseen-object comparisons on core BOP datasets and analysis of coarse / fine template distributions. Source: ICCV 2025 paper PDF. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

Cross-dataset tables support ray-bundle diffusion beyond one object set. Template distribution experiments address the method more directly: coarse training needs broad viewpoint coverage before fine training narrows the neighborhood. Starting with only nearby templates would again let retrieval initialization constrain performance; retaining a broad distribution throughout would hurt fine rotation estimates. The two-stage sampling policy is therefore part of the algorithm, not an interchangeable training detail.

Related papers

2026 · CVPR

3PT

3PT unifies detection, segmentation, and 6-DoF pose with two Transformers for 2D / 3D object-centric understanding. Continuous refinement of object representations and rendered-feature alignment support zero-shot cross-object perception on BOP-H3, BOP-Industrial, and BOP-Classic.

2026 · AAAI

6DAttack

Studies backdoor attacks on PnP-based and end-to-end 6-DoF pose pipelines. Structurally 3D triggers and modified pose labels are injected into training data so that models behave normally on clean inputs but produce controlled incorrect poses when triggered.

2026 · ICRA

Active Tactile Exploration

A violation-implicit loss incorporates physical-constraint violations into joint estimation of freely moving unknown convex rigid-body trajectories and cuboid / 20-vertex polytope shapes. Observed / Fisher information at a local MLE estimates EIG to select the next touch. In simulation, the EIG strategy yields stable errors below 4 mm after 5 cuboid actions; real Trifinger results at step 5 substantially outperform random selection. Convex geometry, local optima, multiple initializations, and tactile / proprioceptive errors limit the method.