Any6D: Model-free 6D Pose Estimation of Novel Objects
Any6D learns joint object alignment and scale from one RGB-D anchor, then uses render-and-compare to select and refine pose / size hypotheses for unknown objects.
Authors / affiliations: Taeyeop Lee et al.; KAIST and NVIDIA.
Task: Establish a reference for a novel object from a single RGB-D anchor, then predict relative pose and scale in a query.
Reading mode: Overview.
2. Positioning in one sentence
Any6D combines a single reference, scale estimation, and pose alignment through joint object alignment, then selects and refines multiple hypotheses using render-and-compare.
3. Problem and motivation
Existing model-free methods often require multiple views or video. A single anchor is particularly challenging under nonoverlapping views, occlusion, and changes of environment.
RGB-D provides geometry, but does not guarantee that anchor and query dimensions / coordinates can be registered directly.
4. Method overview
Jointly align the anchor’s object representation with the query to estimate pose and metric size.
Construct several pose / size hypotheses and select the most consistent candidate through rendered comparisons.
Use geometric alignment and subsequent comparison to handle occlusion and large viewpoint changes, beyond pixel matching alone.
5. Experiments at a glance
Reported HO3D mean AUC results are ADD-S 98.7 and ADD 40.4, with AR 38.3. AR exceeds the compared Oryon 1.0, LoFTR 3.2, and Gedi 7.4.
AR is 51.0 on REAL275. On LM-O, the same CNOS + Wonder3D setup yields AR of 28.6, compared with GigaPose’s 17.5.
Original Any6D paper, main HO3D results: ADD-S is more tolerant of symmetric objects, ADD is stricter, and BOP AR uses a different combination of errors. These metrics are not interchangeable. Source: original CVPR 2025 paper. Copyright belongs to the paper's authors and publisher; reproduced here solely for paper discussion.
The same method achieves ADD-S AUC 98.7, ADD AUC 40.4, and AR 38.3 on HO3D, illustrating how a single symmetry-aware score can mask rotation or scale errors. Any6D has clear strengths, but the compared methods use different modalities: Oryon uses RGB-D and language, Gedi uses depth, and Any6D uses RGB-D. This is a system-level comparison, rather than a controlled causal ablation of an RGB or depth module.
6. Quick assessment
Contribution: Recasts a single reference as an explicit alignment and scale-estimation problem, rather than a reduced multi-view setup.
Boundaries: Requires reference / query masks and depth; regions absent from the single reference remain a fundamental information gap.
Available src/paper-notes/any6d/deep.md
In-depth paper notes
Support reproduction, comparison, related-work writing, and identifying opportunities for further research.
Deep Reading Notes: Any6D: Model-free 6D Pose Estimation of Novel Objects
Key problem: A single RGB-D anchor cannot cover the entire object, yet must provide pose, scale, and geometric evidence for an occluded query.
Main answer: Joint object alignment combines 2D–3D alignment and metric scale in one object representation; render-and-compare then selects and refines candidates.
2. Positioning in one sentence
Any6D treats single-reference unknown-object pose estimation as alignment with scale recovery, rather than merely regressing a relative transform from two images.
2.1 Guide to the paper’s figures and tables
Figure / table
What to look for
Method diagram
How does the anchor object representation connect to query pose / size hypotheses?
Table 1–5
Input conditions and metrics differ across HO3D, YCBInEOAT, Toyota-Light, REAL275, and LM-O.
Figure 3 / 6
Do object point-cloud alignment and shape quality support the subsequent pose claims?
Original paper, method diagram region: Any6D forms an object representation from a single RGB-D anchor and estimates query pose and size through alignment and render-and-compare. Source: CVF Open Access. Copyright belongs to the paper's authors and publisher; reproduced here solely for paper discussion.
3. Problem setting and boundaries
Inputs are one anchor RGB-D image and a query RGB-D image of the same novel object. The paper uses object masks in its benchmark evaluations.
Outputs are anchor-to-query relative pose and object size. Known coordinate relationships for the anchor can convert the result to object pose when needed.
Model-free means no manually supplied CAD / textured 3D model is required. It does not eliminate the need for a reference, depth, or segmentation.
4. Method details
Joint object alignment: Establish common object coordinates and geometric cues from anchor and query RGB-D, so pose and shape scale are estimated together.
Hypothesis generation: Sample orientation and translation / scale candidates to avoid immediate failure of a single match when views do not overlap.
Render-and-compare: Select and refine candidates using rendered / observed consistency, handling occlusion, lighting, and scene changes.
5. Experimental analysis
5.1 Directly checkable main results
Dataset and metric
Oryon
Gedi
Any6D
HO3D mean AUC ADD-S
23.0
71.9
98.7
HO3D mean AUC ADD
0.0
9.7
40.4
HO3D mean AR
1.0
7.4
38.3
Methods in the same table use different modalities: Oryon uses RGB-D+language, Gedi uses depth, and Any6D uses RGB-D. This supports performance under the stated setup but does not establish a causal advantage of any individual modality.
5.2 Interpreting cross-dataset results
REAL275: Any6D reaches AR of 51.0, above Oryon’s reported 46.5; Toyota-Light AR is 43.3.
LM-O: With the same CNOS + Wonder3D frontend, Any6D reaches AR of 28.6 versus GigaPose’s 17.5. This is closer to a pipeline comparison using predicted shapes.
Original Any6D paper, cross-dataset results. Frontends and metrics differ between subtables, so comparisons should stay within each subtable: mean ADD-S AUC is 89.3 on YCBInEOAT, AR is 43.3 / 51.0 on Toyota-Light / REAL275, and AR is 28.6 on LM-O with CNOS + Wonder3D. Source: original CVPR 2025 paper. Copyright belongs to the paper's authors and publisher; reproduced here solely for paper discussion.
In detail, Any6D achieves mean ADD-S AUC of 89.3 and ADD AUC of 45.6 on YCBInEOAT, exceeding Gedi’s 82.7 and 7.7, respectively. AR is 43.3 on Toyota-Light and 51.0 on REAL275. LM-O provides a stress test closer to real onboarding: with CNOS detections and Wonder3D-generated shapes, Any6D’s AR of 28.6 exceeds GigaPose’s 17.5 with the same frontend. Replacing Wonder3D with InstantMesh lowers AR to 25.2, showing that single-image 3D generation quality materially limits downstream pose accuracy.
Original paper, qualitative results and ablations: The upper region shows occlusion and cross-view cases; the lower region compares baseline shapes / alignment and reports the complete system's HO3D metrics. Source: original CVPR 2025 paper. Copyright belongs to the paper's authors and publisher; reproduced here solely for paper discussion.
When reading qualitative figures, examine more than whether the final outline appears to fit. Check visible-surface depth alignment, incorrect occlusion caused by unseen-surface completion, whether a symmetric object’s rotation is merely visually equivalent, and whether metric size stays consistent across scenes. The complete model reaches HO3D ADD-S AUC / ADD AUC / AR of 98.7 / 40.4 / 38.3. Their gap shows that even a very high symmetry-aware score can coexist with substantial asymmetric point errors and BOP errors.
6. Authors’ claims and my assessment
Authors’ claims
My assessment
One RGB-D anchor supports unknown-object pose + size
Supported on multiple benchmarks, but anchor/query overlap and mask quality remain conditions.
Handles large changes of environment
Domain-specific results on HO3D, YCBInEOAT, and Toyota-Light are persuasive; more extreme missing depth and transparent objects still need testing.
7. Reproduction considerations and limitations
Alignment requires consistent depth scale, anchor/query camera intrinsics, and object-mask boundaries.
Record candidate counts, pose / size sampling ranges, and refinement iterations; these determine the accuracy–runtime tradeoff.
Unobserved surfaces in a single reference still lack measured geometry. Render-and-compare should not be interpreted as recovery of the complete object.
Cross-method comparisons must also record whether the reference is real RGB-D, a predicted mesh, or synthetic views. Model-free approaches may shift CAD costs to reference capture, segmentation, or single-image 3D generation, rather than eliminating object priors altogether.
