UA-Pose: Uncertainty-Aware 6D Object Pose Estimation and Online Object Completion with Partial References
UA-Pose distinguishes observed and unobserved regions through a hybrid object model with uncertainty, allowing incomplete references to support pose confidence, robust estimation, and online shape completion.
Authors / affiliations: Ming-Feng Li et al.; CMU, Stony Brook, Tsinghua University, and Amazon.
Task: estimate query poses from references that show only part of an object, and use later observations to complete the object model online.
Reading mode: Overview.
2. Positioning in one sentence
UA-Pose explicitly marks seen / unseen regions of a partial object model with uncertainty, aligns reliable surfaces for pose estimation, and feeds high-confidence queries back into online completion.
3. Problem and motivation
Complete CAD models or dense multiview references are difficult to obtain in practice; missing regions in partial references should not count as negative evidence.
Pose and object completion are coupled: incorrect poses contaminate completion, while completion quality affects subsequent poses.
4. Method overview
Build an initial hybrid model from a few posed RGB-D images or a single unposed RGB image.
Learn / infer an uncertainty map and estimate poses with uncertainty-aware sampling.
Store queries with estimated poses in a memory pool, iteratively complete the model, and update the object representation.
Original Figure 1: either input setting produces an initial model with seen / unseen uncertainty, followed by RGB-D pose estimation and a memory pool of test-sequence observations for online completion. Source: CVPR 2025 paper PDF. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.
5. Experiments at a glance
With two posed RGB-D references on YCB-Video, UA-Pose achieves ADD / ADD-S / CD of 92.8 / 96.5 / 0.53, versus 87.4 / 94.3 / 0.57 for FoundationPose; initialized from a single RGB image through image-to-3D, UA-Pose achieves 93.2 / 96.9 / 0.62.
On YCBInEOAT, results are 89.99 / 94.35 / 0.57 with two references and 89.24 / 93.99 / 0.60 with a single RGB image, showing effectiveness beyond the relatively clean YCB-Video setting.
Under heavy hand-object occlusion on HO3D, the two-reference setting raises ADD from FoundationPose’s 45.67 to 91.51; CD improves only from 0.78 to 0.69. Low geometric reconstruction error and accurate pose are therefore not equivalent; texture and confidence in visible regions also matter.
6. Quick assessment
Value: explicitly account for unknown regions of a partial model during inference, rather than treating an incomplete model as complete CAD.
Risk: the online loop depends on confidence calibration. The full model triggers only 58 reconstructions on YCBInEOAT, versus 181 without uncertainty-aware completion; poor scheduling damages accuracy and substantially increases computation.
Available src/paper-notes/ua-pose/deep.md
In-depth paper notes
Support reproduction, comparison, related-work writing, and identifying opportunities for further research.
Deep Reading Notes: UA-Pose: Uncertainty-Aware 6D Object Pose Estimation and Online Object Completion with Partial References
Key problem: partial references lack an object’s back or texture; unobserved regions should not be treated as incorrect surfaces. Iterating pose and completion also creates error feedback.
Core answer: add uncertainty to a hybrid object representation, prioritize reliable observed regions during pose alignment, and use high-quality new observations for online completion.
2. Positioning in one sentence
UA-Pose turns incomplete references from a passive limitation into an explicit system state through uncertainty, linking pose confidence, sampling, and online completion.
2.1 Guide to the paper’s figures and tables
Figure / table
What to look for
Figure 1
Two initialization settings, partial RGB-D and single RGB, and the pose–memory–completion loop.
Figures 2–4
Roles of the hybrid representation, uncertainty-aware pose, memory pool, and image-to-3D initialization.
Tables 1–3
Different occlusion / hand-object interaction difficulty on YCB-V, YCBInEOAT, and HO3D.
Tables 4–5
Whether uncertainty and object completion provide independent benefits.
Original Figure 2 and adjacent text: a neural SDF yields a mesh with texture / geometry, then reference-view visibility defines a seen / unseen uncertainty map. This crop does not show the full pose-estimation or online-completion pipeline. Source: CVF Open Access. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.
3. Problem setting and boundaries
Two initial-reference settings: a few posed RGB-D images, or a single unposed RGB image converted into an initial model through image-to-3D.
Queries may form an RGB-D sequence, with a pose output for each frame and gradual model completion; this is therefore not a strictly single-frame-only benchmark.
Object association, camera calibration, and a reasonable initial partial model are required; this is not object discovery from scratch in arbitrary scenes.
4. Method details
Hybrid model: combine texture, geometry, and uncertainty, explicitly marking which model regions were actually observed in the references.
Uncertainty-aware pose: give low-uncertainty regions more trust during matching / sampling, preventing hallucinated unseen surfaces from misleading alignment.
Online completion: add queries and estimated poses to the memory pool, select informative observations to update the object representation, and improve subsequent poses.
5. Experimental analysis
YCB-Video input
Method
ADD
ADD-S
CD↓
2 references
FoundationPose
87.4
94.3
0.57
2 references
UA-Pose
92.8
96.5
0.53
single RGB + Image-to-3D
FoundationPose
88.9
93.6
0.67
single RGB + Image-to-3D
UA-Pose
93.2
96.9
0.62
UA-Pose improves both pose and reconstruction CD under the same initial-reference setting. Notably, the single-RGB-generated model has worse CD than two real RGB-D references, yet slightly higher ADD / ADD-S. Thus, closer geometry does not necessarily imply better pose: matching-friendly texture, visible-surface coverage, and uncertainty weighting matter more directly.
Left side of original Table 1 and the Table 2 heading: YCB-Video input / Comp. / ADD columns are visible, but the ADD-S / CD columns and YCBInEOAT table body are outside the crop. Full values in the text come separately from the original tables. "Comp." indicates online completion. Source: CVPR 2025 paper PDF. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.
On YCBInEOAT, two-reference ADD / ADD-S rises from FoundationPose’s 68.52 / 84.80 to 89.99 / 94.35; single-RGB initialization also improves from 83.39 / 90.66 to 89.24 / 93.99. The larger gains than on YCB-Video fit the expectation that online completion is more useful in manipulation sequences with more occlusion and new viewpoints.
The HO3D gap is more extreme: FoundationPose achieves 45.67 / 61.57 versus UA-Pose’s 91.51 / 95.56 with two references, and 72.06 / 87.87 versus 83.23 / 93.59 with single-RGB-generated models. Incomplete models alone are prone to misalignment under hand occlusion; uncertainty-aware sampling avoids treating unseen regions as reliable evidence.
Original Figure 5 and Tables 3–5: rendered alignment with / without online completion, HO3D results, ablations of four key components, and an additional comparison with RGB-D pose tracking methods. Source: CVPR 2025 paper PDF. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.
Table 4 makes the component effects clearer: without object completion, ADD / ADD-S is 68.52 / 84.80; without uncertainty-aware completion, it is 85.83 / 92.39 with 181 reconstructions; without image filtering, 87.70 / 93.37 with 87; without uncertainty-aware sampling, 87.88 / 93.39 with 59; and the full model achieves 89.99 / 94.35 with 58. Uncertainty governs both which surfaces support pose estimation and when / which frames update the model, with separate contributions from both paths.
6. Authors’ claims and my assessment
Authors’ claims
My assessment
Uncertainty improves partial-reference pose
HO3D’s 45.67→91.51 and multiple removals in Table 4 strongly support this; results should still be stratified by initial visible coverage to rule out scene-composition effects.
Online completion improves with more views
The full model achieves the best scores with 58 updates, whereas scheduling without uncertainty needs 181 and performs worse; selective completion matters more than blind accumulation.
7. Reproduction considerations and limitations
Record initial-model coverage, uncertainty parameterization, sampling policy, memory-admission threshold, and object-update schedule.
Incorrectly marking unseen regions as high-confidence damages both pose and completion, the main failure mode of the loop.
Fast motion, strongly specular / transparent surfaces, and frequent occlusion still face basic limits from temporal consistency and depth quality.
