Object Pose Estimation · 2025 · CVPR · CCF A

Leveraging Global Stereo Consistency for Category-Level Shape and 6D Pose Estimation from Stereo Images

The method represents shared shape within a category as an occupancy field and uses global stereo consistency to constrain coupled shape, scale, and pose optimization, targeting materials with unreliable depth sensing.

Paper

Research area

Object Pose Estimation

Task level

Category-level

Object setting

Unseen objects

Input modalities

calibrated stereo RGB

Requirements

left-right camera calibrationobject mask / cropcategory occupancy prior

Full paper metadata Authors, affiliations, research area, datasets, metrics, method tags, and reading status Expand

Authors: Junning Qiu, Minglei Lu, Fei Wang, Yu Guo, Yonggen Ling
Affiliations: Xi’an Jiaotong University / Tencent Robotics X
Conference / journal: Conference · CVPR · CCF A
Research area: Object Pose Estimation
Code availability: Unavailable / not yet recorded
Reading status: Overview:Available Deep Dive:Available
Task setting: Category-level · Unseen objects
Datasets: TOD S-RGBD
Evaluation metrics: 3D IoU n° mcm
Method tags: Category-level Pose Estimation 2D–3D Correspondences Pose Refinement Synthetic Data & Foundation Models Absolute Pose Estimation
Requirements: left-right camera calibrationobject mask / cropcategory occupancy prior

Reading Notes

Overview / Deep Dive

Available src/paper-notes/global-stereo-pose/coarse.md

Paper overview notes

Quickly assess the paper's scope, problem setting, core method, experimental findings, and whether to read it in depth.

Paper Reading Notes (Coarse): Leveraging Global Stereo Consistency for Category-Level Shape and 6D Pose Estimation from Stereo Images

1. Basic information

Paper: CVF Open Access
Authors / affiliations: Junning Qiu et al.; Xi’an Jiaotong University, Tencent Robotics X.
Task: category-level shape, scale, and 6D pose from calibrated stereo RGB.
Reading mode: Overview.

2. Positioning in one sentence

Learn a cross-view-consistent category occupancy field, then incorporate both cameras into global shape fitting to disentangle stereo shape / pose / scale ambiguity.

3. Problem and motivation

RGB-D can fail on transparent / reflective materials, while monocular RGB lacks metric scale.
Stereo supplies a baseline but tightly couples shape and pose optimization.

4. Method overview

Estimate an intra-category occupancy field shared across views.
Iteratively fit pose / scale with normalized-depth rendering and left–right information exchange.
Use baseline geometry to constrain optimization and reduce single-view projection ambiguity.

5. Experiments at a glance

Beyond TOD, the paper introduces / uses material-diverse S-RGBD and reports 3D_avg, 3D IoU, and real grasps.
Separate depth-friendly and depth-challenging objects rather than claiming universal RGB-D superiority.

Global Stereo Consistency main TOD and S-RGBD results — Original TOD / S-RGBD main results: bottle / mug IoU and pose thresholds on TOD; 3D_avg over ten material-diverse categories on S-RGBD. Source: original CVPR 2025 paper. Image copyright belongs to the paper's authors and publisher; reproduced here for paper analysis.

TOD bottle / mug 3D50 is 99.8 / 100 and 10°10cm 100 / 95.2. S-RGBD 3D_avg includes bowl 82.9 and wine glass 74.1; transparent / difficult-depth categories need separate analysis. RGB-D Second still reaches 86.2 / 91.2 / 87.8 on bowl / box / bottle, motivating stereo advantages specifically for challenging materials.

6. Quick assessment

Value: elevates stereo from two-image input to global geometric constraints.
Boundary: precise calibration and stable synchronization; category occupancy priors still limit generalization.

Available src/paper-notes/global-stereo-pose/deep.md

In-depth paper notes

Support reproduction, comparison, related-work writing, and identifying opportunities for further research.

Paper Reading Notes (Deep): Leveraging Global Stereo Consistency for Category-Level Shape and 6D Pose Estimation from Stereo Images

1. Key assessment

Paper: CVF Open Access
Key problem: without explicit depth, shape, scale, and pose can all explain projections. Local left–right matching leaves coupled parameters and incorrect optimization directions.
Answer: fix shared intra-category occupancy geometry, then exchange global-fitting information and impose baseline constraints.

2. Positioning in one sentence

Stereo becomes a global consistency constraint rather than an extra image, jointly converging category shape and 6D pose.

2.1 Guide to the paper’s figures and tables

Figure / table	What to look for
Figure 1	Projection ambiguity and baseline constraints on incorrect directions.
Figure 4	Adaptive occupancy queries defining category geometry.
Figures 5–6	S-RGBD collection and fitting convergence.
Tables 1–5	Distinct evidence from TOD, S-RGBD, depth-quality groups, ablations, and grasps.

Original Global Stereo Consistency category-shape and global stereo-fitting diagram — Original method diagram: a category occupancy field defines shared shape, and global left–right consistency constrains pose, scale, and fitting. Source: CVF Open Access. Image copyright belongs to the paper's authors and publisher; reproduced here for paper analysis.

3. Problem setting and boundaries

Calibrated left / right RGB produces shape, scale, and 6D pose for unseen category instances.
Neither monocular RGB nor active-depth RGB-D; baseline and synchronization are essential priors.
Categories are defined during training; occupancy is a category prior, not a universal generative model.

4. Method details

Intra-category occupancy: shared geometry lets both views constrain one latent shape, reducing ambiguity.
Divide-and-conquer fitting: begin with global shape, then iteratively render normalized depth and update pose / size, avoiding unrestricted joint drift.
Stereo consistency: exchange left–right optimization information, constraining scale / depth with baseline geometry.

5. Experimental analysis

TOD is established; S-RGBD adds material / depth-quality coverage of friendly and challenging objects.
Table 3’s depth-quality groups diagnose whether stereo actually addresses depth-sensor weaknesses.
Real grasping tests downstream usability without replacing strict shape / pose metrics.

Global Stereo Consistency constraint ablations and convergence curves — Original S-RGBD ablations and convergence: Table 4 removes monocular / global fitting, shape, rotation, y-translation, scale-ratio constraints, and training / rendering choices; Figure 6 compares convergence. Source: original CVPR 2025 paper. Image copyright belongs to the paper's authors and publisher; reproduced here for paper analysis.

Full-model $3D_{25}/3D_{50}/3D_{75}$ is 95.6 / 83.7 / 60.5. MonoVersion gives 45.7 / 30.5 / 7.6, and removing global fitting 42.4 / 32.4 / 9.6, supporting global stereo optimization beyond an extra image. No shape constraint gives 94.1 / 78.1 / 49.2; removing rotation, y translation, or $\lambda$ constraints yields $3D_{75}$ 49.8 / 51.3 / 44.2. All three shared constraints matter at strict IoU.

Rendering / training also matter: low-resolution occupancy gives $3D_{75}$ 23.8, high-resolution 38.6, and adaptive queries 60.5. Direct joint training gives only 5.7, separate training 46.3, with multistage training best. Occupancy-boundary sampling and curriculum are substantive, not arbitrary end-to-end stereo choices.

Global Stereo Consistency S-RGBD examples, failures, and robot grasps — Original S-RGBD examples, failure categories, and grasp table. Friendly / challenging depth groups are separated; failures include transparent reflection, perspective-height error, mutual / self-occlusion, and mask quality. Source: original CVPR 2025 paper. Image copyright belongs to the paper's authors and publisher; reproduced here for paper analysis.

Depth-friendly bowl / bottle / mug / can / spoon / cup grasp rates are 100 / 100 / 97 / 93 / 88 / 99%, versus depth-challenging 97 / 95 / 95 / 90 / 84 / 95%. Small gaps support material robustness compared with potential active-depth degradation. Masks, planning, controls, and trial counts still matter; grasp success is not pose accuracy.

Failures reveal boundaries: transparency reflects surroundings, perspective biases height, and mutual / self-occlusion permits multiple shapes / poses. The authors report tolerance of moderately poor masks but failures with very poor masks; stereo consistency does not replace reliable segmentation.

6. Authors’ claims and my assessment

Authors’ claims	My assessment
Global stereo consistency reduces coupling	Convergence / ablations agree; add baseline / calibration-noise sensitivity tests.
Stereo supports complex materials	Requires S-RGBD material splits; transparency can also damage both RGB correspondences.

7. Reproduction considerations and limitations

Match left–right intrinsics / extrinsics, rectification, timestamps, and occupancy coordinates.
Report iterations, initialization, and failure recovery; best-converged scores alone understate deployment difficulty.
Weak texture / reflection can create ambiguity in both matching and category priors.
Results use 5 GSF-Net iterations. Iterations, baseline, and calibration noise change accuracy–latency trade-offs and need deployment sensitivity tests.

Related papers

2026 · CVPR

3PT

3PT unifies detection, segmentation, and 6-DoF pose with two Transformers for 2D / 3D object-centric understanding. Continuous refinement of object representations and rendered-feature alignment support zero-shot cross-object perception on BOP-H3, BOP-Industrial, and BOP-Classic.

2026 · AAAI

6DAttack

Studies backdoor attacks on PnP-based and end-to-end 6-DoF pose pipelines. Structurally 3D triggers and modified pose labels are injected into training data so that models behave normally on clean inputs but produce controlled incorrect poses when triggered.

2026 · ICRA

Active Tactile Exploration

A violation-implicit loss incorporates physical-constraint violations into joint estimation of freely moving unknown convex rigid-body trajectories and cuboid / 20-vertex polytope shapes. Observed / Fisher information at a local MLE estimates EIG to select the next touch. In simulation, the EIG strategy yields stable errors below 4 mm after 5 cuboid actions; real Trifinger results at step 5 substantially outperform random selection. Convex geometry, local optima, multiple initializations, and tactile / proprioceptive errors limit the method.