Object Pose Estimation · 2025 · ICCV · CCF A

Unified Category-Level Object Detection and Pose Estimation from RGB Images using 3D Prototypes

A neural mesh with learned vertex features serves as a category-level 3D prototype. A single-stage RGB framework jointly performs detection, 6D pose estimation, and 9D pose refinement, avoiding a cascade of detection and single-object pose networks.

Paper Code

Research area

Object Pose Estimation

Task level

Category-level

Object setting

Unseen objects

Input modalities

RGB

Requirements

camera intrinsics3D category prototypesmulti-model RANSAC

Full paper metadata Authors, affiliations, research area, datasets, metrics, method tags, and reading status Expand

Authors: Tom Fischer, Xiaojie Zhang, Eddy Ilg
Affiliations: Saarland University / University of Technology Nuremberg
Conference / journal: Conference · ICCV · CCF A
Research area: Object Pose Estimation
Code availability: Code available
Reading status: Overview:Available Deep Dive:Available
Task setting: Category-level · Unseen objects
Datasets: REAL275 CAMERA25
Evaluation metrics: 3D IoU Rotation Error Translation Error
Method tags: Category-level Pose Estimation Multi-object Pose Estimation 2D–3D Correspondences Pose Refinement Synthetic Data & Foundation Models Absolute Pose Estimation
Requirements: camera intrinsics3D category prototypesmulti-model RANSAC

Reading Notes

Overview / Deep Dive

Available src/paper-notes/unified-detection-pose-iccv2025/coarse.md

Paper overview notes

Quickly assess the paper's scope, problem setting, core method, experimental findings, and whether to read it in depth.

Overview Notes: Unified Category-Level Object Detection and Pose Estimation from RGB Images using 3D Prototypes

1. Basic information

Year / venue: 2025 / ICCV; reading mode: coarse.
Task: RGB-only category-level multi-object detection, 6D pose, and 9D pose refinement.
Original paper: ICCV Open Access.

2. Positioning in one sentence

This work combines detection and single-object pose estimation around neural-mesh 3D prototypes, using shared 2D–3D feature matching to locate objects, estimate poses, and then estimate scale / deformation.

2.1 Guide to the paper’s figures and tables

Figure / table	What to examine
Figure 2	Unified DINOv2 features, 3D prototype learning, multi-model RANSAC, and 9D refinement.

Unified Detection and Pose Figure 2: unified RGB pipeline — Original Figure 2: unified detection / pose framework. Source: ICCV Open Access. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

3. Method overview

Frozen DINOv2 with trainable adapters and a geometric decoder outputs instance-level / category-level feature maps and foreground information.
Each category’s neural-mesh prototype stores vertices and learned features; dense 2D–3D matches feed multi-model RANSAC-PnP to discover multiple objects and their 6D poses together.
Deformation / scale is then optimized for each instance to obtain 9D pose. RGB-only operation is a key benefit, with independent-detector errors no longer fixed as an unrecoverable prerequisite.

4. Experimental overview

Datasets: REAL275 and CAMERA25, using category-level scale-agnostic metrics.
Authors’ claim: mean scale-agnostic performance on REAL275 exceeds the previous SOTA by 22.9%, with greater robustness than a single-stage baseline.
My assessment: sharing a 3D representation between detection and pose is a strong design; prototype construction, intrinsics, RANSAC multi-instance complexity, and 9D refinement cost still need code-level assessment.

5. Strengths, risks, and insights

Strength: estimate category-level poses of multiple objects directly from RGB, reducing error propagation between two separate models.
Risk: categories must still have predefined 3D prototypes; this is not category-agnostic.
Insight: matchable 3D prototypes can jointly express detection, orientation, and size for category-level pose, rather than assigning each to a separate head.

Available src/paper-notes/unified-detection-pose-iccv2025/deep.md

In-depth paper notes

Support reproduction, comparison, related-work writing, and identifying opportunities for further research.

Deep Reading Notes: Unified Category-Level Object Detection and Pose Estimation from RGB Images using 3D Prototypes

1. Basic information

Authors / affiliations: Tom Fischer, Xiaojie Zhang, Eddy Ilg; Saarland University and University of Technology Nuremberg.
Year / venue: 2025 / ICCV; reading mode: deep.
Paper: ICCV Open Access; code: GitHub.

2. Positioning in one sentence

Neural-mesh 3D prototypes provide a shared category representation, turning detection, 6D pose, and subsequent 9D refinement in RGB images into a common 2D–3D correspondence problem instead of a cascade of two models.

2.1 Guide to the paper’s figures and tables

Unified Detection and Pose Figure 2: single-stage category-level pipeline — Original Figure 2: DINOv2 feature extraction, prototype learning, multi-model RANSAC, and 9D refinement. Source: ICCV Open Access. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

3. Problem setting

Input: an RGB image containing multiple objects and camera intrinsics.
Output: each instance’s category, 2D localization, 6D pose, and 9D refinement including scale / deformation.
Category-level setting: instances may be unseen in training, while the category set and 3D category prototypes are known; RGB-D is not used.
Key difference from two-stage baselines: the model produces foreground / ROIs and pose matches itself, reducing errors fixed by an independent detector.

4. Method details

Module	Mechanism	Output
Feature extraction	frozen DINOv2 + trainable adapters + geometric decoder	instance/category feature maps, foreground maps
3D prototype learning	Contrastive learning of each category neural mesh’s vertices / features	Category 3D prototypes for recognition and matching
Detection & 6D pose	Dense 2D–3D matching + multi-model RANSAC-PnP	Multi-instance categories and 6D poses
9D refinement	Optimize deformation / scale near the instance pose	More accurate size and pose

Neural meshes replace NOCS regression so that 3D points carry matchable features, expressing multicategory detection and pose as the same geometric fitting task.
My assessment: multi-model RANSAC turns detecting several instances into geometric model fitting, while potentially becoming a runtime bottleneck as candidates increase.

5. Experimental analysis

Datasets: REAL275 and CAMERA25, using category-level scale-agnostic pose metrics.
Authors’ claim: mean scale-agnostic metrics improve over the previous SOTA by 22.9% on REAL275, with greater robustness than a single-stage baseline.
Evaluation checks: identical RGB-only settings, category labels as input or not, and matched detection-score thresholds / NMS. These can substantially affect multi-object mAP / pose success.

6. Critical analysis and reproduction

Strengths: RGB-only, multiple objects, and a shared representation demonstrate that category-level pose need not depend on depth or a two-stage design.
Limitations: prototype classification can propagate confusion between similar categories into pose; out-of-category objects, specular / transparent objects, and object-mask quality are not the main problems addressed.
Reproduction challenges: neural-mesh initialization, DINO-adapter training, dense matching, and multi-model RANSAC sampling / thresholds.
Extensions: uncertainty-aware correspondence weights in place of hard RANSAC; an open-vocabulary detector to broaden category entry points.

7. Research insights

3D prototypes challenge the assumption that detection and pose must be separate, making this a useful RGB category-level multi-object baseline.

8. Additional original experimental figures and tables

Unified detection and pose main results on REAL275 and CAMERA25 — Original Tables 1–3: scale-agnostic results on REAL275 and CAMERA25, plus scale-aware / absolute results on REAL275. This crop does not show a CAMERA25 scale-aware table. Source: ICCV 2025 paper PDF. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

Read the protocols separately: scale-agnostic metrics emphasize orientation and translation geometry, while scale-aware metrics also penalize within-category size errors. Gains on REAL275 are more informative about adaptation to the real domain and frontend detection errors; high CAMERA25 scores mainly reflect synthetic-domain fitting. Two-stage comparisons must also match detectors, category priors, and camera intrinsics, rather than attributing every frontend advantage to the pose head.

Related papers

2026 · CVPR

3PT

3PT unifies detection, segmentation, and 6-DoF pose with two Transformers for 2D / 3D object-centric understanding. Continuous refinement of object representations and rendered-feature alignment support zero-shot cross-object perception on BOP-H3, BOP-Industrial, and BOP-Classic.

2026 · AAAI

6DAttack

Studies backdoor attacks on PnP-based and end-to-end 6-DoF pose pipelines. Structurally 3D triggers and modified pose labels are injected into training data so that models behave normally on clean inputs but produce controlled incorrect poses when triggered.

2026 · ICRA

Active Tactile Exploration

A violation-implicit loss incorporates physical-constraint violations into joint estimation of freely moving unknown convex rigid-body trajectories and cuboid / 20-vertex polytope shapes. Observed / Fisher information at a local MLE estimates EIG to select the next touch. In simulation, the EIG strategy yields stable errors below 4 mm after 5 cuboid actions; real Trifinger results at step 5 substantially outperform random selection. Convex geometry, local optima, multiple initializations, and tactile / proprioceptive errors limit the method.