Back to venue collections

3DV 2024 · Paper Collection

Object Pose Estimation Papers

3DV 2024 marks a shift toward more open object-pose settings: a few RGB references begin to replace precise models, NeRF and 3D Gaussians become pose-capable object representations, and pose expands into sparse-view modeling, physics-based tracking, and hand–object interaction.

12

Papers in this collection

5

Direct 6D / CAD pose

7

View alignment and object states

4 + 1

Included Orals + Spotlights

The official 3DV schedule lists 156 papers, including 21 Orals and 26 Spotlights. This collection focuses on direct 6DoF / CAD pose, object-centric sparse-view alignment, and explicit rigid / articulated object-state recovery. Camera localization alone, human pose, generic registration, and 3D detection alone are excluded.

Collection in One Sentence

Object pose expands beyond precise CAD and dense annotations

Cas6D, LocPoseNet, and NeRF-Feat lower onboarding costs through few-shot coarse-to-fine search, unseen-object location priors, and neural-field feature correspondences. YOLO-6D-Pose and HOC-Search represent end-to-end real-time regression and joint model–pose retrieval in scanned scenes.

FORGE, RelPose++, and LFM-3D address sparse-view alignment as a prerequisite for object pose. ContactArt, NCRF, physics-based tracking, and Dynamic 3D Gaussians show how real object states also include contact, articulation, velocity, friction, and dense time-varying motion.

Research Trends

Four themes to watch in 2024

These themes organize the 12 selected papers by task conditions and output variables; they do not represent the full research distribution of 3DV 2024.

01 · Few-shot Pose

New-object onboarding shifts toward a few reference images

Cas6D handles large pose gaps through candidate retrieval and cascaded feature volumes, while LocPoseNet addresses target-location priors often overlooked in unseen-object pipelines. Both treat few-shot conditions as a default system constraint.

02 · Neural Object Model

Neural fields become pose references as well as appearance models

NeRF-Feat learns view-consistent 3D features from NeRF for 2D–3D correspondence. Dynamic 3D Gaussians directly assigns time-varying 6DoF states to persistent Gaussians.

03 · Pose for Reconstruction

Sparse-view pose and shape are estimated jointly

RelPose++ resolves symmetry ambiguities with multiview attention; FORGE couples reconstruction features and relative pose; LFM-3D uses estimated 3D signals to improve wide-baseline matching and relative posing.

04 · Physical Interaction

Pose becomes part of contact and physical state

ContactArt constrains articulated-object pose with structure and contact priors; NCRF jointly optimizes contact, pose, and rendering; physics-based tracking estimates orientation, velocity, and friction together in an EKF.

Data Statistics

Research-area distribution

Each paper belongs to one group by its main output variables; tags retain cross-area attributes.

Direct 6D pose and CAD retrieval

5

Direct instance-level 6DoF pose, or joint CAD-model and object-pose retrieval in scanned scenes.

Object-view alignment and few-view reconstruction

3

Recovers relative camera poses from sparse, wide-baseline object images to establish a common reference frame for novel-object modeling.

Dynamic objects and hand–object interaction states

4

Extends object pose to temporal trajectories, physical properties, dense 6DoF motion, and articulated hand–object states.

Interpreting the statistics: 5 papers form the strict direct object-pose / CAD-pose retrieval core; 3 output object-centric camera poses or matches that establish reference frames for few-view reconstruction; another 4 extend to rigid, articulated, or dense dynamic states and should not be ranked directly with BOP-style metrics.

Presentation Mix

4

Oral

1

Spotlight

7

Poster

Recommended Reading

Recommended papers

Recommendations reflect problem coverage and methodological representativeness. Oral / Spotlight labels indicate official 3DV scheduling, rather than this site's assessment.

Paper List

Paper list and summaries

Each card gives task boundaries, input modalities, and a short summary. Select a title or detail link for coarse and deep notes; official OpenReview links remain available for checking the original paper.

5 papers

Direct 6D pose and CAD retrieval

Direct instance-level 6DoF pose, or joint CAD-model and object-pose retrieval in scanned scenes.

3DV 2024 Method paper Poster

Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable, Model-Free Approach using RGB Images

Panwang Pan · Zhiwen Fan · Brandon Yushan Feng · Peihao Wang · Chenxin Li · Zhangyang Wang

few-shotnovel objectRGB-onlycoarse-to-fineLINEMODGenMOP

Setting: Instance-level 6DoF pose of new objects from a few reference images without CAD

Input: few-shot reference RGB + query RGB

Summary: Cas6D uses a self-supervised ViT to reduce false target detections with very few references, retrieves top-K pose candidates, then refines them through multiscale warped feature volumes and progressively narrower pose bins. In the 32-shot setting, it reports Proj-5 gains over OnePose++ / Gen6D on LINEMOD and GenMOP, respectively.

3DV 2024 Method paper Poster

NeRF-Feat: 6D Object Pose Estimation using Feature Rendering

Shishir Reddy Vutukur · Heike Brock · Benjamin Busam · Tolga Birdal · Andreas Hutter · Slobodan Ilic

NeRFfeature renderingweak supervisionsymmetry2D–3D correspondence

Setting: Learns implicit object models for 6D pose from weakly annotated object views without accurate CAD

Input: weakly labeled RGB object views + query RGB

Summary: NeRF-Feat first learns implicit object geometry with NeRF, then jointly trains CNN features through contrastive learning to account for viewpoint invariance and symmetry. Query-image features match 3D NeRF features to solve pose in the neural-field frame, evaluated on LM, LM-Occlusion, and T-LESS.

3DV 2024 Method paper Poster

LocPoseNet: Robust Location Prior for Unseen Object Pose Estimation

Chen Zhao · Yinlin Hu · Mathieu Salzmann

unseen objecttemplate matchinglocation priortranslationrobustness

Setting: Test objects come from unseen categories, with no ground-truth object location in the query

Input: reference templates + query RGB

Summary: LocPoseNet addresses object locations often assumed known in unseen-object pose, introducing distributed reference kernels and multiscale correlation. Decoupled scale-aware / scale-robust features predict translation-related parameters, while synthetic data separately evaluates multiple location-noise conditions.

3DV 2024 Method paper Poster

YOLO-6D-Pose: Enhancing YOLO for Single-Stage Monocular Multi-Object 6D Pose Estimation

Debapriya Maji · Soyeb N Nagori · Manu Mathew · Deepak Poddar

single-stagemulti-objectmonocular RGBdirect regressionYCB-VLM-O

Setting: Single-forward-pass detection and direct 6D pose regression for multiple known objects from one RGB image

Input: single monocular RGB image

Summary: YOLO-6D-Pose jointly predicts classes, locations, and poses on YOLOX. Pose parameterization, dedicated losses, and 6D augmentation eliminate keypoints, PnP, and refinement postprocessing, shortening multi-object inference through an end-to-end single-stage design.

3DV 2024 Method paper Poster

HOC-Search: Efficient CAD Model and Pose Retrieval from RGB-D Scans

Stefan Ainetter · Sinisa Stekovic · Friedrich Fraundorfer · Vincent Lepetit

CAD retrievalRGB-D scanrender-and-comparepose searchscene understanding

Setting: Given approximate 3D boxes in scenes scanned by a moving RGB-D camera, jointly retrieves object categories, CAD, and poses

Input: RGB-D scan + CAD database + approximate 3D box

Summary: After comparing CAD–scan similarity measures, HOC-Search selects a depth-and-mask render-and-compare objective. A search tree over categories, models, and poses and a Monte Carlo Tree Search-like algorithm accelerate joint retrieval, with reported speedups of approximately 10×–120× over exhaustive search.

3 papers

Object-view alignment and few-view reconstruction

Recovers relative camera poses from sparse, wide-baseline object images to establish a common reference frame for novel-object modeling.

3DV 2024 Method paper Oral

LFM-3D: Learnable Feature Matching Across Wide Baselines Using 3D Signals

Arjun Karpur · Guilherme Mendeleh Perrotta · Ricardo Martin-Brualla · Howard Zhou · Andre Araujo

wide baselinefeature matchingNOCSmonocular depthrelative pose

Setting: Establishes local correspondences for relative pose between wide-baseline images of the same object with little shared visibility

Input: object-centric RGB image pairs + estimated 3D signals

Summary: LFM-3D encodes normalized object coordinates or monocular depth as a low-dimensional 3D positional signal for a graph-neural-network matcher. Its output differs from BOP object-to-camera pose, but improved matching substantially benefits relative posing of in-the-wild object-image pairs.

3DV 2024 Reconstruction and modeling Oral

Few-View Object Reconstruction with Unknown Categories and Camera Poses

Hanwen Jiang · Zhenyu Jiang · Kristen Grauman · Yuke Zhu

few-viewunknown categoryunknown camera poseNeRFjoint reconstruction

Setting: Joint viewpoint and shape recovery from approximately five images, with unknown object categories and camera poses

Input: five sparse RGB object views

Summary: FORGE predicts per-view 3D features, estimates relative camera poses through cross-view correlation, then fuses aligned features into a neural radiance field. Requires no ground-truth 3D shapes and demonstrates generalization to unseen categories and in-the-wild object images.

3DV 2024 Method paper Oral

RelPose++: Recovering 6D Poses from Sparse-view Observations

Amy Lin · Jason Y. Zhang · Deva Ramanan · Shubham Tulsiani

sparse views6D camera posesymmetrytransformerin-the-wild

Setting: Recovers 6D camera poses for a set of 2–8 object images under symmetry and weak texture

Input: 2–8 sparse RGB object views

Summary: Building on pairwise relative-rotation distributions, RelPose++ jointly reasons over multiple images through attention to resolve pairwise symmetry ambiguities and adds translation prediction decoupled from rotation ambiguity. Outputs can initialize sparse-view object reconstruction.

4 papers

Dynamic objects and hand–object interaction states

Extends object pose to temporal trajectories, physical properties, dense 6DoF motion, and articulated hand–object states.

3DV 2024 Method paper Oral

ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation

Zehao Zhu · Jiashun Wang · Yuzhe Qin · Deqing Sun · Varun Jampani · Xiaolong Wang

articulated objecthand-objectcategory-levelcontact priorsimulation data

Setting: Joint hand pose, articulated-object part pose, and contact states in real hand–object images

Input: RGB hand-object observations + simulated interaction priors

Summary: ContactArt collects object-pose and contact annotations in a physics simulator through iPhone-driven visual teleoperation. It learns a part-arrangement discriminator and a contact-region diffusion model, using both priors to constrain hand and articulated-object pose estimation in real images.

3DV 2024 Reconstruction and modeling Spotlight

NCRF: Neural Contact Radiance Fields for Free-viewpoint Rendering of Hand-Object Interactions

Zhongqun Zhang · Jifei Song · Eduardo Pérez-Pellitero · Yiren Zhou · Hyung Jin Chang · Ales Leonardis

hand-objectcontact fieldradiance fieldcanonical spacepose optimization

Setting: Mutual hand–object occlusion in sparse video degrades interaction reconstruction and pose accuracy

Input: sparse multi-view hand-object videos

Summary: NCRF jointly learns a contact optimization field, a hand–object radiance field, and an observation-to-canonical motion field, mutually constraining contact, pose, and appearance. Evaluates novel-view rendering and pose accuracy on HO3D and DexYCB.

3DV 2024 Method paper Poster

Physics-Based Rigid Body Object Tracking and Friction Filtering From RGB-D Videos

Rama Krishna Prasad Kandukuri · Michael Strecke · Joerg Stueckler

RGB-Drigid trackingdifferentiable physicsEKFfriction

Setting: Tracks rigid-object pose while estimating velocity and friction in sliding and collision scenes

Input: RGB-D video + mesh-based object shapes

Summary: Uses a differentiable physics simulator as the Extended Kalman Filter transition model, jointly filtering object position, orientation, velocity, and friction coefficient under contact and friction constraints. Explicitly enforces physically feasible trajectories beyond visual-only framewise tracking.

3DV 2024 Reconstruction and modeling Poster

Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis

Jonathon Luiten · Georgios Kopanas · Bastian Leibe · Deva Ramanan

3D Gaussian Splattingdynamic scenedense tracking6DoF motionanalysis-by-synthesis

Setting: Joint dynamic-scene reconstruction and tracking of all dense scene elements from multiple cameras and times

Input: multi-view, multi-timestep RGB images

Summary: Dynamic 3D Gaussians keeps Gaussian color, opacity, and shape persistent while positions and orientations change over time. Local-rigidity regularization and dynamic novel-view reconstruction yield dense 6DoF trajectories for nonrigid reconstruction and tracking.

Scope & Sources

Counting rules and scope

Inclusion: official 3DV 2024 papers directly outputting object 6DoF pose, CAD models and poses, object-centric relative camera pose, or explicit rigid / articulated object pose and motion states. LFM-3D, FORGE, and RelPose++ are grouped as object-view alignment to distinguish them from standard object-to-camera pose.

Excluded: camera localization / SLAM / SfM alone, generic two-view geometry, 3D detection, point-cloud registration, human pose, motion generation, and hand–object rendering or interaction synthesis using object pose only as a given input. Dynamic 3D Gaussians tracks dense-element 6DoF states and is distinguished from rigid-object benchmarks.

Primary sources: Official 3DV 2024 paper directory, the directory's official schedule CSV , and linked OpenReview paper pages. Summaries, classifications, and trends are editorial work and do not replace the original papers.