Methods
Classify methods by their required inputs, predicted intermediate quantities, and how they turn 2D observations into 3D poses, alongside their network architectures.
Template
Render many viewpoint templates from CAD models beforehand, then match observations against the template library at inference. Large-scale training is not always necessary, but template counts, rendering quality, and retrieval speed become crucial.
Best suited to
Industrial scenes with reliable CAD models, fixed categories, and controlled conditions
Challenges
Viewpoint coverage, lighting differences, and background clutter strongly affect matching
Keypoint
A network predicts 2D keypoints, and PnP recovers pose using their known 3D counterparts. The pipeline is clear and interpretable, while keypoint design and occlusion robustness are critical.
Best suited to
Textured objects with reliably localizable keypoints
Challenges
Occlusion and weak texture make a small keypoint set unstable
Correspondence
Predict each image pixel’s position in the object frame to form many 2D–3D correspondences, then solve pose through PnP or voting. Dense local cues provide more information than a small keypoint set.
Best suited to
General pipelines combining learned predictions with geometric solvers
Challenges
Incorrect correspondences propagate into pose errors, requiring RANSAC or confidence filtering
Dense
These methods typically predict dense object coordinates, surface embeddings, or per-pixel matching features. They provide richer information while being more sensitive to data quality and optimization details.
Best suited to
Complex texture, partial occlusion, and applications requiring precise alignment
Challenges
Higher annotation, GPU-memory, and postprocessing costs
Refinement
An RGB network or geometric matcher supplies an initial pose, followed by refinement using depth point clouds, rendered depth, and ICP-like methods. Refinement usually complements perception, improving stability over the final few millimeters or degrees.
Best suited to
Applications with depth cameras and demanding final alignment accuracy
Challenges
Depth noise, reflective materials, and occlusion boundaries affect point cloud alignment
Multi-object
Explore methods that simultaneously estimate multiple object poses or jointly use inter-object, multiview, and temporal relationships. The topic groups methods by input / output settings, then traces their development chronologically.
Best suited to
Cluttered piles, heavy occlusion, and simultaneous pose estimates for multiple objects
Challenges
Instance assignment, occlusion, inter-object consistency, multiview matching, and temporal fusion
Modern
Recent approaches combine large synthetic datasets, segmentation foundation models, vision-language models, and few-shot templates. They extend fixed-category systems toward more open object sets, increasing the importance of careful evaluation and reliability.
Best suited to
Expensive real annotations, rapidly expanding categories, and few-shot generalization
Challenges
Sim-to-real gaps, open-vocabulary localization, and missing CAD remain difficult
Problem Formulation
The library automatically adds problem-formulation tags from the 6D pose survey, alongside pose types and open-input settings, so papers can be filtered from several perspectives. Reference survey arXiv:2405.07801
Instance-level methods target specific objects seen during training; category-level methods handle unseen instances of known categories; novel-object methods emphasize generalization to new instances or categories outside training.
Absolute pose describes an object in the camera or scene frame. Relative pose describes relationships between reference views, image pairs, or local coordinate frames.
Use this tag when a method explicitly specifies targets through text prompts, language descriptions, or open-vocabulary semantics.
RGB methods are easy to deploy but offer less direct scale and geometric information. RGB-D methods use depth, subject to sensor noise and operating distance.
Direct pose regression offers a more end-to-end pipeline. Intermediate geometric predictions are more interpretable and connect naturally to classical modules such as PnP and ICP.
Fixed-category methods emphasize accuracy, while few-shot, CAD-free, and open-vocabulary methods emphasize adapting to new objects.
Related Directions
This page focuses on object pose estimation methods. For papers on visual localization, open-vocabulary segmentation, feature matching, VLA, or world models, explore the research map and organize them with the library’s research-area filter.
Explore the research map
