6 DoF
The independent dimensions of rigid motion: translation in R³ plus rotation in SO(3), determined by the physical problem.
00 Introductory 3D Vision Visual SLAM Robotics Robotic Grasping
6D Object Pose Estimation · Field Guide
Build the big picture,
then explore the geometry
6D pose specifies where an object is and how it is oriented relative to a reference frame. The actual problem also depends on object priors, the camera, input modalities, detection conditions, temporal scope, scale, symmetry, and the downstream task. This chapter maps these choices together.
Position + Orientation + Reference Frame
Pose describes a rigid relationship between two frames
Position specifies the object origin; orientation specifies how its three axes are expressed in the reference frame. Changing the CAD origin, canonical orientation, or reference frame changes numerical labels while leaving the physical scene unchanged.
Default convention: object → camera
Homogeneous rigid transform
VISIBLE EFFECTProjection into the image
Given camera intrinsics and distortion, apply CTO to CAD points to obtain camera coordinates, then use perspective projection to obtain keypoints, 3D boxes, silhouettes, and rendered depth. Checking their alignment with the image is a direct test of pose conventions.
Explore camera projection →Degrees of Freedom
Distinguish physical DoF, parameter count, and model output dimensions
DoF counts independent dimensions of a system's state. A representation may be redundant, constrained, or many-to-one. Quaternions have four components constrained to unit norm, with q and −q equivalent, so rotation still has only three physical degrees of freedom.
| State | One representation | Physical meaning | Common applications | Keep in mind |
|---|---|---|---|---|
| 3 DoF | t = [x, y, z]ᵀ | Estimate position only; orientation, shape, and scale are known or fixed | Target-center localization and 3D tracking with orientation-independent markers | Not a full object pose: any rotation remains possible at the same position |
| 4 DoF | [x, y, z, yaw] | Gravity fixes roll / pitch, leaving translation and yaw | 3D boxes in autonomous driving, planar mobile robots, and upright objects | Upright orientation is a scene prior that does not apply to arbitrary handheld objects |
| 6 DoF | (R, t) ∈ SE(3) | 3D position + 3D orientation, with fixed rigid-body dimensions | Instance-level object pose, camera localization, and robotic manipulation | R may use 3, 4, 6, or 9 parameters, but rotation still has only 3 DoF |
| 7 DoF | (s, R, t) ∈ Sim(3) | 6D rigid pose + one uniform scale s | Monocular map alignment and objects or trajectories with unknown uniform scale | The 7 outputs of quaternion(4) + translation(3) do not imply 7 DoF |
| 9 DoF | (R, t, sx, sy, sz) | 3 translations + 3 rotations + 3 axis dimensions | Joint category-level pose and instance-size estimation | Nonuniform scaling is not rigid motion; the canonical frame affects label semantics |
7 values can represent 6 DoF
A unit quaternion q(4) + t(3) uses 7 components, but the norm constraint and double coverage add no physical motion.
Outputs may have more dimensions
A network may predict a 6D rotation representation, classification bins, multiple keypoints, or a probability distribution, then map them into the same pose space.
One Relation · Two Directions
Object and camera poses express the same relationship in opposite directions
If ^C T_O transforms object coordinates into camera coordinates, its inverse ^O T_C expresses the camera's position and orientation in the object frame. Inverse translation requires re-expression in the inverse transform's target frame, rather than simple negation.
Rigid inverse
Connect world, camera and object
Which frame contains the input point?
The subscript / right-side O indicates object-frame input coordinates.
Which frame should express the result?
The superscript / left-side C indicates camera-frame output coordinates.
Do t, CAD, and depth share units?
Rotation is dimensionless; translation, the model, and depth must consistently use m or mm.
What are the axes and multiplication conventions?
OpenCV, OpenGL, ROS, and graphics engines may use different camera-axis conventions.
Interactive Overview Lab
Explore degrees of freedom, transform directions, and task settings in one lab
Observe how degrees of freedom change the state space, then invert the same object–camera relationship. Finally combine test-time object priors, observation modalities, and single-frame / video conditions to identify the task you are solving.
Interactive · Pose at a Glance
Degrees of Freedom, Transform Directions & Task Settings Lab
Compare 3/4/6/7/9 DoF on the same object, invert the transform between object pose and camera pose, then identify your task from the priors and modalities available at test time. right-handed column vector object → camera
Object axes Visible 3D box Object origin t Perspective display builds intuition; calculations use the full R and t
Task
3D position
Rotation and size are fixedState vector
Experiment outputs
3 values = 3 DoFTranslation
Object origin in the camera frame
Units must match the CAD model / sceneRotation / size
Orientation and size
— Parameter count differs from physical degrees of freedom. A quaternion q has 4 components constrained to unit norm, and q and −q represent the same rotation. The 7 values in q + t therefore describe a 6-DoF rigid pose, not 7-DoF physical motion.
object → camera
CTO
camera → object
OTC = (CTO)⁻¹
Name the same geometric relationship from either end. Camera pose and object pose describe opposite directions of the same relative transform. Errors often arise from frame names, the side of matrix multiplication, or the frame in which t is expressed.
Your task family
Instance-level · Seen object · Single-frame RGB 6D pose
A fixed CAD model / object frame is available. Learned object-specific appearance and geometric cues recover the object-to-camera transform.Prior
Assets available at test time
Exact CAD model, object ID, and trained weightsMetricity
Scale and depth
CAD provides scale; RGB recovers t through projection constraints and priorsOutput
Typical outputs
6D rigid pose · CTOAmbiguity
Main ambiguities
Occlusion, symmetry, weak texture, and monocular depthRecommended next concepts
Coordinate frames → Rotations → Cameras → PnP → General pipeline
Model-free methods can still require object priors. Some methods avoid CAD but require multiple posed reference images, object scans, or online reconstruction. Check whether references provide scale and camera poses, and whether test-time object onboarding is allowed.
Task Taxonomy
Define a 6D pose task by what is known at test time
Available geometry and result comparability depend on exact CAD, exposure to test objects during training, known object IDs / ROIs, full-image detection requirements, and whether reference images or onboarding are allowed.
| Task family | Test-time priors | Typical outputs | Questions to ask when reading papers | Representative settings |
|---|---|---|---|---|
| Instance-level · Seen objects | Training and testing share object IDs, typically with exact CAD, known scale, and a fixed object frame | A 6D rigid pose for each known instance | Is the target given? Are ground-truth boxes / masks supplied, or must it be detected in the full image? | Classical BOP localization / detection and LINEMOD benchmarks |
| Unseen objects · CAD-based | Test objects are absent from training; exact CAD is supplied at test time, with offline onboarding allowed | Novel-object 6D pose, often estimated from coarse hypotheses to refinement | Does CAD have metric scale? What are the onboarding costs and detector requirements? | Render-and-compare approaches such as MegaPose |
| Category-level · Unseen instances | Category and canonical / shape priors are available, without exact CAD of the instance | 6D pose + size, often called 9-DoF pose | How are category canonical orientation, dimensions, continuous symmetry, and shape variation defined? | NOCS and category-level pose-and-size estimation |
| Reference-based / CAD-free | Reference RGB images, RGB-D scans, video, or neural object representations are supplied before or during testing | Relative or metric 6D pose, depending on whether references provide scale and camera poses | Does model-free mean no CAD, or does it also eliminate object onboarding? | Settings such as FoundationPose's model-free mode |
| Open-world / open-vocabulary | Objects and categories form an open set, often using text, retrieval, general segmentation, and foundation features | Identity, instance, pose, confidence, and rejection status | What does zero-shot mean for objects, categories, and scenes? How are unknown targets discovered? | Novel-object / open-vocabulary extensions |
Are the targets present in the image already known?
Localization typically receives information about target presence; detection must discover and identify instances in the full image. Ground-truth crops, masks, or known object IDs substantially simplify the task.
CAD-free methods can still require object priors
Model-free methods may require posed references, RGB-D scans, object videos, or online reconstruction. Check whether references supply scale and camera poses.
Specify what is unseen
Unseen test instances, unseen categories, unseen scenes, and no per-object training are different claims. List training data, assets, and test-time operations explicitly.
What Does the Sensor Actually Observe?
RGB, depth, point clouds, and multiple views provide different evidence
Modalities define measurement mechanisms and available information, beyond input-channel counts. RGB-D is not automatically reliable: alignment, scale, validity, and time synchronization must be specified.
| Modality | Direct observations | Main capabilities | Missing information and failures | Common geometric approaches |
|---|---|---|---|---|
| RGB | Color, texture, silhouettes, perspective, and occlusion relationships | High-resolution appearance; known CAD and K allow projection constraints to recover metric t | No direct depth measurement; vulnerable to weak texture, lighting, materials, and monocular ambiguity | 2D–3D correspondences + PnP, templates, direct regression, and render-and-compare |
| Depth / Point cloud | Metric 3D geometry of visible surfaces | Scale, surface distances, normals, and evidence for 3D–3D alignment | No color semantics; transparent, specular, and black materials can produce holes and outlying points | Kabsch / Umeyama, RANSAC, global registration, and ICP / GICP |
| RGB-D | Appearance and visible surface geometry captured at the same time | Complementary evidence for checking monocular depth, point-level fusion, and geometric refinement | Alignment is not automatic; depth scale, RGB–D extrinsics, and time synchronization may be incorrect | Early / intermediate / late fusion, point-level voting, PnP + depth checks, and ICP refinement |
| Multi-view / Video | Cross-view parallax, temporal continuity, and progressively revealed surfaces | Resolve single-frame occlusion, triangulate structure, and jointly optimize camera and object states | Requires data association, camera motion estimates, synchronization, and drift management | Tracking, triangulation, bundle adjustment, multiview fusion, and object-level SLAM |
DEEP DIVE
Topic 09 explores modality fusion in depth
It covers RGB-only / depth-only / RGB-D, point clouds and normals, early / intermediate / late fusion, RGB–depth alignment, real sensor noise, transparent and reflective objects, missing modalities, multiple views, and geometric refinement.
View the knowledge map →From Observation to Action
A minimal pipeline: find, gather evidence, solve, validate, and act
Networks may combine stages, but each system responsibility remains. Separate permitted inputs, hypothesis generation, independent validation, and task acceptance to locate failures.
- 01
FIND
Discover targets and identities
Use detection / segmentation / supplied ROIs; establish object IDs, instance IDs, visible masks, and crop transforms. - 02
EVIDENCE
Predict geometric evidence
Predict keypoints, dense object coordinates, NOCS, depth point clouds, local features, template viewpoints, or pose distributions directly. - 03
SOLVE
Generate one or more hypotheses
Use PnP-RANSAC, 3D–3D alignment, voting, retrieval, regression, or global sampling; check positive depth and degeneracies. - 04
VERIFY
Validate and refine with independent evidence
Compare silhouettes, RGB features, rendered depth, normals, and temporal evidence; manage symmetry, multiple hypotheses, rejection, and relocalization. - 05
ACT
Transform into downstream frames
AR projects into the image; SLAM incorporates poses into maps; robotics composes transforms into base / tool frames. Grasping additionally checks TCP, IK, collisions, and closed-loop error.
SYSTEM CHAPTER
Explore the full pose estimation pipeline
Explore task specifications, an eight-stage architecture, intermediate representations, multiple hypotheses, validation, refinement basins, confidence, tracking state machines, evaluation, and debugging by stage.
Open topic 08 →Ambiguity, Failure and Sanity Checks
An R,t prediction may be only one of several plausible poses
Pose estimation is challenged by geometric non-identifiability and by sensor, asset, or protocol errors. Reliable systems retain multiple hypotheses, support explicit rejection, and independently check projections, depth, and transform chains.
| Problem | Why it happens | First response |
|---|---|---|
| Symmetry and repetitive appearance | Multiple rotations produce identical or similar observations; a single canonical label may create artificial conflicts | Predict equivalence classes / multiple hypotheses and use symmetry-aware training and evaluation |
| Occlusion and truncation | Only part of the surface is visible, leaving incomplete keypoints, silhouettes, and depth evidence | Estimate visibility, retain top-K hypotheses, and gather further evidence over time or through active viewpoints |
| Weak texture and domain gaps | RGB correspondences are unstable; synthetic renderings differ from real materials / lighting | Combine silhouettes, geometry, frozen features, and domain randomization, then independently validate hypotheses |
| Scale, units, and origins | Incorrect m/mm units, CAD dimensions, or object origins can make rotation appear correct while translation / projection fails | Record frames, units, and asset versions in the data specification, then check projections |
| Depth failures | Transparent, specular, and black surfaces, multipath, and mixed edge pixels cause systematic missing depth | Keep validity masks, train for missing modalities, and never interpret zero depth as a real surface |
| Comparing incompatible tasks | Results using ground-truth crops, known IDs, predicted detections, CAD-based methods, and CAD-free methods are mixed together | Specify the task conditions first, then compare detection recall, pose accuracy, and runtime costs |
MINIMUM DEBUG PROTOCOL
Check a predicted pose in this order
- 01
Read the specification: Is it CTO or OTC? Column or row vectors? m or mm?
- 02
Check rotation: RᵀR ≈ I and det(R) ≈ +1. Do quaternion order, Euler axis order, and angle units match?
- 03
Check depth: Are the object origin and visible model points in front of the camera? Is the z component of t consistent with the scene's distance scale?
- 04
Check projection: Project axes, 3D boxes, silhouettes, and depth using the current K, distortion, cropped-image coordinates, and the same CAD model.
- 05
Check equivalence: Could an apparent rotation error be caused by object symmetry, a different canonical frame, or a different object-origin version?
- 06
Check the task: Is the endpoint benchmark acceptance, visual AR alignment, robot reachability, or grasp success? Different tasks tolerate different error directions.
One Geometry · Four Communities
3D vision, SLAM, robotics, and grasping share a transform language
Fields may use different names and output directions, but all describe rigid relationships between frames. Understanding these connections is more reliable than memorizing an API's default format.
Recover 3D object–camera relationships from pixels
Camera models, correspondences, PnP, point cloud registration, symmetry, and evaluation form the core. Pose provides a geometric interface for recognition, reconstruction, AR, and measurement.
Coordinate frames → Rotations → Cameras → PnP → Registration → PipelinePut camera and object poses in the same graph
SLAM typically estimates camera poses in the world, while object SLAM also estimates object landmarks. Inversion and transform composition connect both within the same geometry.
Rigid geometry → Cameras → Multiview → Bundle adjustment → Object SLAMTransform camera-frame estimates into base / tool frames
Visual estimates become executable robot targets through camera extrinsics, hand–eye calibration, time synchronization, and the kinematic chain.
Transform chains → Extrinsics → Hand–eye calibration → Error propagation → Visual servoingObject pose and grasp pose serve different roles
Grasping also requires an object-to-grasp transform, gripper TCP, collision and reachability checks, symmetry-equivalent grasps, and closed-loop relocalization. Evaluate task success alongside ADD.
Object pose → Grasp frame → IK / collision → Closed-loop manipulationHow to Continue
Follow prerequisites and choose your own pace
Start with rigid geometry and coordinate conventions, then study rotations, cameras, and solvers. Choose SLAM, modalities, data, robotics, or research frontiers according to your application.
01 · NEXT
Mathematics & Rigid Geometry
Vectors, bases, homogeneous coordinates, SO(3), SE(3), composition, inverses, and local increments.
02Coordinate Frames & Transform Conventions
Object / camera / world / robot / gripper frames and axis conventions across communities.
03Rotation & Orientation Representations
Euler angles, matrices, axis–angle, quaternions, continuous representations, and viewpoint-dependent rotations.
04Camera Models & Imaging Geometry
Intrinsics, extrinsics, perspective projection, distortion, calibration, cropping / resizing, and depth back-projection.
062D–3D Correspondences & PnP
Recover object-to-camera transforms from pixel–model correspondences and understand RANSAC and degeneracies.
073D–3D Alignment & Registration
Kabsch, Umeyama, RANSAC, global registration, and ICP convergence basins.
08The General Pose Estimation Pipeline
Connect detection, evidence, hypotheses, validation, refinement, confidence, and tracking.
Primary Sources
Primary sources for definitions and task settings
01
BOP · Benchmark for 6D Object Pose Estimation, ECCV 2018A foundational benchmark unifying data, models, tasks, and evaluation, with explicit object-model-to-camera rigid transforms
↗ 02 BOP Challenge 2019 · Task DefinitionDistinguishes localization with known target presence from detection that must discover targets in images
↗ 03 NOCS · Category-Level 6D Pose and Size, CVPR 2019Uses a category-shared normalized object coordinate space for unseen instances, jointly recovering pose and size
↗ 04 MegaPose · 6D Pose Estimation of Novel Objects, CoRL 2022Handles objects absent from training using CAD onboarding, coarse poses, and render-and-compare refinement
↗ 05 FoundationPose · Unified Estimation and Tracking, CVPR 2024Unifies model-based / model-free pose estimation and tracking of novel objects
↗ Next topic · 01
Explore the full topic → Mathematics & Rigid Geometry
Formalize this chapter's intuition using vectors, matrices, SO(3), SE(3), and homogeneous transforms.