Object
Metric meshes, part identities, proper groups, axes/offsets, instance differences
Symmetry, Ambiguity & Uncertainty
A symmetric object's pose is an equivalence class under a proper symmetry group, rather than one arbitrary matrix. Occlusion, projection, and sensor failures add alternatives beyond global object symmetry. Separate these sources before choosing losses, distributions, confidence, and robot actions.
01 · Problem Contract
Symmetry is not a constant derived solely from shape. Instances, modalities, task costs, and evaluation tolerances change equivalence relations; give each layer independent fields and versions.
Metric meshes, part identities, proper groups, axes/offsets, instance differences
Whether texture, text, materials, spectrum, and illumination break geometric symmetry
Modes supported by visible surfaces, occlusion, projection, noise, and modality validity
Pose changes with equal grasping, insertion, placement, or control costs
Losses, metrics, correctness thresholds, confidence, calibration splits
02 · Five Distinct Layers
Calling every pair of plausible poses symmetric is a common error. These five layers differ in scope, evidence, and handling.
03 · Pose as an Equivalence Class
Even after fixing the object frame, multiple object-to-camera transforms may align every semantically labeled surface. They are representatives of one pose.
A numeric label can select a canonical representative without making it truer than other orbit elements. Output the symmetry rule/object version with it.
Mapping C₄ yaw to [−45°,45°) jumps at ±45°. Tiny input changes can demand labels separated by 90°.
An ideal cylinder's axial yaw is gauge. Choosing yaw=0 is a convention; low variance without identifying cues expresses false certainty.
Quaternions lie on S³ and map 2-to-1 to SO(3). Handle antipodal equivalence before adding object-specific symmetry.
04 · Symmetry Groups & Metadata
Discrete, continuous, and approximate symmetries require different numerical handling. An off-origin symmetry axis necessarily couples rotation and translation.
| Family | Structure | Pose orbit | Engineering notes |
|---|---|---|---|
| Trivial {I} | Asymmetric objects | One local representative | Occlusion, projection, or weak evidence can still create observational alternatives |
| Cₙ | n discrete rotations around one axis | n representatives | Gears, square prisms; appearance or tasks may reduce n |
| Dₙ proper part | n principal-axis rotations + n perpendicular half-turns | 2n proper rotations | Do not insert planar reflection matrices directly into SO(3) |
| T / O / I | Regular tetrahedron, cube/octahedron, icosahedron/dodecahedron | 12 / 24 / 60 proper rotations | Finite noncommutative groups, beyond one yaw period |
| SO(2) | Continuous rotation around a fixed object axis | One continuous orbit | Axial surfaces of revolution; rotation about the axis is gauge |
| SO(3) | Ideal spheres or fully isotropic tasks | Full rotation orbit | Rotation is unidentifiable, while position may remain meaningful |
| Approximate / partial symmetry | Transformed surfaces are close but not identical | Depends on tolerance and observation resolution | State tolerance, sensor, and task; do not claim an exact group |
05 · Observation-Conditioned Ambiguity
Identifiability depends on the observation model and evidence. More similarly distributed training data cannot recover information hidden by occlusion, projection collapse, or sensor loss.
| Source | Why alternatives arise | Diagnostic evidence | Handling |
|---|---|---|---|
| Self-occlusion / external occlusion | Hidden asymmetric parts let different poses render the same visible region | Visible masks, amodal support, intermode render residuals | Change views, retain modes across frames, output top-K |
| Projection and silhouettes | 3D differences collapse along viewing rays; frontal, weak-perspective, or nearly planar structures yield alternatives | Multiview baseline, depth/normals, oblique silhouettes | Add oblique views, metric depth, or structural priors |
| Repeated parts / local patches | Local descriptors match multiple locations in periodic structures | Correspondence-vote mode counts, part identity | Group-aware matching, global verification, multimodel RANSAC |
| Weak texture / lighting changes | Blur, overexposure, shadows, or low resolution remove unique appearance cues | cue visibility, MTF / exposure, appearance ablation | Improve capture or explicitly use a geometric subgroup |
| Mirrors / transparency / depth holes | Mismatched observation models make intensity/depth unreliable surface evidence | sensor confidence, multi-view consistency, OOD score | Sensor fallback, polarization/touch/silhouettes, rejection |
| Cropping / truncation | Key parts leave the image; scale and center compensate for one another | Full bboxes, truncation, proposal-boundary stress tests | Widen ROIs, use full-image context, multiple frames, or active fallback |
| Category and instance variation | Canonical frames, proportions, or local structures differ across instances | instance metadata, shape posterior, scale residual | Joint shape–scale–pose distributions; label disagreement is not simply noise |
| Annotation / canonical conflicts | Annotators or datasets choose different orbit representatives | quotient-aware inter-annotator distance | Save equivalence sets/rule versions and marginalize during training |
Yes → quotient pose; new views cannot break true symmetry.
No → view-induced modes; inspect occlusion, truncation, and validity.
No → retain the posterior; avoid forcing an average pose.
Different → acquire / top-K / abstain; equal → action quotient.
06 · Supervision, Loss & Matching
Parameter regression, surface matching, correspondences, and probability densities must specify finite/continuous groups, visible ambiguity, and representation double cover.
Minimize geodesic / parameter-consistent losses over finite symmetry representatives
Compare nearest surfaces or apply explicit symmetry transforms first
Treat equivalent surface points as one target set, or predict fragments/regions before composition
Sum or integrate over equivalent ground truth rather than rewarding one representative
Cover the ground-truth orbit with at least one hypothesis; train mode scores/diversity
Expected downstream costs, worst-case modes, or chance constraints under the posterior
07 · Interactive Lab
The first experiment changes geometric/appearance/task equivalence. The second creates modes through occlusion of an asymmetric object. The third compares top-K, NLL, entropy, epistemic signals, and calibration.
Interactive · Quotient Pose, Ambiguity & Calibration
Symmetry group
Naïve angle error
Quotient-pose error
Equivalence verdict
Canonicalization risk
Ambiguity type
Supported modes
Observation entropy
Identifiability
Next evidence
Posterior family
Symmetry-aware NLL
Top-K coverage
Calibration proxy
System decision
08 · Pose Distributions
Choose distributions/interfaces after distinguishing small unimodal errors, finite separated modes, and continuous symmetry orbits.
| Output | Representation | Suitable use | Limits |
|---|---|---|---|
| Point + score | One transform and a potentially unnormalized score | Low latency and simple interfaces | Cannot express distant modes; scores are not inherently success probabilities |
| Top-K hypotheses | K transforms, weights, and equivalence deduplication rules | Suitable for PnP / templates / tracking / verification | Report K, coverage, ranking, and downstream costs |
| Local mean + 6×6 covariance | A local ellipsoid with specified left/right perturbations, frame, and tangent point | Unimodal small errors, filtering, optimization | Cannot span distant symmetry modes; scale rotation/translation units appropriately |
| Bingham on S³ | antipodal-symmetric quaternion orientation distribution | Respects q/−q and models directional axes/anisotropy | One Bingham does not represent an arbitrary multimodal SE(3) posterior |
| Matrix Fisher on SO(3) | Exponential family defined directly on rotation matrices | No coordinate singularities; structured local uncertainty | A finite parameter family cannot represent every complex/continuous multimodal density |
| Mixture / particles | Multiple local distributions or weighted samples | Explicitly retains symmetry, occlusion, and association modes | Protocols must specify component alignment, deduplication, normalization, and compute budgets |
| Implicit density / flow | Queryable relative / normalized density for any R or T | Arbitrary SO(3) modes and higher-order structure | More complex sampling, normalization, training measures, and calibration |
Conditional spread for occluded, blurred, or weak-return samples under one model
Additional views/better sensors may reduce it; retraining alone may notensemble / posterior disagreement, density support, OOD stress test
Add relevant data/models/priors; one softmax entropy does not identify this aloneObject symmetry group and quotient orbit
Merge into equivalence classes instead of counting as task risk/errorMultiview / modality likelihoods and mode separability
Propagate modes, sense actively, or reject; avoid Euclidean averagingannotation provenance, round trip, inter-annotator quotient distance
Fix data contracts rather than teaching models to absorb the errorexpected cost, chance constraint, collision / grasp simulation
Can disagree with pose entropy; compute for the specific task09 · Confidence & Calibration
Pose scores, detector scores, render residuals, densities, and robot success probabilities are distinct. Bind calibration to symmetry-aware events, domains, and versions.
| Tool | Measurement / procedure | Reporting and limits |
|---|---|---|
| Correctness event | Define a symmetry-aware threshold event, such as MSSD<τ, task success, or top-K orbit coverage | Different events need separate calibration; one confidence cannot support every claim |
| Reliability diagram / ECE | Bin confidence and compare empirical correctness; ECE is a weighted bin gap | Report bins, counts, and class/visibility strata; one ECE hides local miscalibration |
| NLL / proper scoring | Rewards probability assigned to the correct equivalence set and penalizes false confidence | Sum finite groups; continuous groups need a relative measure/tolerance or quotient density |
| Brier score | Compare probability with outcome for an explicit binary success event | Reveals overconfidence beyond accuracy, but still depends on the event definition |
| Coverage / set size | Does a prediction set / top-K contain equivalent ground truth, and how large is it? | Report coverage with K, angular/surface thresholds, and symmetry rules |
| Risk–coverage / selective pose | Reject low-confidence samples and plot retained coverage versus risk | Choose deployment thresholds by failure cost and capacity, beyond the most attractive operating point |
| Temperature scaling | Scale logits on validation data, usually preserving ranking | Can reduce global over/underconfidence, but cannot create missing modes or fix OOD/symmetry metadata |
| Shift / OOD audit | Retest calibration by instance, scene, occlusion, sensor, and time | In-distribution calibration does not guarantee deployment reliability; version it and trigger fallback |
Every Deployed Confidence Field Should Carry
10 · Temporal Inference & Robotics
Track in quotient space while retaining necessary mode identity. Robot decisions then merge, verify, or reject modes using contact, collision, reachability, and safety costs.
Gate using the minimum distance from track predictions to equivalent representatives, avoiding false jumps for one physical pose.
For continuous display/control, select the representative nearest the previous state while retaining quotient pose. Unwrapping chooses temporal gauge, not recovered truth.
Under occlusion, preserve weighted modes rather than resetting to an average pose; merge/prune when identifying cues return.
Motion supplies parallax/new surfaces to resolve view-induced modes, but cannot break true global geometric-plus-appearance symmetry.
EKF / factor-graph tangent covariances need left/right perturbations, reference frames, rotation/translation scaling, and relinearization rules.
Identity, pose modes, and occlusion are coupled; fixing the wrong identity before pose filtering produces confident wrong tracks.
| Task | Visual facts | Task equivalence | Decision interface |
|---|---|---|---|
| Parallel-jaw grasping | The object may have C₄ symmetry | Jaw contacts are usually equivalent only under 180°; collisions/cables may break that equivalence | Map pose modes to grasp sets and merge action-equivalent modes by collision/force closure |
| Suction grasping a disc | Continuous unidentifiability of yaw about the normal | Central suction is approximately insensitive to yaw | Estimate position and surface normal, declare yaw gauge, and avoid fabricated angular precision |
| Insertion / assembly | Approximately axially symmetric housing | Notches, keys, or pins make final roll unique with tight tolerances | Task symmetry is smaller than visual geometry; inspect key features actively or search by touch |
| Placement / orientation | Multiple poses explain the current RGB-D | Centers of mass, supports, labels, and reachability induce different costs | Evaluate stable placement / reachability per mode; output expected risk and safe fallback |
| Visual servoing | Canonical representatives can jump across cells | Controllers need continuous errors; jumps can trigger large actions | Compute quotient-space errors and unwrap temporally; include velocity limits and mode switching in the state machine |
| Active sensing | Two modes differ only in handle visibility | Reduce collision / grasp risk within limited time | Choose reachable, collision-free views maximizing expected intermode observation differences, beyond visible-pixel count |
11 · Failure Modes & Release Gates
These checks cover group validity, metadata, canonical discontinuities, distributions, confidence semantics, and leakage.
| Failure modes | Symptoms | Detection | Fix |
|---|---|---|---|
| Treating q/−q as object symmetry | Artificial bimodal labels or quaternion-loss jumps for arbitrary objects | Check R(q)=R(−q), then audit object symmetry separately | Use |qᵀq̂| or rotation geodesics for the double cover |
| Encoding reflections as rotations | det(R)=−1 and flipped handedness; meshes may overlap but the transform is outside SO(3) | Assert RᵀR=I and det(R)=+1 for every symmetry matrix | Retain proper rotations; model mirrored-instance semantics separately |
| Missing axis offsets | The model drifts when rotated about an off-origin axis | Check S(M)=M on sampled surfaces and t=(I−R)o | Store full 4×4 transforms or axis+point |
| Excessive canonicalization | Labels/predictions jump 90° / 180° between representatives near boundaries | Perturb inputs around cell boundaries and plot output continuity | Marginalize/minimize over groups during training; expose orbit rules at the interface |
| Averaging modes | Arithmetic averages of valid rotations can give a wrong pose or nonrotation matrix | Check whether the mean lies in a high-probability mode / SO(3) | Output mixtures/top-K; define the decision meaning of any Fréchet mean |
| Incorrect symmetry groups | Models/evaluation ignore stickers, notches, or instance differences | Audit geometry, appearance, instance, and task layers separately | Version subgroups/evidence and manually review approximate symmetries |
| Substituting entropy for epistemic uncertainty | Symmetric samples are labeled OOD while incorrect sharp modes appear reliable | Combine symmetry, OOD, false-confidence, and weak-evidence stress sets | Model multiplicity, aleatoric/epistemic uncertainty, and calibration separately |
| Reporting oracle top-K alone | Ground truth is inside K, but actual ranking sends the robot a wrong pose | Report top-1, top-K, NLL, score calibration, and downstream selection together | Train comparable weights and validate the verifier |
| One covariance spanning multiple modes | Ellipsoids cross low-probability regions and updates converge to nonexistent intermediate poses | Plot mode separation and tangent validity radii | Use per-mode covariance with mixture weights / particles |
| Inconsistent calibration events | A rotation-threshold score is interpreted as grasp success probability | Register target event, threshold, split, and version for each score | Recalibrate for the task event or report only the original meaning |
| Coarse continuous-group discretization | Metrics/NLL change strongly with symmetry sample counts | Refine sampling and check maximum surface displacement and convergence | Minimize analytically or choose justified steps from diameter/threshold |
| Evaluation leakage | Tuning temperatures, rejection thresholds, or symmetry tolerances on test data | Audit calibration / validation / test provenance | Fix the protocol and fit on independent calibration splits |
Release Gate · Before Outputting Pose and Confidence