Object Pose Estimation · 2025 · ICRA · CCF B

High-Precision Object Pose Estimation Using Visual-Tactile Information for Dynamic Interactions in Robotic Grasping

A particle filter fuses visual and tactile displacements: vision tracks the contacted object in real time, while touch corrects slippage and pose changes of the held object. The abstract evaluates dynamic adaptation on 13 objects of different geometries but gives no cm / degree errors.

Paper

Research area

Object Pose Estimation

Task level

Instance-level

Object setting

Seen objects

Input modalities

visiontactile displacementrobot motion

Requirements

initial object posetactile displacement sensingknown hand-object transform conventionparticle-filter state model

Full paper metadata Authors, affiliations, research area, datasets, metrics, method tags, and reading status Expand

Authors: Zicai Peng, Te Cui, Guangyan Chen, Haoyang Lu, Yi Yang, Yufeng Yue
Affiliations: Beijing Institute of Technology
Conference / journal: Conference · ICRA · CCF B
Research area: Object Pose Estimation
Code availability: Unavailable / not yet recorded
Reading status: Overview:Available Deep Dive:Available
Task setting: Instance-level · Seen objects
Datasets: 13-object dynamic interaction test
Evaluation metrics: Translation Error Rotation Error
Method tags: Instance-level Pose Estimation Pose Refinement Multi-object Pose Estimation Absolute Pose Estimation
Requirements: initial object posetactile displacement sensingknown hand-object transform conventionparticle-filter state model

Reading Notes

Overview / Deep Dive

Available src/paper-notes/high-precision-visuotactile-icra2025/coarse.md

Paper overview notes

Quickly assess the paper's scope, problem setting, core method, experimental findings, and whether to read it in depth.

Paper Reading Notes (Coarse): High-Precision Object Pose Estimation Using Visual-Tactile Information for Dynamic Interactions in Robotic Grasping

1. Basic information

Authors: Zicai Peng, Te Cui, Guangyan Chen, Haoyang Lu, Yi Yang, Yufeng Yue
Venue / year: IEEE ICRA 2025
Affiliation: Beijing Institute of Technology
Paper: DOI
Task: object pose tracking during dynamic robot interactions, covering contacted-object motion and in-hand slip of grasped objects
Experimental scope: 13 objects with different geometries
Availability: full text is not open access; no public author manuscript, code, or data was found

2. One-sentence summary

The method fuses vision and tactile displacement with a particle filter. Vision tracks contacted objects in the environment in real time, while tactile displacement corrects slip-induced pose changes of the grasped object. The authors validate dynamic adaptability on 13 objects, but the abstract provides no translation / rotation errors.

3. Main method

Static grasping assumes a fixed $\mathbf T_{hand\rightarrow object}$ , but real interactions involve:

In-hand slip caused by changing grasp force;
A grasped object pushing an environmental object;
Visual occlusion by the robot hand;
Abrupt motion-model changes at contact.

A particle filter maintains a pose belief with multiple hypotheses:

Predict particles using a motion model;
Update the contacted object using vision;
Update the grasped-object increment using tactile displacement;
Reweight / resample to obtain the current pose.

Visual observations and tactile displacement enter a particle filter for dynamic object pose estimation — Redrawn by this site from the authors' public abstract; not an original paper figure. Particle states, likelihoods, and sensor models are undisclosed; the diagram shows only verifiable fusion relationships. Source: paper DOI.

4. Evidence boundaries

The abstract confirms 13 geometrically diverse objects, dynamic scenes, compelled motion, and a high-precision pose conclusion, but gives no cm / degree errors, update rates, baselines, or task success rates.

Public conclusions and undisclosed accuracy metrics for dynamic visuotactile pose — Compiled by this site from the authors' public abstract. Without numerical evidence, “high precision” is not plotted as a millimeter-level result. Source: paper DOI.

5. Coarse-reading assessment

The key value is separating two object roles: vision can supply global pose for an environmental contacted object, while tactile displacement supplies local changes for an occluded grasped object. Particle filtering suits noise, multiple hypotheses, and different update rates.

Limitations include dependence on initial pose, tactile-to-hand calibration, and motion models. Tactile sensing supplies local displacement rather than an inherent global pose, and long-term integration drifts. Without public numbers, millimeter-level accuracy for assembly cannot currently be established.

Available src/paper-notes/high-precision-visuotactile-icra2025/deep.md

In-depth paper notes

Support reproduction, comparison, related-work writing, and identifying opportunities for further research.

Paper Reading Notes (Deep): High-Precision Object Pose Estimation Using Visual-Tactile Information for Dynamic Interactions in Robotic Grasping

1. Research problem

Most grasp-pose pipelines assume a rigid object–gripper attachment after grasping. During assembly, insertion, and pushing, contact forces can cause grasped-object slip and move environmental objects. Initial visual poses then become outdated, while mutual hand-object occlusion hinders continuous visual observation.

This work combines global visual information with local post-contact tactile-displacement increments and maintains a dynamic pose using a particle filter.

2. Two object roles

Contacted object

An environmental object contacted and pushed by the robot / grasped object. It often remains visible to an external camera, allowing real-time visual pose tracking, but contact invalidates the static-motion assumption.

Grasped object

The object held in the gripper. The hand may occlude it, and slip changes its hand-relative transform. Tactile displacement directly observes contact-surface changes and helps estimate pose increments.

Separating these roles avoids indiscriminately fusing all tactile information into one observation.

3. Particle filter

Let the state be the object pose $\mathbf x_t$ . The particle-filter recursion is:

p(\mathbf x_t\mid \mathbf z_{1:t}) \propto p(\mathbf z_t\mid \mathbf x_t) \int p(\mathbf x_t\mid \mathbf x_{t-1}) p(\mathbf x_{t-1}\mid \mathbf z_{1:t-1}) d\mathbf x_{t-1}.

Particles $\{\mathbf x_t^{(i)},w_t^{(i)}\}$ first propagate through the motion model, then are reweighted by visual / tactile likelihoods. This is a standard filtering interpretation, not a disclosed implementation from the paper.

Vision suits absolute correction; tactile displacement more closely represents relative motion:

\Delta \mathbf T_t^{touch} \approx \mathbf T_{t-1}^{-1}\mathbf T_t.

Combining them can reduce drift from tactile-only integration and maintain estimates through short visual occlusions.

Visuotactile particle filtering for dynamic contacted and grasped object states — Redrawn by this site from the authors' abstract; not an original paper figure. Particle count, state parameterization, and likelihoods are undisclosed and are not guessed here. Source: paper DOI.

4. Experimental evidence

The abstract discloses:

13 objects with different geometries;
Dynamic interactions and compelled object motion;
A claim of high-precision pose estimation;
Adaptability to uncertain practical scenarios.

Undisclosed items include:

translation / rotation error;
Visual / tactile sensors;
Particle count and update rate;
vision-only / tactile-only baseline;
Slip magnitude and occlusion ratio;
grasp / insertion success.

Public conclusions from 13-object experiments and missing quantitative accuracy — Compiled by this site from the public abstract. “High-Precision” in the title cannot replace quantitative errors; this page therefore claims neither millimeter accuracy nor a specific angular error. Source: paper DOI.

5. Technical challenges

Coordinate frames: tactile displacement must be transformed from sensor to hand / object coordinates;
Observability: a single contact patch is insensitive to some rotations or tangential motions;
Contact transitions: stick / slip / detach changes the measurement model;
Visual occlusion: the grasped object may remain invisible, preventing absolute drift correction;
Symmetry: both vision and touch may admit multiple solutions;
Asynchrony: different camera / tactile update rates make timestamp errors appear as motion.

6. Strengths and limitations

Strengths:

Addresses dynamic interactions beyond static pre-grasp pose;
Clear roles for both modalities;
Particle beliefs express multiple hypotheses;
13 objects provide some geometric diversity.

Limitations:

Requires initial pose and sensor calibration;
Tactile displacement gives local increments and drifts over time;
Particle-filter computation grows with state dimension and particle count;
Without public numbers or code, assembly accuracy / frequency requirements cannot be assessed.

7. Reproducibility checklist

Once the full text becomes public, document the robot / hand / tactile sensor, visual tracker, state and motion model, tactile-displacement definition, measurement likelihood, resampling, particle count, synchronization, 13-object list, errors / task success, ablations, and runtime.

8. Research implications

Future work could use contact-mode switching filters with different dynamics for stick / slide / roll; combine low-frequency absolute visual correction with high-frequency relative tactile updates; analyze SE(3) observable directions from contact geometry and tighten covariance only in observable subspaces.

This paper reflects the shift from pre-grasp perception to state estimation during interaction. The problem matters, but the public abstract cannot establish the actual accuracy level; method value and numerical evidence are therefore kept distinct.

Related papers

2026 · CVPR

3PT

3PT unifies detection, segmentation, and 6-DoF pose with two Transformers for 2D / 3D object-centric understanding. Continuous refinement of object representations and rendered-feature alignment support zero-shot cross-object perception on BOP-H3, BOP-Industrial, and BOP-Classic.

2026 · AAAI

6DAttack

Studies backdoor attacks on PnP-based and end-to-end 6-DoF pose pipelines. Structurally 3D triggers and modified pose labels are injected into training data so that models behave normally on clean inputs but produce controlled incorrect poses when triggered.

2026 · ICRA

Active Tactile Exploration

A violation-implicit loss incorporates physical-constraint violations into joint estimation of freely moving unknown convex rigid-body trajectories and cuboid / 20-vertex polytope shapes. Observed / Fisher information at a local MLE estimates EIG to select the next touch. In simulation, the EIG strategy yields stable errors below 4 mm after 5 cuboid actions; real Trifinger results at step 5 substantially outperform random selection. Convex geometry, local optima, multiple initializations, and tactile / proprioceptive errors limit the method.