Deep Reading Notes: Active Tactile Exploration

1. Key assessment

• Paper: arXiv 2510.13595
• Project and code: Active Tactile Exploration
• Core problem: Touch-only identification of unknown objects provides sparse local observations while contact moves the object. Shape and the entire pose trajectory must be estimated together, with active selection of the next informative contact.
• Core approach: Violation-implicit loss avoids poor rigid-contact gradients. Observed / Fisher information around the local MLE approximates EIG, and CEM selects candidate two-finger approaches.
• My assessment: System identification and active perception form a complete loop, recovering constrained convex shapes from 5–8 seconds of tactile data. However, local continuous parameters and local Gaussian information do not automatically represent multiple global pose / shape hypotheses.

2. Problem setting

The scene contains:

• A known ground plane;
• One unknown, freely moving convex rigid object;
• Convex robot end effectors with known geometry and poses;
• Tactile contact boolean $c_t$ and measured contact normal $\hat n_{t,m}$;
• Robot pose / proprioception $r_t$.

The goal is to estimate current object pose $x_T$ and geometry parameters $\theta$ with as few actions as possible. Cuboids have three positive side-length parameters; general convex polytopes are convex hulls of 20 vertices. Evaluation transforms the estimated surface into the current pose and computes bidirectional Chamfer distance (bCH) to the ground-truth surface, jointly reflecting pose and shape error rather than separate pose / size components.

FoundationPose and RealSense supply evaluation references only, outside the tactile-only estimator. The estimator uses no vision, but the real-world bCH reference still relies on visual pose.

3. Tactile Likelihood

The signed robot–object distance is $\phi_t(\theta,x_t,r_t)$. Contact booleans use a logistic likelihood:

$$p(c_t|\theta,x_t,r_t)=\frac{c_t+(1-c_t)e^{\alpha\phi_t}}{1+e^{\alpha\phi_t}}.$$

When contact occurs, a Gaussian likelihood matches measured normal $\hat n_{t,m}$ to geometrically predicted normal $\hat n_t$. The combined negative log-likelihood $L_t$ penalizes:

• Inconsistency between contact / noncontact and signed distance;
• Inconsistency between measured and predicted normals.

Direct optimization under the rigid-dynamics constraint

$$x_{t+1}=f_\theta(x_t,r_t,\lambda_t)$$

encounters large flat regions and abrupt gradients at contact switches. A graph-distance analogy with a one-dimensional step function motivates finding a nearby point on the function graph instead of comparing values only at the same horizontal coordinate.

4. Violation-implicit loss

The authors treat the entire $x_{0:T}$ trajectory as decision variables and penalize dynamics residuals and physical-constraint violations:

$$\mathcal L_v=\sum_t\min_{\lambda_t\in FC} \left[ L_t+\|x_{t+1}-f_\theta(x_t,\lambda_t,r_t)\|_2^2+ g_\theta(x_t,\lambda_t,r_t) \right].$$

$g_\theta$ includes:

1. Complementarity: $\lambda_t^n\phi_t$, allowing normal force only during contact;
2. Maximum power dissipation: Friction opposes tangential sliding;
3. Nonpenetration: $\min(0,\phi_t)^2$;
4. Inelasticity: Normal impulses cannot return energy and create unforced rebounds.

Inner optimization of $\lambda_t$ is a QP with second-order-cone friction constraints, solved through CVXPYLayers. By the envelope theorem, outer backpropagation need not explicitly differentiate through optimal $\lambda_t$. Replacing hard physical feasibility with soft penalties can produce nonphysical local minima; multiple trajectory initializations mitigate this.

5. From Local MLE to EIG

Let $\omega=\{\theta,x_T\}$. Instead of maintaining a full belief, the method uses a Laplace-like local approximation from the negative-log-likelihood Hessian at MLE $\tilde\omega$:

$$H[\omega|D]\approx \frac12\log\det I(D,\tilde\omega)+c,$$

The observed information is

$$I(D_{\rm obs})\approx \nabla_\omega^2\sum_tL_t$$

The implementation omits the Hessian-of-logit term, whose expectation is zero although individual samples need not be, retaining the gradient outer-product component.

For future action $r_{T:H}$, simulation from the current estimate produces a future trajectory and ideal contacts / normals, from which Fisher information $F(D_{\rm acq})$ is computed. Finally:

$$\operatorname{EIG}\propto \log\det\!\left(F(D_{\rm acq})I(D_{\rm obs})^{-1}+I\right).$$

This measures expected future information relative to information already available. Repeated contacts with confident surfaces may yield large $F$, but little gain after normalization by $I(D_{\rm obs})^{-1}$. Unobserved surfaces are more informative.

6. A Critical Approximation: Historical-State Sensitivity

EIG concerns $\omega=\{\theta,x_T\}$, while past observations $L_t$ depend on historical $x_t$. This theoretically requires backward sensitivities, analogous to $x_{T-1}=f^{-1}(x_T)$. Rigid contact is many-to-one: A stationary state may have always been stationary or just stopped after collision, so inverse dynamics is nonunique.

The implementation differentiates each time-step gradient only with respect to its own state and sums them, effectively assuming dynamics Jacobian $dx_t/dx_{t-1}=I$. This suffices for the sliding objects tested, but EIG may become inaccurate under tumbling, bouncing, or major contact-mode changes. The approximation is a central theoretical limit on extending mild sliding to complex manipulation.

7. Action Space and Hardware

Each end effector executes a 1-second straight approach from a fixed radius toward the estimated object centroid. Right ascension and declination parameterize each finger’s action. Bounds are $[-\pi/4,\pi/4]$ on hardware and $[-2\pi/5,2\pi/5]$ in simulation.

Iterative Gaussian CEM begins with uniformly sampled actions, ranks EIG, fits a Gaussian to the top $n_{\rm best}$, and randomly chooses from the final elite set. Search is restricted to a manually defined 4D two-finger approach family, rather than global optimal control over arbitrary robot trajectories.

DenseTact produces images at 30 Hz. Optical flow separates normal, shear, and torsional motion, followed by Helmholtz–Hodge decomposition. A weighted center of the irrotational scalar potential gives center of pressure; projection onto the nominal surface yields normals, and a local-potential threshold detects contact.

8. Shape Parameterization

8.1 Cuboid

$\theta\in\mathbb R_{>0}^3$ represents three side lengths. With so few parameters, one corner / multiface contact may substantially increase observed information. This tests joint pose–shape–contact optimization but remains far from arbitrary objects.

8.2 Convex polytope

The convex hull of 20 3D vertices defines the shape. Vertices falling inside the hull cease to affect the surface or bCH, permanently reducing representational capacity; the authors randomly replace them on the hull. A regularizer encourages at least three vertices to lie on the ground at trajectory end, preventing the violation loss from accepting unstable shapes.

Nonconvex objects could theoretically use unions of convex parts, but this is unimplemented. Complex topology, recesses, and thin structures are also unvalidated.

9. Simulation Results

Drake simulates at 1 kHz and reports observations at 30 Hz.

9.1 Cuboid

Ground-truth side lengths are sampled from 3–6 cm, with initial guesses of 1–3 cm. Both initial and true poses are randomized within 2 cm of the workspace center. After first contact, 5 actions are executed, with $N=10$.

Random exploration reaches about 5 mm bCH within 5 steps; EIG consistently falls below 4 mm. An independent two-tailed t-test at step 5 remains significant at $p<0.05$ after Bonferroni correction over five time points.

9.2 Polytope

Ground truth consists of Platonic solids with 2 cm bounding radius; initialization uses 20 random points on a 1 cm sphere. After first contact, 8 actions follow. Both methods reliably approach 5 mm within 8 steps, with lower variance and faster convergence for EIG. Random exploration takes roughly 3 additional actions for comparable 95% CIs, but differences are less statistically significant than for cuboids.

The random-polytope confidence intervals exclude one catastrophic outlier: The estimated centroid lay entirely outside the object, causing two consecutive no-contact actions before realignment. Exclusion improves readability, but local-optimization failures remain real online-system risks.

10. Real-Robot Results

The real object is a 5.8 × 4.9 × 5.3 cm cuboid weighing about 0.5 kg, moved roughly 1–2 cm during each trial. Initialization matches simulation, with 5 actions after first contact and $N=6$.

EIG significantly outperforms random exploration at step 5 (independent two-tailed t-test, $p<0.05$ after Bonferroni correction over five time points). Final bCH nevertheless remains around the centimeter scale, well above simulation. Each end effector has 3–5 mm proprioceptive error; two-finger errors, tactile normals, and visual evaluation references jointly set a lower bound.

11. Limitations and Reproduction Risks

1. Convexity: Both object and end effector must be convex to provide a unique collision point.
2. Local MLE / EIG: A confident Hessian at the wrong mode misdirects the next action.
3. Multiple-initialization costs: As data accumulates, multiple trajectory initializations slow online optimization. No complete wall-clock curve is provided.
4. Insensitivity to light touch: DenseTact struggles with light contact, restricting experiments to moderately heavy objects.
5. Identity Jacobian: Historical-to-current sensitivity is approximated and validated only under mild sliding.
6. Ground-plane prior: Objects must rest on a known plane; free-space, in-hand, and unknown-table settings require different models.
7. Centroid-dependent actions: Large pose errors can make planned straight approaches miss the object entirely.
8. Small evaluation samples: Simulation N=10 and hardware N=6, with only one real cuboid.
9. Visual-reference errors: FoundationPose is not motion-capture ground truth; quantify evaluation noise separately.

12. Code and Reproduction

The released code is the DAIR physics-based learning library’s egordon/exploration branch, with migration to a cleaner JAX version in progress. Reproduction requires locking versions and settings for:

• Drake / contact geometry;
• CVXPYLayers QP and friction-cone parameters;
• JAXopt collision normal / distance derivatives;
• tactile normal extraction;
• CEM sample / elite counts;
• Violation-penalty weights and multiple-initialization strategies.

Essential ablations include random versus EIG, removing violation terms, different MLE initializations, identity Jacobians versus differentiable forward sensitivities, and real proprioceptive perturbations.

13. Methodological Insights

Two lessons for 6D-pose research:

• Unknown-object pose and shape can be identified jointly along one trajectory through contact physics, without requiring sequential estimation.
• Active perception can use local information matrices as an engineering compromise without maintaining a full high-dimensional belief, but mode failures must remain explicit.

Combining this with visual initial models, nonconvex neural implicit shapes, global particle beliefs, and safe closed-loop control would move toward building an actionable model of an unknown object after a few exploratory contacts.