Neural Introspection Gating for Adaptive KV-Cache Reuse in Vision-Language-Action Models

Wu, Zhijie; Kawaharazuka, Kento; Okada, Kei

Neural introspection for real-time robotics

Neural Introspection Gating for Adaptive KV-Cache Reuse in Vision-Language-Action Models

Zhijie Wu Kento Kawaharazuka Kei Okada

Graduate School of Information Science and Technology, The University of Tokyo
Accepted at IROS 2026

VLA models are fast to describe and expensive to run. This work keeps the cache when the model is confident and invalidates it when the action distribution becomes uncertain, using a zero-cost logit-margin signal extracted during decoding.

Paper GitHub arXiv

Confidence gate Mean top-1 / top-2 logit margin

Reuse is trusted only when the model stays decisively ahead of its second-best action token.

LIBERO-Long 54.8%

Recovers the accuracy lost by blind caching on the hardest benchmark split.

Compute 1.54 TFLOPs

Stays well below full inference while avoiding stale cache reuse at critical moments.

Takeaway Cache reuse becomes selective, not blind.

The gate preserves the savings of VLA-Cache when frames are redundant and pays the recompute cost only when the model itself is unsure.

Problem

Observation-space similarity misses model uncertainty.

Static-looking frames can still demand a fresh forward pass when the action distribution narrows or shifts at grasp and alignment steps.

Method

Use the decoder's own margin as a gate.

The cache is invalidated when the average gap between the best and second-best action tokens falls below a threshold.

Effect

Recover accuracy without giving up the speedup.

On LIBERO-Goal and LIBERO-Long, the gated variant restores the lost accuracy while retaining most of the compute savings.

Abstract

Adaptive cache reuse with model-intrinsic confidence

Vision-Language-Action models map camera images and language instructions directly to motor commands with a single autoregressive transformer. In real-time control, however, they still spend substantial compute recomputing KV states. This paper extends VLA-Cache with a training-free neural introspection gate that monitors the mean top-1/top-2 logit margin during decoding. When the margin falls below a threshold, the cache is invalidated and the model performs a full recompute; otherwise, the standard patch-similarity and layer-wise reuse pipeline proceeds.

Evaluated on the LIBERO benchmark suite with OpenVLA and OpenVLA-OFT, the gated method improves reliability whenever blind caching hurts. On LIBERO-Goal and LIBERO-Long, it recovers the accuracy lost by VLA-Cache while keeping most of the compute savings, and it remains safe when caching is already stable.

Method

Three stages, plus one gate

Static patch detection

Consecutive images are compared patch-by-patch. Similar patches become candidates for KV reuse.

Task-relevant filtering

Patches with strong text-to-vision attention are excluded so semantically important regions are recomputed.

Entropy-adaptive reuse

Each transformer layer reuses a different share of tokens based on its attention entropy.

Logit-margin gate

If the model's average top-1/top-2 margin drops below threshold, the entire cache is discarded and recomputed from scratch.

Frame t-1 Decoder scores

→

Introspection gate Margin below threshold?

→

Frame t Reuse or recompute

The gate is zero-cost at inference time because it uses the logits already produced by the decoder.

Results

Accuracy-compute trade-off on LIBERO

OpenVLA

Gating rescues the benchmark where blind caching loses accuracy on the long-horizon suite.

Numbers shown as Success rate (%) / TFLOPs

Task	Full	VLA-Cache	Ours
Spatial	78.8 / 1.89	78.8 / 1.43	79.4 / 1.55
Object	70.4 / 1.86	69.4 / 1.44	67.8 / 1.54
Goal	77.2 / 1.83	74.0 / 1.40	77.4 / 1.50
Long	54.0 / 1.88	50.2 / 1.43	54.8 / 1.54
Avg.	70.1 / 1.87	68.1 / 1.43	69.9 / 1.53

OpenVLA-OFT

When caching is already stable, the gate keeps performance on par with the baselines and adds little overhead.

Numbers shown as Success rate (%) / TFLOPs

Task	Full	VLA-Cache	Ours
Spatial	93.8 / 4.00	93.8 / 3.11	94.0 / 3.23
Object	98.4 / 3.96	98.6 / 3.05	98.6 / 3.08
Goal	97.2 / 3.94	96.8 / 3.06	97.0 / 3.07
Long	93.8 / 3.98	94.0 / 3.05	93.0 / 3.13
Avg.	95.8 / 3.97	95.8 / 3.07	95.7 / 3.13

Bottom line: the gate is selective enough to restore accuracy on brittle episodes, but cheap enough to keep most of VLA-Cache's compute savings.

Citation

BibTeX

@article{wu2026neuralintrospection,
  title   = {Neural Introspection Gating for Adaptive KV-Cache Reuse in Vision-Language-Action Models},
  author  = {Wu, Zhijie and Kawaharazuka, Kento and Okada, Kei},
  year    = {2026},
  eprint  = {2608.10824},
  archivePrefix = {arXiv}
}