Value as Memory: Online Value-Guided Post-training for General Robot Policies
Preprint · 2026
Value as Memory (VaM) combines value caching with online real-world reinforcement learning for general robot policies. Recent value predictions provide task-progress memory to reduce ambiguity under partial observability, while concurrent data collection and training reuse offline demonstrations, off-policy human interventions, and on-policy rollouts for continued policy improvement.