Physical intelligence dinner: Astra's in-context learning and the memory debate
At a Palo Alto robotics dinner, discussion focused on GPT-6 Astra's strong zero-shot grasping numbers against a 10% result on contact-rich tasks, with the real bottleneck framed as continuity rather than policy.
Physical intelligence is no longer just a policy problem. It’s increasingly a continuity problem. Tonight at @shawnshenjx & @memories_ai Physical Intelligence dinner in Palo Alto, that was the argument that survived. @OpenAI GPT-6 Astra has put up striking zero-shot numbers on the less contact-intensive side of embodiment: 70% on a MolmoSpaces subset vs. 53% for Cosmos3, 95% block-to-bowl on real YAM arms, 98% on a RoboLab subset. Yet contact-rich precision remains a wall: 10% on the puzzle-in-groove task. The more interesting jump is in-context learning. In one recent robot-learning study, human-video context takes Astra’s red-towel task progress from 55% to 100%; in that study’s repeated trials it succeeds 2/3 with the video in window. That is not a skill written into weights. The context changed what it could do now. A million-token context window is a rented past, not a hippocampus. A model that can paint the Golden Gate with a robot arm can still be a goldfish. The painting improved because the attempts never left context. @Figure_robot #Helix2.5 shows the collective past transferring as weights: Index pretraining moves zero-shot success from 9% to 56% across 30 unseen homes. @physical_int’s MEM and @memories_ai’s visual memory layer push the other side — how an embodied system keeps experience after the episode ends. Memory without in-context learning is just storage. Weights → what the robot knows. Memory → what the robot remembers. Context → what it can use now. Policy → what it does next. Astra may have pushed in-context planning into the foundation-model layer. It didn’t solve continuity. The next bottleneck may not be another foundation model. It may be compiling a robot’s accumulated experience into the right context at the right moment — without retraining. Generalist robots don’t just need an autobiography. They need to learn from it in context. Actuation is commoditizing. Memory is not. Context is the interface. @aurorafeng_01, @AndrewDai, @HongweiYi2