Testing GPT-6 Astra for robot control on Ditto-Bench
The authors spent a large number of tokens comparing Astra with recent VLA models on control interfaces, learning from failure, reasoning effort, and performance without memory, with all code open-sourced.
How useful is GPT-6 Astra for robot control, really? 🤔 Lin and I burned plenty of tokens to explore questions: - How does Astra compare with recent VLAs across different tasks? - Which control interfaces work best with Astra (linear motion vs. action chunking)? - Can it learn from its own failures or others' demonstrations? - Does higher reasoning effort (more thinking) improve performance? - What happens when we remove its memory? (Many VLAs have little or no memory) Check Ditto-Bench and our findings in the blog post. Full code included! 👇 https://llong-cs.github.io/llms-for-robot-control/ This was a fun analysis led by Lin, one of my first PhD students at JHU. He joined without much robotics experience but quickly produced a thorough benchmark and thoughtful analysis! 💪
This post quotes another post: “Ditto-Bench tests GPT-6 Astra on complex physical robot control”