Claude and GPT-6 Astra build robot physics twins from phone video
UC Berkeley and NVIDIA's RoboSimStudio had Claude and GPT-6 Astra independently build a robot-task physics twin from a smartphone video, one in 56 minutes, then fix a constraint after real-world failure.
That's genuinely insane! 🤯 An AI agent built a physics twin of a robot task from a phone video in under an hour. This project RoboSimStudio was built on Newton by researchers from UC Berkeley and NVIDIA Robotics. Open toolkit, every physics parameter exposed as something a human can slide or an agent can edit. The demo is a bimanual setup, one arm holding a bar, the other wiping a marker stripe off it with a foam sponge. Input was a smartphone video. What's more important no CAD, and no calibrated cameras. Claude and GPT-6 Astra each built a simulation of it independently. One finished in 56 minutes. Then they ran it on the real robot and the bar support moved. In simulation it had been modelled as fixed, so the sponge kept contact. In reality it shifted and contact was lost. The agent went back and replaced the fixed constraint with a free rigid body. The revised simulation then reproduced the tipping the original could not. That is the loop actually closing. The real world found the modelling error and the fix went back into the twin. 🔗 Check the project here: https://lnkd.in/dgXJxAhw ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → http://ziegler.substack.com