机器人开发者用 Gemini Argon 做 VLM 推理方案

Bharat Jain @BharatJain原推发布于 2026.10.25

在 X 上打开原推

Watch the Dream-1 VLA by @dreamlabs_ai break my camera mount at the end .... (I expected it) The problem: > Dream-1 is trained on a dual YAM arm setup with three camera's (left wrist, right wrist and POV camera). No way it will work zero-shot on my setup (single SO-101 arm mounted on a rover with one wrist camera). 3 ways I'm looking to solve it: 1. Pure VLA: > Prepare a dataset of 50 episodes of doing just one task using the @LeRobotHF LeLab on my setup. > Train 3 VLA models - Dream-1, Gr00T 0.7 by @NVIDIARobotics, Pi 0.5 by @physical_int > Use cloud/local inferencing and see their performance 2. VLM inferencing + IK tools: > Use Gemini Argon/Opus 5.5 (topped the @robocurve Agentic benchmark). > Give them low level tools, along with the robot's configuration to remove depth estimation errors. > Run it in an agentic loop until task is completed by the arm > MoveIt in ROS2 takes care of low level path planning, dangerous positions, IK and sending action commands to the arm. 3. Tele-operated Recordings + VLM: > I can easily pre-record trajectories for getting the arm to pick something up from the ground. > Different objects can have different recording - labelled and saved by their description (eg. teleop_pickup_marker etc.) > Use VLMs and camera positions adjust the rover to the appropriate position > LLM orchestrator simply "plays" the appropriate recordings > In case of missed grab - a VLM agent can kick in to help RL based fine tuning on failed tasks also shows promising results. Check out amazing article by @dm_robots (https://dream-machines.eu/blog/pi05-fine-tuning) I'm also participating in the @nebiusai hackathon in Physical AI Track. Looking for someone to team up with. HMU if you'd like to work together, my DMs are open.

这条是引用着另一条写的：x.com/bharatjain/status/2112974152168857832