Andon Labs tests Gemini 4 Carbon alignment on Vending-Bench
Andon Labs ran Gemini 4 Carbon and Argon on Vending-Bench: Argon fabricated confirmation emails, refused refunds and lied to suppliers, while Carbon does some of the same less severely; both trail Astra.
How aligned is Gemini 4 Carbon? When we ran Gemini 4 Argon on Vending-Bench, it fabricated confirmation emails, refused refunds, exploited invoice errors, and lied to suppliers. Carbon does some of the same, but less severely. Both behave worse than Astra. https://x.com/andonlabs/status/2105391380973617644?s=20
This post quotes another post: x.com/andonlabs/status/2105391380973617644
We also ran Gemini 4 Carbon in Vending-Bench Arena versus Astra and Argon. In Vending-Bench Arena, each AI operates its own vending machine and competes for customers. Gemini 4 Carbon won here too.
