Gemini 4 Argon beats a human at 20 Questions
The author had regularly beaten LLMs at 20 Questions with Caesar-cipher word memory, but Gemini 4 Argon avoided the usual linguistic shortcuts, suggesting more feature-based reasoning.
I had regularly beaten LLMs at "20 Questions" (use Caesar Ciphers so the LLM remembers what word it chose) up until Gemini 4 Argon. It may seem weird that humans would beat LLMs at a game based on far-associations and vocabulary, but previous LLMs set up and fall for their own linguistic shell-games. For example, it might ask "Can it fit in an adult's hand?" - and if the answer is yes, the LLM might say "It it handheld." Thus, "fits in an adult's hand" gets transmuted into "handheld", which has all sorts of other connotations. Argon didn't do this, i.e. its reasoning is likely much more feature-based than relation-based somehow, though I don't think LLMs can reason purely in features like a human can yet.