Kimi K3.1 deliberately limits cyber capability, scoring below Astra
Moonshot filtered datasets to lower Kimi K3.1's cyber capability, scoring 74.7 on Exploitbench and 20.7 on ExploitGym versus GPT-6 Astra's 100 and 42.7.
Moonshot - on purpose - reduces cyber capability in open weights Kimi K3.1 and ensured much lower performance on Exploitbench (74.7 compared to 100 of GPT-6 Astra) & ExploitGym (20.7 compared to 42.7 of GPT-6 Astra) than frontier models. They did this by filtering out datasets, as these capabilities were 'emerging' by themselves & not by RLing on those type of tasks. This is a better approach rather than releasing a very cyber capable model with 'safeguards' that can be easily removed. This is the rigth way, with cyber capable models handed to only responsible parties and put in open domain slowly. But, at the same time frontier labs need to expand access for cyber capabilities to the responsible parties (to their credit - they are doing it). Moonshot - to their credit - has done gentlemanly thing. This is why respectful dialogue is important. But this also tells you China has as Cyber capable models as capable as with United State agencies - even if they may not be public. Capability score screenshot on next page.. @teortaxesTex @zephyr_z9 @AGraylin https://www.kimi.ai/blog/kimi-k3-1-pro