VulnPR-100: Kimi K3.1 Pro finds 47, GPT-6 Astra 46
On BugBunny's VulnPR-100 benchmark, Kimi K3.1 Pro found 47 vulnerabilities for $124.69, one more than GPT-6 Astra and Muse Wonder 1.0 at 46, but still behind DeepSeek V4.1 Flash.
We gave Moonshot’s new Kimi K3.1 Pro 100 pull requests with known vulnerabilities and one hour per PR to find them in the @BugBunny_ai VulnPR-100 benchmark. It found 47/100 for $124.69. That is 10 more vulnerabilities than Kimi K3 while costing $20.54 less. Cost per finding dropped from $3.93 to $2.65, a 33% improvement. K3.1 Pro beat GPT-6 Astra and Muse Wonder 1.0, both at 46, and landed just one finding behind Claude Opus 5 while costing 3.6× less. It still trails DeepSeek V4.1 Flash 1017, which found 54 vulnerabilities for only $7.53. That is seven more findings at roughly 16.6× lower cost. A strong generational improvement from @Kimi_Moonshot, but DeepSeek remains the model to beat on cost-adjusted security performance.