MiniMax M3.1 Flash on cybersecurity benchmark: 24/32 CVEs at pass@3, behind DeepSeek V4.1 Flash
An independent security benchmark found M3.1 Flash hit 24/32 CVEs at pass@3 (75%) vs DeepSeek V4.1 Flash's 28/32 (87.5%), with 59.4% vs 70.8% single-run recall, ~40% more false positives, and $4.20 vs $2.40 per CVE; the author does not recommend it for security work.
We ran MiniMax M3.1 Flash on our cybersecurity benchmark. It's still a few months behind others for cybersecurity. - At pass@3, it found 24/32 CVEs (75.0%). DeepSeek V4.1 Flash 1017 found 28/32 (87.5%) - Single-run recall is 59.4%, vs 70.8% for DeepSeek - Precision is 80.5%, meaning it reports ~40% more false positives than DeepSeek - Cost: $4.20 per CVE found, vs $2.40 for DeepSeek V4.1 Flash 1017 It's honestly not that bad, but compared to the alternatives we have today, I would not recommend using it for cybersecurity tasks. 1/3 🧵