DeepSeek V4.1 Flash on VulcanBench: far lower cost per task than Sol and Opus 5.5
On VulcanBench's Frontier v4 suite, V4.1 Flash lands in the 80–90 good-enough range with far lower cost per task than GPT-6.1 Sol and Opus 5.5, though raising effort barely helps.
Okay, just finished my DeepSeek v4.1 Flash benchmark run on @VulcanBench 🖖 And when I woke up to see the results, I kinda did a double take. You really have to look at the cost-per-task comparison in a bar chart to get it. This is an insanely cost effective model, I am impressed. For those who missed me saying this in the past. On my Frontier v4 eval suite, anything in the 80 - 90 range is more than good enough for routine daily work. The only weird thing about DeepSeek v4.1 Flash is that increasing effort level doesn't seem to really matter. Will be digging into the traces to understand this better as this is the only real head scratcher for me besides that fact that it's practically free to run this model. I mean seriously, look at cost-per-task, the green bar is GPT-6.1 Sol, the orange is Claude Opus 5.5 and the blue is DeepSeek v4.1 Flash. Everything is now live on the site so for anyone who wants to do a deeper dive, here ya' go: https://vulcanbench.com/benchmarks/swe-v4-deepseek-flash-v327.html