DeepSeek V4.1 Flash hands-on tests and comparisons

On the new AA index Opus 5.5 leads at 58, Kimi K3.1 Pro tops open weights at 46, and GLM-5.3 and DeepSeek V4.1 Flash 1017 both land at 32.

Ethan Mollick comments on Artificial Analysis's v5 leaderboard: with private tests added, open-weight models trail further, with released-weight DeepSeek V4.1 Flash 1017 and GLM-5.3 both at 32 versus a top score of 58.

The author reads Artificial Analysis' Intelligence Index v5, where private data carries 60% weight: Kimi K3.1 Pro leads open weights at 46 but is unreleased, while released DeepSeek V4.1 Flash and GLM-5.3 both score 32.

Intelligence Index v5 moves 60% of tests to private sets, widening the open-vs-closed gap from 3 to 12 points; top open-weight scores are DeepSeek V4.1 Flash and GLM-5.3 at 32, still under Claude Haiku 5.5's 36.

The post details v5 changes: 60% private datasets to cut contamination plus new FrontierCode 2.0 and Terminal-Bench-Science tests, with Kimi K3.1 Pro at 46 and DeepSeek V4.1 Flash tied with GLM-5.3 at 32 among open weights.

The author argues DeepInfer mainly benefits hosting vendors: it only supports datacenter cards like B200 and Ascend 950 plus offline DeepSeek-V4.1-Flash inference, while individual devs gain indirectly via lower costs and more providers.

Artificial Analysis v5 shows open-weights models falling further behind; among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) tie at 32, still below Claude Haiku 5.5.

The author ranks recent open-weight Flash models by hands-on experience, placing DeepSeek V4.1 Flash 1017 first ahead of Ling-3.1-Flash and MiniMax-M3.1-Flash.

After 100+ hours digging through eval tasks and running closed-source tasks across frontier models, the author finds V4.1 Flash often on par with frontier models and 13–20x cheaper even when 5–8% behind; on Cursor bench it is 13x cheaper with fully open weights down to the inference stack.

V4.1 Flash has led OpenRouter since Oct 25 at roughly 30% of token traffic, yet one GB300 rack could serve a day of its traffic; API is $0.15/$0.60 per 1M tokens off-peak.

The author tries a DeepSeek V4.1 Flash 1017 IQ2_XXS-Uncensored quant on Strata, swapping the SSD-resident Engram part from 4-bit to original MXFP8 weights using only the weights already on hand.

A user reports running a 748B DeepSeek V4.1 Flash build locally on stock llama.cpp and links GSQ-RCO upstream GGUF quantized weights on Hugging Face.