MiniMax M3.1 Flash at 2.15 bpw on two DGX Sparks: 33 tok/s but weaker quality
Two DGX Sparks run the 428B M3.1 Flash at 2.15 bpw, hitting 33 tok/s on rewriting and 29 on code with 13.6s for an 8k prompt, but IFEval drops to 60.5% versus 89.0% for DeepSeek V4.1 Flash at 2.9 bpw, so a 3.06 bpw pack is being tested.
MiniMax-M3.1-Flash (428B MoE) running on my two DGX Sparks at 2.15 bpw. 33 tok/s on rewrite, 29 on code. @ViC305's EXL3 pack and vLLM EXL3 plugin, with @Tech2Wild's two-Spark patches. Half the model on each Spark over QSFP, 30 GiB still free on each. An 8k prompt reads in 13.6 s. Quality is the weak part. IFEval (thinking off) is 60.5%, DeepSeek-V4.1-Flash at 2.9 bpw got 89.0% on the same set. Most of the experts here are 2-bit, so the 3.06 bpw pack is running now on the same setup to see how much of that is the quant.
This post quotes another post: “MiniMax M3.1 Flash EXL3 quants: 428B on one DGX Spark at 96.89% token agreement”