Rapid-MLX 0.16.5: DeepSeek V4.1 Flash 1017 runs locally on 256GB Macs at ~20 tok/s
Rapid-MLX shipped an MLX build of DeepSeek-V4.1-Flash-1017 keeping all 40×384 experts with mixed 2/3/4-bit quantization, KL divergence 0.439, and ~20 tok/s at 206.5GB resident on an M3 Ultra.
🚀 Rapid-MLX 0.16.5 is LIVE: Turn your Mac into a top-tier AI compute center! 🌟 DeepSeek-V4.1-Flash-1017-MLX Run the frontier model locally on a 256GB Apple Silicon Mac—without compromising its soul: • 🔪 ZERO experts removed: We kept the full 40×384 architecture. No brutal pruning. • ⚖️ Smart Quantization: Extreme mixed-precision (2/3/4-bit for experts, while crucial routing & attention stay at 8-bit/BF16). • 📉 Near-Lossless Quality: We crushed the KL divergence down to 0.439 (compared to a catastrophic 2.02 on standard 2-bit). Code, math, and agentic capabilities are preserved! • ⚡ Speed: ~20 tok/s on an M3 Ultra (206.5GB resident RAM). 🔥 Also dropping today: • 👁️ MiniCPM-V 4.7: Vision + Tool Calling! Reads screenshots, charts, and photos flawlessly. Available in a tiny 1B for laptops or 35B-A3B for deep reasoning. • ⚡ LFM2.5-24B-A2B (Liquid AI): 24B parameters of knowledge, but only ~2B active per token. Blazing fast on everyday Macs with out-of-the-box tool calling. Stop paying for expensive cloud APIs and waiting in queues. Your Mac is all you need. 👇 💻 pip install -U rapid-mlx 🚀 rapid-mlx serve deepseek-v41-flash-1017 Learn more: http://rapidmlx.com
This post quotes another post: x.com/artificialanlys/status/2112218661432000694