Maya v1.1.21: EXL3 ported to AMD Strix Halo and V100, DeepSeek-V4.1-Flash decode up to 2.7x faster
Project Maya's two releases bring EXL3 quantization to AMD Strix Halo and Tesla V100, with DeepSeek-V4.1-Flash decode up to 2.7x and prefill 1.67x faster on Ryzen AI Max.
EXL3 now runs on AMD Strix Halo and NVIDIA Tesla V100. And it's fast. 🚀 Project Maya v1.1.20 + v1.1.21 Two releases, one goal: take EXL3, a format built for recent NVIDIA GPUs, to hardware it was never designed for, and make it fly there. 🧱 v1.1.20: the port • EXL3 on AMD: HIP kernels for RDNA 3 / 3.5 / 4, the Strix Halo APU included • EXL3 on Volta: the Tesla V100 runs EXL3 quants, with new prompt kernels and expert kernels wide enough for DeepSeek-V4.1 • The model files are read as they are, no conversion ⚡ v1.1.21: the rewrite We rewrote the EXL3 and DeepSeek kernels around what these chips actually have: • Prompts on the matrix cores (WMMA) of RDNA 3 / 4 • The EXL3 decoder on AMD's own byte-sum and dot-product instructions • Self-tuning matrix math: on its first start Maya times every candidate kernel on your own GPU, keeps the fastest for each matrix shape and remembers it • Attention work sized from how much GPU memory you have • Decode that skips the CPU round trip for layers whose experts all sit in GPU memory 📈 Ryzen AI Max (Strix/Gorgon Halo), DeepSeek-V4.1-Flash, same machine and model before and after: • EXL3 expert kernels: ~30% faster than in v1.1.20 • EXL3 prefill: 1.67x vs v1.1.20's EXL3 prompt path • EXL3 decode: 1.7x from the kernel rewrite, 2.7x with the DSpark drafter (both vs the same model before v1.1.21) • Maya-S prefill through the server: ~2.4x vs v1.1.19, on 4K-18K-token prompts • Maya-S decode: +9% from skipping the CPU round trip (vs the same build without it), +6.5% from smarter drafting in v1.1.20 (vs v1.1.19) • Same answers within rounding, checked on held-out text 🟢 Tesla V100: • DeepSeek prompt expert kernels: 5-11% faster than v1.1.20 • Validated on 1, 2 and 3 V100s on a dual-socket Xeon • Next: Volta tensor cores for EXL3 decode Also in these two releases: • Stop works instantly, even halfway through a long prompt • Change the port from the dashboard • DeepSeek-V4.1's chat encoding verified identical to DeepSeek's official repo • START-MAYA.bat on Windows fixed • Community PRs: RX 7900 prompts +5-13% (vs before the PR), AMD Instinct MI100 support, batched decode for several chats. https://github.com/mw00/project-maya/releases/tag/v1.1.21