DeepSeek V4.1 Flash on Radeon R9700: Strata 0.2.6 decode speedup
Strata 0.2.6 preloading CPU-side experts into RAM lifts single R9700 code decode from 24.5 to 27.6 tok/s and 4x R9700 from 77 to 84.7 tok/s, at the cost of 55s instead of 34s load time.
Radeon AI PRO R9700 test PCs with 251 GB of RAM. DeepSeek V4.1-Flash on Strata 0.2.6 (Linux, HIP), which now copies every CPU-side expert into RAM at load when they all fit with 8 GiB to spare. Decode, before / now: 1x R9700, set 2.00, code: 24.5 / 27.6 tok/s 1x R9700, set 2.00, agent: 21.9 / 27.1 1x R9700, set 2.25, code: 26.1 / 29.6 1x R9700, set 2.25, agent: 23.1 / 27.0 4x R9700, set 2.00, code: 76.6-77.0 / 84.7 4x R9700, set 2.00, agent: 74.2 / 83.3 Trade-off: one R9700 now loads the 2.00 set in about 55 s instead of 34 s, because it copies 108 GB. If the RAM is already in the box, that's free speed on the same card. via Strata release notes: https://github.com/Niko1221/Strata/releases/tag/v0.2.6 · cc @AMD @RadeonPRO #LocalLLM #ROCm