Cutting the last 12 layers lets 3B decision model RSI-Jev beat 4B models
The author found that decisions don't need the full depth of an LLM: truncating Qwen3.5-4B at layer 20 barely changed performance, so they released RSI-Jev v5.0-VL 3B.
A System One model may not need the full depth of an LLM. AutoScientists found that much of the later network can be removed without hurting decision performance. On Qwen3.5-4B, stopping at layer 20 of 32 performed basically the same as using the full model. So they cut the last 12 layers and shipped RSI-Jev v5.0-VL 3B. On Decision Index, it scores 38.38 vs 28.97 for the next best model at 3B and below, and it also beats eleven 4B models. In hindsight, this kind of makes sense: deciding and speaking are different jobs. A System One model doesn’t need to generate the next word. It just needs to read and make a call. The next question is even more interesting: why should every question use the same depth? If easy ones can stop early while hard ones go deeper, this could get a lot faster. https://github.com/Shanghua-Gao/RSI-Jev
This post quotes another post: x.com/gaoshanghua/status/2103572751466312150