Bonsai2 27B 量化模型在单张 RTX 5060 Ti 16GB 上跑 262k 上下文
PrismML 的 Bonsai2 27B（Qwen 3.8 27B 的 1.75 bit 压缩）配合作者 fork 的 llama.cpp v1.1，在单张 16GB 显卡上同时加载 262k 上下文、MTP 头和视觉，生成约 67 tok/s。
every single rtx 5060 ti 16gb vram owner, bonsai2 27b dense flies on your gpu with the full 262k context window, the mtp head and vision all loaded at once, and it runs at 67 tok/s. > 262k context + mtp head + vision on one card: 15.2 of 15.9 gb > 67 tok/s with the mtp head, 71 tok/s on code, 53 tok/s without it, the stock build does 42 tok/s > with the head at depth: 39.5 tok/s at 39k tokens of context, 21 tok/s at 119k > 4 agents at once: 127 tok/s total with the head, ~33 tok/s each, the stock build does 58 tok/s without it > prefill: ~1,060 tok/s, a 119k token prompt read in about 3 minutes > vision: read all ten stats off a dashboard screenshot, answered at 66 tok/s > power: ~155 of 180 w the whole run what you're watching is the first 8 minutes of it thinking through a build i gave it, at 10x. every box is live: speed in the server log, power and memory on the card, prefill and generation speed in the llama.cpp web ui. the model is bonsai 2 27b, @PrismML's 1.75 bit compression of qwen 3.8 27b, with qwen's mtp head grafted back on so it drafts its own next tokens, served by my fork of llama.cpp. v1.1 shipped today with two blackwell fixes, a bug in my kernel that turned output into garbage on rtx 50 cards, and one in llama.cpp's attention that ran long context at less than half speed. next on this card: computer use and browser use. all links below. if you have a single rtx 5060 ti 16gb you can run this tonight, and there's a line for every 30, 40 and 50 series card with 8gb or more.
这条是引用着另一条写的：x.com/sudoingx/status/2103922552838009174