pd-bridge：跨 NVIDIA 与 Apple 芯片的异构推理工具链

Volatile Markets @volatilemarkts原推发布于 2026.09.23

在 X 上打开原推

EVERY GPU IN THE HOUSE. ONE NERVOUS SYSTEM. A follow-up to our heterogeneous inference post. These past few weeks we've been lucky to find some incredible people on X. What they taught us is simple: when builders share their breakthroughs and their roadblocks in the open, the whole community moves faster. This is what that looked like from our basement. How we got here • A couple weeks back we shipped pd-bridge: prefill on DGX Sparks, decode on Mac Studios. Blackwell's compute reads the conversation, Apple's unified memory answers it, and the KV cache moves between them. http://github.com/chadhurley25075-png/pd-bridge • @ashxhart shipped MCDMA, RDMA verbs on macOS. It changed what we thought was possible across architectures. Once cache can move over RDMA, NVIDIA and Apple silicon stop being islands. http://github.com/ashhart/MCDMA • The more we learned, the clearer it got: KV cache plus RDMA isn't just a speed trick. It's a transport, a way for model memory to move from one machine, and one architecture, to another. • We have a lot of machines, so we gave them a CNS, a central nervous system: one always-on message bus (NATS) with a small agent on every machine. Each sends a heartbeat every 5 seconds, and any node can run a command on any other in milliseconds. SSH is the hands; the CNS is the nerves. • Then Ash shipped Drift, and that was another paradigm shift: shipping model memory, not text, between models of the same family and of different families. This is not RAG, and it's not a clever memory repo. It's the next step in how models remember. http://github.com/ashhart/Drift We're fortunate to have a wide spread of hardware, so we could field-test all four pieces together: prefill/decode, MCDMA, Drift and the CNS. What happened next had us feeling the proto-AGI. The test One private journal entry, written that day and in no training set. Six questions about it, and one asks for a fact that isn't there. Every question went out over the CNS, and every machine answered on its own local model. The machines • 7× GB10 Sparks (5 NVIDIA DGX Spark + 2 ASUS GX10): GLM-5.3-Flash. Six serve models, and one runs the memory and the nervous system. • 5× Mac Studio (M3 Ultra): GLM-5.3-Flash on MLX • 1× Tesla V100 (2017): Llama 3.1 8B • 1× RTX 5080: Qwen2.5 7B 9 local models, 3 model families, 5 kinds of hardware. 14 machines and 2 MikroTik CRS812 switches, unified. The result: 48 of 54 right! And the part that matters: two of those Mac Studios never read the entry. Two GB10s read it, and their KV cache crossed an RDMA link into Apple silicon in 0.1 seconds. The Macs answered from memory they never computed: 6 of 6 and 5 of 6. Then we built our own translators Here's what makes @ashxhart and Drift special. Ash didn't just ship a demo between two models. He built it so anyone can train a translator for the models they already run. You record how one model reads, record how another reads the same text, and fit the bridge. He wants people to succeed with this, and he made that possible. So tonight we built two. GLM → Qwen2.5-3B, on the RTX 5080. Twenty questions about private passages Qwen never saw: Given nothing: 0 of 20 Given GLM's memory through Drift: 12 of 20 Reading the passage itself: 17 of 20 GLM → Llama-3.2-1B, on an Alienware laptop (RTX 4050). A small model and a small signal, but a real one. About 3 minutes to fit each translator. A gaming GPU and a laptop, answering from another model's memory. Credit where it's owed: None of this exists without @ashxhart's MCDMA and Drift. @MiaAI_lab makes the model builds we run. We're a basement, not a lab. Take these ideas, improve them, run with them, and tell us where we're wrong. Compute where compute is cheap. Memory where memory is cheap. RDMA in between. A nervous system over all of it. Open source must win! @NVIDIAAI @Apple @ASUS @mikrotik_com @Dell