围绕 GPT-6 Astra 搭生产栈的 7 个开源仓库
从推理服务、路由、记忆到护栏与可观测性，作者列出 Astra 上线前该配齐的七层基础设施。
GPT 6 Astra needs a f...cking operating system around it 7 GitHub repos cover serving, routing, memory, guardrails and proof 01 vLLM ▸ https://github.com/vllm-project/vllm → serves high throughput inference for real traffic 02 llama.cpp ▸ https://github.com/ggerganov/llama.cpp → keeps a local fallback alive when the cloud path dies 03 RouteLLM ▸ https://github.com/lm-sys/RouteLLM → sends easy prompts to cheaper models automatically 04 Portkey Gateway ▸ https://github.com/Portkey-AI/gateway → one gateway with retries, keys and guardrails 05 Zep ▸ https://github.com/getzep/zep → assembles long running agent context without stuffing the prompt 06 LLM Guard ▸ https://github.com/protectai/llm-guard → scans inputs and outputs before anything leaves the system 07 Opik ▸ https://github.com/comet-ml/opik → traces production runs and catches quality drops early A stronger model does not replace an operating system Astra should decide, route, remember and ship through dedicated layers Swap the model tomorrow and the company stack still stands
这条是引用着另一条写的：x.com/marlenuii/status/2109673372091126158