Microsoft Agent Lightning trains agents on GPT-6 Sol and Claude Opus 5.5 traces
A long post describes Microsoft's Agent Lightning framework, which intercepts Claude Opus 5.5 and GPT-6 Sol/Luna calls through an API proxy to run reinforcement learning on agent trajectories, lifting Qwen3.5-9B by 14.6 points on SWE-bench.
this is pure f*cking treasure Microsoft turned production agent training into PyTorch Lightning Frontier economics shifted overnight with Claude Opus 5.5 cutting cache reads to $0.20 per million, alongside GPT-6 Sol at $2 input and high-volume GPT-6 Luna at $0.10. Yet raw model intelligence hits an operational ceiling once multi-turn agents coordinate tools, recover from syntax errors, and operate inside production sandboxes Microsoft released Agent Lightning, a 3,500-line framework that runs reinforcement learning directly on production agent harnesses with zero codebase modifications Claude Opus 5.5 & GPT-6 Sol harness → API Gateway proxy → Trajectory capture → verl RL engine → vLLM policy update Four architectural pillars behind the lightweight agent trainer: 1. Zero-Harness Code Modifications The API proxy intercepts calls between the agent and frontier providers like Claude Opus 5.5 and GPT-6 Sol. It captures tool arguments and environment state while the underlying agent orchestration remains completely untouched 2. Multi-Model Trajectory Capture The gateway routes complex planning to Claude Opus 5.5 and high-volume worker steps to GPT-6 Luna. It aggregates multi-turn tool traces directly into verified training datasets for policy optimization 3. Distributed Training via verl and vLLM The trainer coordinates parallel rollouts across local hardware or native Kubernetes clusters. It combines high-throughput vLLM generation with verl reinforcement learning to update model weights at scale 4. Verifiable SWE-bench Verified Gains Trained on 6K task samples distilled from production traces, an end-to-end Qwen3.5-9B model increased SWE-bench Verified accuracy from 41.8% to 56.4%. The 14.6 percentage point gain required zero manual labels and prevented reward hacking Trace engineering establishes execution traces as an active causal feedback loop. Connecting frontier models like Claude Opus 5.5 and GPT-6 Sol to Agent Lightning turns runtime traces directly into permanent model weights Read the complete breakdown in the article below ↓
This post quotes another post: x.com/marfinxx/status/2094016175617241109