把 Karpathy 自动研究循环套到 Claude Code 上，一夜 63 次实验省 43%

Max Vmr @vmrmax原推发布于 2026.10.09

在 X 上打开原推

I POINTED KARPATHY'S AUTORESEARCH AT MY CLAUDE CODE. 63 EXPERIMENTS IN ONE NIGHT, AND EVERY TICKET GOT 43% CHEAPER. Karpathy's loop is 3 files and one number. A human writes program.md. The agent edits http://train.py, runs it for 5 minutes flat and keeps the change only if the number goes down. Otherwise it's a git reset and the next idea. On his own model it tried about 700 changes in 2 days. 20 survived, and his time to GPT-2 dropped from 2.02 to 1.80 hours. His conclusion is the part I stole. The loop works anywhere you have one number, one file to edit and a fixed time per run. My Claude Code bill has all three. So I swapped the parts. → My CLAUDE.md and 3 subagent files play http://train.py. → One page of rules plays program.md. What the researcher may touch, what it may not, and the simplicity rule. → Dollars per accepted ticket, with every test green, play val_bpb. → 10 fixed tickets from my 47-ticket list play the 5-minute run. Karpathy lets his loop run forever. Mine runs under /goal, where a small fast model checks the stop condition after every turn. The condition fits in one line. Print the cost after every run, and stop at $75 spent or after 3 runs in a row without a better number. The starting number was 10.66 cents a ticket. That's the Opus + Sonnet + Jev setup that closed 47 tickets for $5.01. By morning the loop had run 63 experiments. 56 got reset, 7 stayed, and a ticket cost 6.12 cents. The whole night cost $63. The 7 survivors: → Opus skips the review on diffs under 40 lines. Zero new failures. → Haiku reads only the files the failing test touches. → CLAUDE.md went from 38 lines to 22, and the cache hit rate went up. → Big tool output becomes a file path plus a 20-line excerpt. → The edit and the test run go out in one call. → Sonnet gets one retry before Opus steps in. → Jev decides whether a ticket needs a plan at all. The simplicity rule killed experiment 28. It saved 0.3% and added 40 lines of new rules. Then I ran the check Karpathy ran on his bigger model. The 37 tickets the loop never saw went from $3.94 to $2.33. That's 41% cheaper with every test green. Experiment 41 almost fooled me. It printed "all tests green" before the suite had finished, and the /goal evaluator only reads the transcript, so it believed it. The holdout run found 4 red tests, and I rolled it back. Now every run prints the raw test summary line. The proof has to sit in the transcript, because that's the only thing the evaluator ever sees. The evaluator counted 191 turns and stopped the loop at experiment 63, after 3 runs without a better number. At 4.54 cents saved per ticket, the $63 night pays for itself after 1,388 tickets. My program.md and the exact /goal line are in the reply ↓