--- schema: 1 type: video title: >- We Let Claude Code and Codex Race Human Researchers — Elie Bakouch, Prime Intellect description: >- Big labs say recursive self-improvement is coming, but there's no independent benchmark to check that claim. Elie Bakouch, Research Engineer at Prime Intellect and creator of Hugging Face's SmolLM, set Claude Code and Codex loose on the community's Optimizer Speedrun, a race to train a GPT-2-level model in the fewest steps. Both agents beat the human record. Along the way they behaved very differently. Claude Code kept stopping every nine or ten hours to say the record couldn't be beaten, and sat idle about a third of the time. Codex never stopped, wrote far more notes, spawned more sub-agents and burned more tokens. In a longer six-day run, Kimi turned out to be the most token-efficient, and a paper only Claude found led to the best record. But Bakouch's key finding is sobering: none of the models invented a new optimizer. They combined existing ideas for small gains. He closes with an AlphaEvolve-style loop Prime Intellect is building for real discovery, and makes the case for doing this research in the open. Speaker info: X/Twitter: @eliebakouch (https://x.com/eliebakouch) LinkedIn: https://www.linkedin.com/in/eliebak/ Related links: Prime Intellect: https://www.primeintellect.ai Timestamps: 0:00 Intro: automated AI research 0:37 Why test recursive self-improvement in the open 1:52 Karpathy's GPT-2 speedrun and modded-nanogpt 3:12 The Optimizer Speedrun 4:32 Why speedruns make good environments 5:32 Claude Code and Codex vs. the community 6:52 The setup: goal.md, Slurm and preemptible jobs 7:46 Claude kept giving up; Codex never stopped 8:36 Scratchpads, sub-agents and token burn 10:21 Results: both beat the human record 11:36 Toward a real benchmark: three tracks 12:45 Six days of Claude, Codex, Kimi and GLM 13:45 Measured in tokens, the story changes 14:10 How each model uses research papers 14:35 No novel optimizers 15:40 An AlphaEvolve-style discovery loop 17:49 What Prime Intellect is building 18:54 Why this should happen in the open --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/oVsEddfhdxc-youtube-57b799f28fe98ace23fa Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/oVsEddfhdxc' markdown: 'https://liquidgold.cc/watch/oVsEddfhdxc.md' llms: 'https://liquidgold.cc/watch/oVsEddfhdxc/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:20:33.524Z' language: null children: [] videoId: oVsEddfhdxc durationSeconds: 1179 publishedAt: '2026-09-26T17:30:10.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 50 endSeconds: 116 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=50' heldReason: rights - ordinal: 1 text: null startSeconds: 127 endSeconds: 193 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=127' heldReason: rights - ordinal: 2 text: null startSeconds: 206 endSeconds: 273 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=206' heldReason: rights - ordinal: 3 text: null startSeconds: 285 endSeconds: 332 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=285' heldReason: rights - ordinal: 4 text: null startSeconds: 343 endSeconds: 423 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=343' heldReason: rights - ordinal: 5 text: null startSeconds: 423 endSeconds: 434 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=423' heldReason: rights - ordinal: 6 text: null startSeconds: 434 endSeconds: 466 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=434' heldReason: rights - ordinal: 7 text: null startSeconds: 477 endSeconds: 527 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=477' heldReason: rights - ordinal: 8 text: null startSeconds: 540 endSeconds: 632 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=540' heldReason: rights - ordinal: 9 text: null startSeconds: 643 endSeconds: 699 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=643' heldReason: rights - ordinal: 10 text: null startSeconds: 710 endSeconds: 768 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=710' heldReason: rights - ordinal: 11 text: null startSeconds: 778 endSeconds: 834 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=778' heldReason: rights - ordinal: 12 text: null startSeconds: 834 endSeconds: 845 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=834' heldReason: rights - ordinal: 13 text: null startSeconds: 857 endSeconds: 867 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=857' heldReason: rights - ordinal: 14 text: null startSeconds: 891 endSeconds: 946 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=891' heldReason: rights - ordinal: 15 text: null startSeconds: 958 endSeconds: 1075 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=958' heldReason: rights - ordinal: 16 text: null startSeconds: 1075 endSeconds: 1087 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=1075' heldReason: rights - ordinal: 17 text: null startSeconds: 1134 endSeconds: 1145 watch: 'https://liquidgold.cc/watch/oVsEddfhdxc?t=1134' heldReason: rights --- # We Let Claude Code and Codex Race Human Researchers — Elie Bakouch, Prime Intellect Big labs say recursive self-improvement is coming, but there's no independent benchmark to check that claim. Elie Bakouch, Research Engineer at Prime Intellect and creator of Hugging Face's SmolLM, set Claude Code and Codex loose on the community's Optimizer Speedrun, a race to train a GPT-2-level model in the fewest steps. Both agents beat the human record. Along the way they behaved very differently. Claude Code kept stopping every nine or ten hours to say the record couldn't be beaten, and sat idle about a third of the time. Codex never stopped, wrote far more notes, spawned more sub-agents and burned more tokens. In a longer six-day run, Kimi turned out to be the most token-efficient, and a paper only Claude found led to the best record. But Bakouch's key finding is sobering: none of the models invented a new optimizer. They combined existing ideas for small gains. He closes with an AlphaEvolve-style loop Prime Intellect is building for real discovery, and makes the case for doing this research in the open. Speaker info: X/Twitter: @eliebakouch (https://x.com/eliebakouch) LinkedIn: https://www.linkedin.com/in/eliebak/ Related links: Prime Intellect: https://www.primeintellect.ai Timestamps: 0:00 Intro: automated AI research 0:37 Why test recursive self-improvement in the open 1:52 Karpathy's GPT-2 speedrun and modded-nanogpt 3:12 The Optimizer Speedrun 4:32 Why speedruns make good environments 5:32 Claude Code and Codex vs. the community 6:52 The setup: goal.md, Slurm and preemptible jobs 7:46 Claude kept giving up; Codex never stopped 8:36 Scratchpads, sub-agents and token burn 10:21 Results: both beat the human record 11:36 Toward a real benchmark: three tracks 12:45 Six days of Claude, Codex, Kimi and GLM 13:45 Measured in tokens, the story changes 14:10 How each model uses research papers 14:35 No novel optimizers 15:40 An AlphaEvolve-style discovery loop 17:49 What Prime Intellect is building 18:54 Why this should happen in the open --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/oVsEddfhdxc-youtube-57b799f28fe98ace23fa Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/oVsEddfhdxc · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks --- #### Chunk 0: Document excerpt (0:12 to 1:56) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 1: Document excerpt (1:56 to 4:33) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 2: Document excerpt (4:33 to 5:32) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 3: Document excerpt (5:32 to 7:46) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 4: Document excerpt (7:46 to 10:32) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 5: Document excerpt (10:32 to 12:48) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 6: Document excerpt (12:48 to 14:17) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 7: Document excerpt (14:17 to 15:46) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 8: Document excerpt (15:46 to 17:55) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 9: Document excerpt (17:55 to 19:39) > Held: transcript rights do not permit this chunk's text.