--- schema: 1 type: video title: >- The Death of the Code Review: What the Data Actually Says — Laurie Voss, Arize AI description: >- Developers using autonomous agents wrote 741% more code but shipped only 30% more software. The bottleneck is human review, and "just review harder" doesn't scale. Reviewer effectiveness collapses past about 400 lines, and agents now open 10,000-line PRs. Laurie Voss (Head of Developer Relations at Arize AI, co-founder of npm) reviews what the industry is actually doing about it. The evidence covers OpenAI's zero-human-code product, METR's finding that about half of SWE-bench-passing PRs wouldn't be merged, and Cognition's FrontierCode (88% on SWE-bench Pro vs. 29% on real mergeability). She explains why a mergeability benchmark would immediately become a training signal for frontier models. She also covers how Cursor and GitHub run review in production, why multi-pass review and default suspicion cut false positives, and what Carlini's agent-built C compiler and Bun's million-line Zig-to-Rust port (13,044 unsafe blocks) reveal about taking humans out of the loop. Automated reviewers can be fooled by prompt injection that humans catch, which leaves production as the last reviewer standing. Speaker info: X: https://x.com/seldo Bluesky: https://bsky.app/profile/seldo.com LinkedIn: https://www.linkedin.com/in/seldo/ GitHub: https://github.com/seldo COMPANY Arize AI: https://arize.com X: https://x.com/arizeai LinkedIn: https://www.linkedin.com/company/arizeai Timestamps: 0:00 The new bottleneck: human review 1:37 741% more code, 30% more software 2:37 Generation is no longer the bottleneck 3:22 Why "review harder" fails: the Cisco study 4:52 Stop reading code? Loops and OpenAI's zero-human-code product 6:42 Do passing tests mean mergeable? METR's SWE-bench study 8:01 FrontierCode: 88% vs. 29% 9:11 Mergeability as the next training signal 11:06 Automated review today: GitHub Copilot and Cursor 13:16 Review is fusing with repair 13:51 CodeRabbit, Greptile, Graphite and the acceptance metric 15:00 Can you skip the human? Carlini's C compiler 16:00 Bun's Zig-to-Rust port and 13,044 unsafe blocks 17:30 How OpenAI rebuilt review as a system 18:45 Dex Horthy: "Please read the code" 19:30 Context the tests can't see 19:55 Where the human checkpoint survives 20:40 Who reviews the reviewers? Prompt injection 22:04 Production: the last reviewer standing 22:44 Code review is being rebuilt, not killed 23:29 What to do today: build a review harness --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/_mi3alkqy4s-youtube-459d7ff58475ac5b98d1 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/_mi3alkqy4s' markdown: 'https://liquidgold.cc/watch/_mi3alkqy4s.md' llms: 'https://liquidgold.cc/watch/_mi3alkqy4s/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:20:17.229Z' language: null children: [] videoId: _mi3alkqy4s durationSeconds: 1481 publishedAt: '2026-09-30T13:30:35.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 23 endSeconds: 102 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=23' heldReason: rights - ordinal: 1 text: null startSeconds: 102 endSeconds: 114 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=102' heldReason: rights - ordinal: 2 text: null startSeconds: 114 endSeconds: 160 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=114' heldReason: rights - ordinal: 3 text: null startSeconds: 160 endSeconds: 170 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=160' heldReason: rights - ordinal: 4 text: null startSeconds: 170 endSeconds: 204 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=170' heldReason: rights - ordinal: 5 text: null startSeconds: 204 endSeconds: 216 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=204' heldReason: rights - ordinal: 6 text: null startSeconds: 216 endSeconds: 295 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=216' heldReason: rights - ordinal: 7 text: null startSeconds: 295 endSeconds: 306 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=295' heldReason: rights - ordinal: 8 text: null startSeconds: 306 endSeconds: 408 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=306' heldReason: rights - ordinal: 9 text: null startSeconds: 408 endSeconds: 419 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=408' heldReason: rights - ordinal: 10 text: null startSeconds: 419 endSeconds: 485 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=419' heldReason: rights - ordinal: 11 text: null startSeconds: 485 endSeconds: 497 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=485' heldReason: rights - ordinal: 12 text: null startSeconds: 497 endSeconds: 551 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=497' heldReason: rights - ordinal: 13 text: null startSeconds: 551 endSeconds: 561 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=551' heldReason: rights - ordinal: 14 text: null startSeconds: 561 endSeconds: 675 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=561' heldReason: rights - ordinal: 15 text: null startSeconds: 675 endSeconds: 685 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=675' heldReason: rights - ordinal: 16 text: null startSeconds: 685 endSeconds: 796 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=685' heldReason: rights - ordinal: 17 text: null startSeconds: 796 endSeconds: 806 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=796' heldReason: rights - ordinal: 18 text: null startSeconds: 806 endSeconds: 840 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=806' heldReason: rights - ordinal: 19 text: null startSeconds: 840 endSeconds: 851 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=840' heldReason: rights - ordinal: 20 text: null startSeconds: 851 endSeconds: 909 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=851' heldReason: rights - ordinal: 21 text: null startSeconds: 909 endSeconds: 920 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=909' heldReason: rights - ordinal: 22 text: null startSeconds: 920 endSeconds: 964 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=920' heldReason: rights - ordinal: 23 text: null startSeconds: 964 endSeconds: 975 watch: 'https://liquidgold.cc/watch/_mi3alkqy4s?t=964' heldReason: rights --- # The Death of the Code Review: What the Data Actually Says — Laurie Voss, Arize AI Developers using autonomous agents wrote 741% more code but shipped only 30% more software. The bottleneck is human review, and "just review harder" doesn't scale. Reviewer effectiveness collapses past about 400 lines, and agents now open 10,000-line PRs. Laurie Voss (Head of Developer Relations at Arize AI, co-founder of npm) reviews what the industry is actually doing about it. The evidence covers OpenAI's zero-human-code product, METR's finding that about half of SWE-bench-passing PRs wouldn't be merged, and Cognition's FrontierCode (88% on SWE-bench Pro vs. 29% on real mergeability). She explains why a mergeability benchmark would immediately become a training signal for frontier models. She also covers how Cursor and GitHub run review in production, why multi-pass review and default suspicion cut false positives, and what Carlini's agent-built C compiler and Bun's million-line Zig-to-Rust port (13,044 unsafe blocks) reveal about taking humans out of the loop. Automated reviewers can be fooled by prompt injection that humans catch, which leaves production as the last reviewer standing. Speaker info: X: https://x.com/seldo Bluesky: https://bsky.app/profile/seldo.com LinkedIn: https://www.linkedin.com/in/seldo/ GitHub: https://github.com/seldo COMPANY Arize AI: https://arize.com X: https://x.com/arizeai LinkedIn: https://www.linkedin.com/company/arizeai Timestamps: 0:00 The new bottleneck: human review 1:37 741% more code, 30% more software 2:37 Generation is no longer the bottleneck 3:22 Why "review harder" fails: the Cisco study 4:52 Stop reading code? Loops and OpenAI's zero-human-code product 6:42 Do passing tests mean mergeable? METR's SWE-bench study 8:01 FrontierCode: 88% vs. 29% 9:11 Mergeability as the next training signal 11:06 Automated review today: GitHub Copilot and Cursor 13:16 Review is fusing with repair 13:51 CodeRabbit, Greptile, Graphite and the acceptance metric 15:00 Can you skip the human? Carlini's C compiler 16:00 Bun's Zig-to-Rust port and 13,044 unsafe blocks 17:30 How OpenAI rebuilt review as a system 18:45 Dex Horthy: "Please read the code" 19:30 Context the tests can't see 19:55 Where the human checkpoint survives 20:40 Who reviews the reviewers? Prompt injection 22:04 Production: the last reviewer standing 22:44 Code review is being rebuilt, not killed 23:29 What to do today: build a review harness --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/\_mi3alkqy4s-youtube-459d7ff58475ac5b98d1 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/_mi3alkqy4s · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks --- #### Chunk 0: Document excerpt (0:12 to 1:42) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 1: Document excerpt (1:42 to 2:40) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 2: Document excerpt (2:40 to 3:24) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 3: Document excerpt (3:24 to 4:55) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 4: Document excerpt (4:55 to 6:48) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 5: Document excerpt (6:48 to 8:05) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 6: Document excerpt (8:05 to 9:11) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 7: Document excerpt (9:11 to 11:15) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 8: Document excerpt (11:15 to 13:16) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 9: Document excerpt (13:16 to 15:09) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 10: Document excerpt (15:09 to 16:04) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 11: Document excerpt (16:04 to 17:32) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 12: Document excerpt (17:32 to 18:49) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 13: Document excerpt (18:49 to 19:38) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 14: Document excerpt (19:38 to 20:43) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 15: Document excerpt (20:43 to 22:13) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 16: Document excerpt (22:13 to 22:46) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 17: Document excerpt (22:46 to 23:30) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 18: Document excerpt (23:30 to 24:41) > Held: transcript rights do not permit this chunk's text.