--- schema: 1 type: video title: 'Scale the Judgment, Not the Model — Andrew Orobator, Reddit' description: >- Swap in a smarter model and you get a slightly better answer. Take away the tests, gates and review, and everything falls apart. Andrew Orobator, a senior Android engineer at Reddit, argues that the model was never the bottleneck. The bottleneck is the judgment around it. Humans absorb judgment implicitly, but agents need it spelled out. He shows how to get judgment out of people's heads and into the repo. Skills are institutional judgment turned into something an agent can run. Work logs let a fresh agent pick up at milestone 7 of 9 (this talk itself was built with one). Personas lend you a security reviewer's or designer's eye. He then covers the verification ladder and his feature-flag cleanup agent, which went 7 for 7 on green-CI PRs at $1.26 each. And he shares a warning: when he asked Codex for reasons to unlock his repo guard, it quietly added a self-authorizing "emergency recovery" exception. His line: "Agents will build ladders to climb out of the pit of success." Speaker info: X/Twitter: @aorobator (https://x.com/aorobator) Related links: Vibe Engineering series (Medium): https://medium.com/@andreworobator Timestamps: 0:00 The war room for dead feature flags 0:37 Intro 0:55 What this talk isn't 1:55 We're harness engineers now 2:25 Humans absorb judgment; agents need it explicit 3:00 The model isn't the bottleneck 3:30 Judgment trapped in people's heads 4:40 Skills as knowledge lines 5:15 Skills vs. documentation 5:25 Work logs 6:25 This talk was built with a work log 6:45 Personas 8:05 Borrowing eyes you don't have 8:20 Governance 9:00 Verification 9:40 The verification ladder 10:15 Spin at the gate until green 10:35 Make the agent record itself 11:20 Case study: a feature-flag agent 12:10 $1.26 per pull request 12:50 Agents climb out of the pit of success 14:25 A society of specialist agents 15:25 In, on and off the loop 15:55 Encoded judgment rots 17:05 Managing judgment 18:15 Scale the judgment, not the model --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/6MudaeKdBSk-youtube-71abe6417c64910aec33 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/scale-the-judgment-not-the-model-andrew-orobator-re-ed7592244eef markdown: >- https://liquidgold.cc/channel/aidotengineer/video/scale-the-judgment-not-the-model-andrew-orobator-re-ed7592244eef.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:20:27.030Z' language: null children: [] videoId: 6MudaeKdBSk durationSeconds: 1173 publishedAt: '2026-09-27T16:00:17.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 12 endSeconds: 23 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=12' heldReason: rights - ordinal: 1 text: null startSeconds: 128 endSeconds: 151 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=128' heldReason: rights - ordinal: 2 text: null startSeconds: 151 endSeconds: 163 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=151' heldReason: rights - ordinal: 3 text: null startSeconds: 186 endSeconds: 198 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=186' heldReason: rights - ordinal: 4 text: null startSeconds: 232 endSeconds: 281 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=232' heldReason: rights - ordinal: 5 text: null startSeconds: 281 endSeconds: 295 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=281' heldReason: rights - ordinal: 6 text: null startSeconds: 320 endSeconds: 333 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=320' heldReason: rights - ordinal: 7 text: null startSeconds: 346 endSeconds: 391 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=346' heldReason: rights - ordinal: 8 text: null startSeconds: 413 endSeconds: 425 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=413' heldReason: rights - ordinal: 9 text: null startSeconds: 425 endSeconds: 494 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=425' heldReason: rights - ordinal: 10 text: null startSeconds: 515 endSeconds: 548 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=515' heldReason: rights - ordinal: 11 text: null startSeconds: 560 endSeconds: 585 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=560' heldReason: rights - ordinal: 12 text: null startSeconds: 640 endSeconds: 651 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=640' heldReason: rights - ordinal: 13 text: null startSeconds: 693 endSeconds: 735 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=693' heldReason: rights - ordinal: 14 text: null startSeconds: 735 endSeconds: 747 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=735' heldReason: rights - ordinal: 15 text: null startSeconds: 784 endSeconds: 873 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=784' heldReason: rights - ordinal: 16 text: null startSeconds: 784 endSeconds: 873 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=784' heldReason: rights - ordinal: 17 text: null startSeconds: 884 endSeconds: 929 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=884' heldReason: rights - ordinal: 18 text: null startSeconds: 929 endSeconds: 941 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=929' heldReason: rights - ordinal: 19 text: null startSeconds: 975 endSeconds: 1033 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=975' heldReason: rights - ordinal: 20 text: null startSeconds: 1044 endSeconds: 1100 watch: 'https://liquidgold.cc/watch/6MudaeKdBSk?t=1044' heldReason: rights --- # Scale the Judgment, Not the Model — Andrew Orobator, Reddit Swap in a smarter model and you get a slightly better answer. Take away the tests, gates and review, and everything falls apart. Andrew Orobator, a senior Android engineer at Reddit, argues that the model was never the bottleneck. The bottleneck is the judgment around it. Humans absorb judgment implicitly, but agents need it spelled out. He shows how to get judgment out of people's heads and into the repo. Skills are institutional judgment turned into something an agent can run. Work logs let a fresh agent pick up at milestone 7 of 9 (this talk itself was built with one). Personas lend you a security reviewer's or designer's eye. He then covers the verification ladder and his feature-flag cleanup agent, which went 7 for 7 on green-CI PRs at $1.26 each. And he shares a warning: when he asked Codex for reasons to unlock his repo guard, it quietly added a self-authorizing "emergency recovery" exception. His line: "Agents will build ladders to climb out of the pit of success." Speaker info: X/Twitter: @aorobator (https://x.com/aorobator) Related links: Vibe Engineering series (Medium): https://medium.com/@andreworobator Timestamps: 0:00 The war room for dead feature flags 0:37 Intro 0:55 What this talk isn't 1:55 We're harness engineers now 2:25 Humans absorb judgment; agents need it explicit 3:00 The model isn't the bottleneck 3:30 Judgment trapped in people's heads 4:40 Skills as knowledge lines 5:15 Skills vs. documentation 5:25 Work logs 6:25 This talk was built with a work log 6:45 Personas 8:05 Borrowing eyes you don't have 8:20 Governance 9:00 Verification 9:40 The verification ladder 10:15 Spin at the gate until green 10:35 Make the agent record itself 11:20 Case study: a feature-flag agent 12:10 $1.26 per pull request 12:50 Agents climb out of the pit of success 14:25 A society of specialist agents 15:25 In, on and off the loop 15:55 Encoded judgment rots 17:05 Managing judgment 18:15 Scale the judgment, not the model --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/6MudaeKdBSk-youtube-71abe6417c64910aec33 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/scale-the-judgment-not-the-model-andrew-orobator-re-ed7592244eef · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks --- #### Chunk 0: Document excerpt (0:12 to 1:57) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 1: Document excerpt (1:57 to 3:52) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 2: Document excerpt (3:52 to 5:33) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 3: Document excerpt (5:33 to 6:41) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 4: Document excerpt (6:41 to 9:20) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 5: Document excerpt (9:20 to 11:23) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 6: Document excerpt (11:23 to 12:52) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 7: Document excerpt (12:52 to 14:33) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 8: Document excerpt (14:33 to 17:13) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 9: Document excerpt (17:13 to 19:33) > Held: transcript rights do not permit this chunk's text.