A talk from AI Engineer Paris 2026 (Coding Agents & Software Factories track) in which Matt Pocock examines why pull requests remain the bottleneck in software development, even as AI agents make opening PRs trivial. He argues that agents make it only slightly easier to open a PR worth reviewing, that code is the environment an agent works in, and that agent experience (AX) deserves the same rigor developer experience (DX) received — meaning the fix lies in what happens around the PR, not in how fast it gets written. # Video Identity Observations — "Fixing the PR Bottleneck" (Matt Pocock, AIHero) ## Observable Presentation - **Conference talk format**: This is a live technical conference presentation, delivered to an in-person audience ("Raise your hands if you've used this skill, by the way... Yeah, okay" and closing with "Thank you, Paris"). - **Creator identity**: The speaker is **Matt Pocock**, associated with **AIHero** per the title, and promotes a skills repository at "aihero.dev/skills" ("these are my skills. I'm here at dot dev \slash\ skills, and we're shipping version 1.3 this week"). - **Product-driven content**: The talk is structured around announcing and demonstrating the creator's own skills ("I have a couple of new skills to announce that are going to hopefully improve the way that you do PRs"), including a **code review skill**, a **code base design skill**, an in-progress **PR skill**, and a new **retro skill**. ## Intended Audience - **Engineering organizations struggling with AI-era code volume**: "a grand title for trying to fix the thing that most organizations struggle with... we have always had huge numbers of PRs just laying around that no one's bothered to review." - **Practitioners using AI coding agents**: Assumes familiarity with agent workflows, context windows, sub-agents, and terms like "slop cannon" and "software factory" — "probably the biggest buzzword of the day." - **Teams, not solo hobbyists**: Advice framed for group settings — "Share them across your team," "a consistent language that you can use in your team to talk about this stuff." - **Professional engineers with legacy context**: References red-green-refactor, John Ousterhout's *A Philosophy of Software Design*, mocking pitfalls, and AWS one-way/two-way door terminology ("for the retro developers among us"). ## Claimed Scope - **Core claim**: The PR bottleneck is a structural problem requiring a layered "brakes" system: **automated checks → automated review → human review** — "more speed means more PRs... the goal here is to make human review faster by leaning on those first two phases." - **Prescriptive opinionated stance**: The creator frames strong recommendations as personal positions ("my first piece of advice is don't put coding standards in your implementer agent"; "don't outsource automated review. Build your own"; "The reviewer should commit. It should make fixes"). - **Claimed efficacy of own tools**: "You can have like the weirdest vibe coded like code base... and you can run this skill on it and it will make it better"; the retro skill creates "the compounding effect where... by doing human review, you're making the quality of the next human review higher." ## Self-Description Evidence - **Acknowledges prior related work**: "I've talked about this before in previous AI engineer talks I've given, which are deep modules." - **Positions self as a skills ecosystem maintainer**: Active development cadence and community awareness ("maybe you've used them"; "shipping version 1.3 this week"), plus credit to others ("show me skill from the human layer skills repo by Dex Hadley"). - **Hedged, self-aware tone**: "this is kind of a grand title"; "I've been mulling over a long, long time. Haven't quite figured out what the state of the art is"; "I think of this, and this is uncomfortable." ## Notes - No curated metadata was supplied beyond the title; audience/scope inferences above are grounded in transcript evidence only. - This is a single conference talk; channel-wide conclusions (e.g., that the channel is exclusively about AI-assisted engineering) are not warranted from one video, though the demonstrated scope here is clearly **AI-agent engineering workflow tooling**. # Delivery Observation Report — "Fixing the PR Bottleneck" (Matt Pocock, AIHero) ## Overall Impression A conference talk with a **casual, conversational register** layered over structured technical argument. Evidence: opens with "All right. I'm here to talk about..." and closes with "It has been glorious hanging out with you... Thank you, Paris." Frequent self-corrections and mid-sentence reformulations ("improve the way that you do PRs, improve the way that's or improve the speed at which...") suggest live, unscripted delivery. ## Tone & Register - **Informal intensifiers and mild profanity** recur: - "you're going to end up with a slop cannon... crappy PRs that you're not going to be able to touch or review, or even freaking look at." - "It's really freaking nice." - "the weirdest vibe coded like code base, the crappiest code base that you've ever set your eyes on." - **Self-aware humor about the topic's framing**: "this is kind of a grand title for trying to fix the thing that most organizations struggle with." - **Direct address to the audience** appears multiple times, though the pattern is thin in this transcript (see uncertainty below). ## Vocabulary & Recurring Terms - **"Slop"** as a signature term: "you're going to end up with a slop cannon," and later "we're going to stop the slop." - **"Bottleneck" / "PR bottleneck"** — titular framing, reinforced: "fixing the PR bottleneck. So many folks say, 'Oh yeah, we just use a third-party service...'" - **The acceleration/brakes metaphor** is the talk's central extended image: "these accelerate your software factory... But then you need brakes, right? If you just have permanent acceleration... you need brakes." - **"One-way door / two-way door"** borrowed AWS terminology, with a comic contrast: "the glorious benefit of being a software engineer as opposed to a civil engineer... If you get something wrong, that bridge is going down." - **"Blast radius"**: "we need to understand radius of this PR" — blending "blast radius" and "radius" mid-phrase. ## Teaching Habits - **Rhetorical question → answer pattern** is heavily used as a teaching device: - "Does a green CI mean that the code is ready for merge? No, it does not." - "Can anyone guess the test that was written to test this behavior, right?" - "But how do we categorize them?" - "So, what does a good PR body look like?" - **Concrete anti-pattern examples**: he builds arguments around named failure modes — "tautological tests," "tests that literally cannot fail" — each with a real code anecdote (the "X post character limit equals 280" test that just reasserted the implementation). - **Layered/stacked mental models**: recurring three-part structures — three "brakes," three review layers forming "this kind of cake," implementation budgets split into explore/change/debug. - **Analogical mapping to prior practice**: "for the retro developers among us, this is essentially a red-green-refactor approach"; "I was using seams before they were cool." - **Named attributions** to sources of ideas: "This is a John Ousterhout idea from the philosophy of software design," "the show me skill from the human layer skills repo by Dex Hadley." - **Audience interaction moments**: "Raise your hands if you've used this skill, by the way," "Does that make sense? Yeah?" ## Communication Patterns - **Rule-like imperatives** delivered as punchy one-liners: - "Stop trying to one-shot good code." - "Don't put coding standards in your implementer agent." - "Don't outsource automated review. Build your own." - "You never want to write the same comment twice." - **Contrast pairs (X vs. Y)** structure most arguments: implementer "overloaded" vs. reviewer "underloaded"; deep module vs. wide interface; one-way vs. two-way door; checks as liars vs. review as "lie detectors." - **Self-deprecating / persona-building asides**: "I've talked about this before in previous AI engineer talks I've given," "I've been talking to lots of people today." - **Product-forward but framed as process**: skills are introduced as artifacts ("This is my code review skill... This is called retro") embedded within a principles-first narrative. ## Uncertainty & Thin Evidence - **Audience engagement** ("raise your hands," "does that make sense") appears only twice; frequency in the live talk may be higher than the transcript captures. - **Tone consistency across the full talk** can't be fully verified — repetition of the profanity-adjacent intensifiers ("freaking") is noted twice; treating it as stable but not constant. - The "lie detector" metaphor appears once explicitly; its status as a recurring motif is unconfirmed. The **brakes/acceleration** metaphor, by contrast, is clearly load-bearing (stated at setup and invoked at close: "review it a little... as painless, as simple"). - Isolated moments include the [sighs] reaction ("uh so, [sighs] what your code review agent is doing...") — a single observable beat, not a pattern. # Observation Record — "Fixing the PR Bottleneck" (Matt Pocock, AIHero) ## Speaker - **Matt Pocock** (AIHero) — sole speaker identified in the transcript; presenting at a conference (thanks audience "Paris"). ## Central Thesis - Organizations have historically struggled with unreviewed PRs, and AI has **massively increased** the strain ("we have always had huge numbers of PRs just laying around that no one's bothered to review"). - He proposes three "brakes" on the AI-driven "software factory," arguing they let teams **counterintuitively go faster**. ## Claims on the "Software Factory" - Defines a software factory as one where **agents perform some of the work initiation**, with deterministic code (classifiers, hooks like Planet Scale query reports) triggering downstream steps rather than humans. - **Acceleration without brakes = "slop cannon"**: uninterrupted acceleration produces low-quality PRs no one can review. - Key justification: *"code is the environment your agent operates in. And if you have bad code in your code base, that is going to beget more bad code."* ## Three-Layer Review Model 1. **Automated checks** — deterministic (linting, tests, type checking, code quality metrics). 2. **Automated review** — agents catching "things the tests didn't catch." 3. **Human review** — the final layer. - Goal: make human review faster by leaning on the first two layers; raising shipped code quality reduces the need for human intervention ("stop the slop"). ## Claim 1: Automated Checks Are Cheap but Can Lie - **Cheapness**: they "just cost CPU cycles," unlike tokens or human effort; teams are "probably not using enough of them." - **Qualification**: test-fixing does consume tokens, but those are "tokens pretty well spent." - **Green CI ≠ merge-ready**: human and automated review act as "lie detectors" on the checks. ### Documented failure modes (from his own agents' output) - **Tautological tests**: a test re-asserting the implementation (X post character limit 280 → `expect(X_POST_CHARACTER_LIMIT).toBe(280)`). He notes **"Opus 5 got addicted to these"** and admits *"I don't quite understand why."* Problem: extreme structure sensitivity — can't change or rename constants without failures. - **Egregious structure-coupled test**: a test reading source files into memory to assert UI element ordering in the source text rather than rendered behavior. - **Tests that cannot fail**: over-mocking (e.g., stubbing the Audio Context API) hides complicated error modes, surfacing as production errors. - **Qualification**: *"the AI isn't trying to write bad tests here. It's just taking our instructions"* — bad tests are framed as good-faith outputs of bad constraints, not deception. ## Claim 2: Design Your Way Out of Bad Checks - **Deep modules** (attributed to **John Ousterhout**, *A Philosophy of Software Design*): complex behavior behind simple interfaces → fewer structure-sensitive tests; force agents to test at the interface. - Promotes his **codebase design skill**, producing an HTML before/after document of "opportunities for deepening modules," plus a defined vocabulary: **locality, leverage, seams** ("I was using seams before they were cool"). Motivation: existing literature offers "20 different approaches and they all going to be called DDD." ## Claim 3: Don't Put Coding Standards in the Implementer Agent - Mental model: implementer context is budgeted across **explore → change → debug**; imposing coding standards makes it perform worse ("implementation is overloaded"). - Review, by contrast, is **"underloaded"** — so his **code review skill** receives a diff, reads a customizable `coding-standards.md`, and runs in a **sub-agent with its own context window**. - Two-part process: **implement (make it work) → code review (make it good)**, which he likens to **red-green-refactor** across context windows. He flags this as **"uncomfortable"** because everyone wants good code first-shot. - Standards belong in `coding-standards.md`, **not global scope / agents.md**, where they "drown out your implementer." - **Against outsourcing automated review**: generic third-party reviewers (Cursor Bug Bot, CodeRabbit) either give irrelevant false positives or are too language-specific. Advice: **"Build your own. Build up your own coding standards over time."** - **Reviewer should commit fixes, not just comment**: PR comments merely create more work for the human; commenting only for genuine questions, with commits as default. "Stop trying to one-shot good code." ## Claim 4: Human-Friendly PRs New **PR skill** (in progress, "stealing the best ideas"), with three principles: 1. **Not all reviews matter equally** — classify via AWS one-way/two-way door framing: - Most PRs are two-way doors (revertible) — light review. - **Nuance**: a simple change could "blast out an email to 60,000 people," making it effectively one-way; expensive migrations or data loss demand heavy review ("Review the hell out of that PR"). - Each PR gets a **merge danger** summary (door type + blast radius). 2. **Pseudo code / visuals for the "why"** — credits the **"show me" skill from the Human Layer skills repo by Dex Hadley**; Mermaid diagrams and simple visual summaries "make understanding the why as fast as possible." 3. **Review the system, not just the code** — "the process that produces your code is just as important as the code itself"; goal: *"never write the same comment twice."* ### Retro skill - New **"retro"** skill: runs retrospectives on sessions/PRs (even a week's worth) and suggests **automated checks, coding-standards.md updates, navigation pointers** (in agents.md), **tool economy** improvements (token efficiency), and **bloat reduction** in steering files/skills. - Framed as a **compounding effect**: human review raises the quality of the next review and reduces future work. ## Closing - Skills repo at **dot dev/skills**, shipping **version 1.3** "this week." - Final framing: make human review "as painless, as simple, and as kind of optional as we need it to" — reviewing every two-way door is unnecessary; every one-way door is not. ## Tensions / Qualifications Preserved - Checks are "cheap" **but** their failures cost tokens — acknowledged and accepted. - The reviewer-committing-fixes model is **"uncomfortable"** and contradicts the desire for one-shot quality, which he explicitly tells the audience to abandon. - The PR skill's "state of the art" is **unresolved** — he admits he hasn't figured it out and is still gathering ideas. - Automating review via generic tools is presented as a **disputed area**: many attendees reportedly use third-party services, which he argues against. # Video Attribution & Provenance Record ## Speakers & Roles | Speaker | Role | Basis | |---|---|---| | **Matt Pocock** (AIHero) | Creator / conference presenter | Title metadata: "Fixing the PR Bottleneck — Matt Pocock, AIHero"; transcript is a first-person conference talk | - **No guest speakers, interviewers, or narrators** appear; the transcript is a single continuous talk. - Audience questions or comments are not present in the transcript (rhetorical audience prompts only, e.g., "Raise your hands if you've used this skill"). - Setting: a conference talk (closing references: *"It's been a really nice conference… Thank you, Paris."*). ## Referenced Third-Party Sources & Ideas - **John Ousterhout** — *The Philosophy of Software Design*; source of the "deep modules" concept. - **Dex Hadley** — "show me" skill in the **Human Layer skills repo**; speaker credits it for PR visualization ideas (*"points of gratitude here to the show me skill"*). - **AI engineer talks** — speaker references his own previous talks at this conference series on deep modules. - **AWS terminology** — one-way vs. two-way door decisions framework. - **Third-party tools/services mentioned** (as claims/observations, not endorsements by us): Cursor bug bot, CodeRabbit, Planet Scale (slow query reports), "Jeff" classifier, "software factory" buzzword. ## Self-Promotional Segments — ⚠️ Flagged The talk promotes the speaker's own products. Provenance note: recommendations are tied to his commercial offering. - **aihero.dev/skills** (rendered as "dot dev slash skills") — his skills repo; **version 1.3 shipping that week**. - **New skills announced in the talk:** - Codebase design skill (module-deepening, HTML opportunity reports; defines locality, leverage, seams) - Code review skill (reads `coding-standards.md`, runs in a sub-agent, commits fixes) - PR skill (in progress, "releasing soon"; merge danger summaries, pseudocode/diagram PR bodies) - **Retro** skill (retrospectives suggesting checks, standards, navigation pointers, tool economy, bloat analysis) - Mentions of previous skills ("raise your hands if you've used this skill") serve audience engagement for the same product line. ## Referenced Artifacts (speaker's own materials) - `coding-standards.md` file convention (contrast with `agents.md`) - Claims about model behavior: **"Opus 5 got addicted to" tautological tests** — third-party claim about an Anthropic model, unverifiable from transcript - Example code claims (X post 280-char tautological test, file-order test, `useAudioBoost` mocking example) presented as *"real code… that I found my agents doing"* ## Availability & Access Limits - The **PR skill and Retro skill availability**: PR skill described as "in progress" / "releasing soon"; Retro described as "a new skill." Exact release status unverifiable. - **No metadata supplied** beyond title; no comment availability, publication date, or venue name confirmed (venue inferred only from "Thank you, Paris" and conference context). - Slides/diagrams referenced in speech are **not available** — only the transcript. ## Uncertainty Notes - Skill names are transcribed phonetically in places (e.g., "dot dev \{slash\} skills"); exact URLs not verifiable. - "Jeff" classifier reference is unclear — may be a tool or person name; flagged as ambiguous. - No second speaker ever appears; all quoted dialogue ("we use a cursor bug bot") is paraphrased speech by the presenter. All right. I'm here to talk about fixing the PR bottleneck and this is kind of a grand title for trying to fix the thing that most organizations struggle with, I think, and have kind of historically struggled with before AI, you know, we have always had huge numbers of PRs just laying around that no one's bothered to review and this is now increased massively because of the new strains on us because of AI imposing all these weird constraints. And to do this, I'm going to use the rubric of my skills, which is kind of heard about, maybe you've used them. And I have a couple of new skills to announce that are going to hopefully improve the way that you do PRs, improve the way that's or improve the speed at which you can review them and do them. Speed. Now, we've been pushed to do more with less, essentially, or more PRs, more work, more stuff. And this is the kind of the central promise of AI, that we're going to be able to use these agents to scale ourselves up to do more work. And this has resulted in the software factory, probably the biggest buzzword of the day. Everyone's talking about software factory that I have chance to. And I think of a software factory as primarily something where instead of the human initiating all of this work, we're going to pass some of that initiation, some of the initiation is going to be done by agents. And I think of that Maybe from there you have a classifier like Jeff come in and say, "Okay, let's turn that into a fix or turn that into a reproduction." Or maybe I ping people straight away. And maybe you have other things. Maybe you have Planet Scale hooked up, so it gives you query reports on the slow queries on your database. Maybe that then triggers a different thing of your software factory. All of this is not humans triggering it, it's uh deterministic code triggering it, right? And so these accelerate your software factory. They push more code through it. But then you need brakes, right? If you just have permanent acceleration pushing stuff through your factory, you're going to end up with a slop cannon, right? You're just going to end up with a ton of slop crappy PRs that you're not going to be able to touch or review, or even freaking look at. So you need brakes. These are mechanisms that slow down, that increase quality, that make sure that your code base doesn't turn into a software entropy nightmare, because code is the environment your agent operates in. And if you have bad code in your code base, that is going to beget more bad code. And so I'm going to talk about these three brakes in this talk, and talk about how we can use them to counterintuitively go faster. So what are automated checks? These are the deterministic checks in your repo that we've had for thousands of years, or you know, since the '50s. Deterministic checks, where we have linting and tests and type checking and code quality metrics. All of these things going together, and they all work the same every time. Layered on top of that, we have automated review. So we have agents who look at our code and say, "Okay, you know, these are for the things that the tests didn't catch, or this is looking at the structure of the code base in general." And then on top of that, the third layer, the final layer, is human review. So people looking at the PR. And these three layers form this kind of cake that we end up with when we get to human review. And so more speed, of course, means more PRs. And so the goal here is to make human review faster by leaning on those first two phases. And we're going to stop the slop. That's the first principle here, which is if you raise the quality of the code that you're shipping, you're going to end up doing less human review because it's just going to be better work, and so you're going to end up needing to uh make fewer interventions. So, automated checks. Now, automated checks are cheap. That's the cool thing about them is that they don't cost tokens like automated review does, they don't cost human effort, they just cost CPU cycles. So, these are for instance, you know, you run your tests on every code change. Maybe those tests do incur some tokens because, let's say um an agent uh creates a bug and the test catch it, then you need to spend some tokens to go and fix it. But, those are tokens pretty well spent in my opinion. So, checks are cheap. That means you can layer on loads and loads and loads of them on your repos, and you're probably not using enough of them or not being creative enough with your use. But, checks can lie. Does a green CI mean that the code is ready for merge? No, it does not. And so, we've always needed, on top of these checks, some extra layer to figure out if there's anything catastrophically wrong with the code before we ship it. And so, all of the other phases, the human review and automated review, these are lie detectors. These are for finding lies in the automated checks. Now, I want to show you some of these lies first of all because this helps when we're thinking about code and thinking about automated checks to see how bad it is and how bad things can get. The first is tautological tests. A test that just reasserts the implementation. Opus 5 got addicted to these. I don't quite understand why. It would have, for instance, X post character limit equals 280. Can anyone guess the test that was written to test this behavior, right? You've probably seen this a thousand times. This is real code from agents or from stuff that I found my agents doing. it said expect X post character limits to be 280. So, the implementation looked like that. And the test essentially reasserted the implementation. That is a tautological test. And tautological tests are bad because they're extremely structure sensitive. They're very sensitive to the actual internal workings of the system. So, it means I cannot change that constant without a test failing. But I cannot rename that constant without a test failing. Like I have to do it's so tied into the structure of my system. And I found another one which is even more egregious, I would say. This is incredible uh test. What it's doing here is it's essentially testing whether two things in the UI appear in the right order. So, it's checking that the pitch detail page uh or rather the video section comes after the content plan. What it does it doesn't render it to a screen, it just reads the actual file, the module, into its own memory, and then it finds the right thing, so finds content plan, finds videos, and then it expects the videos to be after it in the source material. Which is crazy if you think about it cuz I can just like change the way the source material looks and this test will fail. It's too sensitive to the structure of my code base. So, that's another way that automated checks can fail. And there are also Oh, sorry. Automated checks can lie. There are also tests that literally cannot fail. And this will feel familiar to you if you've abused mocking in the past or have used various things. For instance, here we have a use audio boost function. And this use audio boost function, uh its internals use the audio context API in the DOM. Don't worry if you don't know any of this. But what we're doing here is we're just stubbing it out with some fake methods. And it turns out the audio context has some complicated error modes and it will fail if you use it under strange conditions. And so just doing this means our tests cannot fail using those modes and we're going to hit strange errors in production that our tests can't fix. And so the question is then if you can cheat on automated checks, if you know, and even in good faith ways as well, the the AI isn't trying to write bad tests here. It's just taking our instructions and writing tests that are too tied into the structure instead of actually executing code. So how do we make automated checks harder to cheat? If we can do that, then we can increase the quality of those checks, which means what's increases our quality bar. And the first thing I really like about this is code base design. So you can actually design your way out of these bad checks. So what does good code base design look like? I've talked about this before in previous AI engineer talks I've given, which are deep modules. These are modules that hide complex behavior behind simple interfaces. This is a John Ousterhout idea from the philosophy of software design. If we look at these two modules we've got A which has a large implementation hiding behind a tiny little interface at the top. Okay? And then B is a large interface, lots of functions you can call and those functions individually don't do very much. Does that make sense? Yeah? Now if you have a deep module like A here, you're going to have fewer structure sensitive tests because you're hiding more of the implementation behind that interface. If it's just testing at that interface, you're going to get better tests. And so that your job here is to force the agents to use that little interface instead of reaching into the implementation to test these weird implementation details. So I've got to go for this. You can have like the weirdest vibe coded like code base, the crappiest code base that you've ever set your eyes on, and you can run this skill on it and it will make it better. What this does essentially gives you opportunities for deepening modules. And kind of looks like this. Raise your hands if you've used this skill, by the way. Not sure how many of my folks are in this room. Yeah, okay. It's really freaking nice. Essentially, it gives you a HTML document. I'll get out of the way here. Of all of the different um potential opportunities it sees. So, we can see here we have a before and we have an after where we're sort of like reducing duplication, we're creating a nice deep testable module. And then you can go ahead and implement that. And attached to this, there's also this kind of language that I've put together for describing modules because like if you ever try and read up about how to structure a code base, you're going to find 20 different approaches and they're all going to be called DDD. And like what you need is a consistent language that you can use in your team to talk about this stuff. And so, I have a little code base design skill that defines what locality is, defines what leverage is, defines what a seam is. I was using seams before they were cool. And what locality means is kind of how uh well located together all of the code is. How can you change like a small change in one module and have it ripple out? And also leverage is what you get when you have a deep module because the caller, the person who's actually calling that module, gets a lot of value of calling a simple function. Both of those are very good in code bases and good for agents, too, it turns out. But I'm sort of describing all of these high-falutin coding standards, but how do we actually make sure the agent does them, right? How do you make sure the agent creates deep modules and creates good tests and doesn't write these crap tautological ones or structure sensitive tests? Well, I do think most people get this wrong. And my first piece of advice is don't put coding standards in your implementer agent. Okay? Let me explain this. If you imagine the implementer agent kind of looks like this. Where this is all of the things the agent needs to be able to do in its single context window. It needs to be able to explore, like to look for the code that it's going to change. It then needs to actually change it, so make the updates to the files in the green. And then it needs some budget for actually debugging the thing. So if we're running those automated checks for actually checking and verifying that it works. Now this is quite a lot of work, it turns out. And if you try to impose your coding standards on it as well, it's going to perform worse. So implementation is overloaded. That's the mental model I want you to have. And so can we find a way to impose those coding standards in a way that isn't so overloaded? Well, this is my effort. This is my code review skill. And it receives a diff. And it reads a file inside the repository called coding standards, which you can write, you can customize. And then it checks if the code follows those standards. And so if you look at the reviewer agent here, it also crucially runs it in a sub agent. So it's got its own context window to kind of handle here. It's got its own budget. This one, it needs to do some exploration, right? Because sure, it receives the diff, so it knows exactly where it's located, where the code is. But it should probably do a bit of exploration just so it has the wider context, understands the code. But it doesn't need to do any implementation. Doesn't need to do any debugging. So while implementation is overloaded, review is actually underloaded. So it doesn't have that many jobs to do. This means you can pile in a bunch of coding standards to it, and And will do a much better job than if you try to do it with implement. So, I think of this, and this is uncomfortable, right? Cuz we we all want to be able to just get good code out the first time. But I think of this as the two-part process for writing good code, which is implement, you make it work, and then code review, you actually make it good. You impose your coding standards. And for, you know, the retro developers among us, this is essentially a red-green-refactor approach. We use one context window to make it okay, do the red-green, and then we do another context window to refactor it. That's at least how it works in my head, and it's been very successful for me. This means that you when you have coding standards, you don't put them in global scope. You don't put them in agents.md, cuz then they sort of drown out your implementer. It may read them, it may not. You put them in coding standards.md, and that way just the code review agent does it. Now, I've got another idea here, which is I've been talking to lots of people today. Lots of people saying, you know, I've been talking about review and automated review, increase, you know, fixing the PR bottleneck. So many folks say, "Oh yeah, we just use a third-party service. We use a cursor bug bot. We use code rabbit or something like that." I think that I've I've really tried to make a generic code review skill in the past that finds all the bugs and does security review and that kind of thing. It turns out it's really, really hard because you either make it too general and it just gives you false positives that aren't actually relevant to your use case, or you make it too specific. You say, "Okay, find all the TypeScript stuff," and then Rust people can't use it. So, I would say don't outsource automated review. Build your own. Build up your own coding standards over time. Share them across your team. And if you have an opportunity to impose coding standards, if you've got some docs sitting around that no one reads, this is the place to put them in. And also, when you have this automated reviewer, a really natural inclination for lots of people is to say, "Oh yeah, my code review agent, what it does is it reads the code and then it comments on the PR." Uh so, \[sighs\] what your code review agent is doing in that case is it's providing more work for the human reviewer. The human reviewer then has to read all of these verbose comments and figure out, "Okay, do we implement this? Do we implement that?" The reviewer should commit. It should make fixes. So, it should actually change the things that it finds. Cuz then when the human comes around, you're reviewing a really nice artifact. If it finds anything that it has any questions over, then of course it can comment, but the default should be commits. Stop trying to one-shot good code. Stop trying to make the implementer agent the only thing that you do and go, "Okay, I'm going to force it to be amazing." It takes a little bit of, you know, thinking your way out of there, but once you realize it, it is fabulous. So, okay. With all of that process, we've run our automated checks, we've now run our automated review to make sure the automated checks aren't lying. How do we then maximize the PR's quality in terms of human review? How do we get it like working the best it can? So, we need a human-friendly PR. And this is a new skill coming into the repo, which is currently in progress, but I'll be releasing it soon, which is the PR skill. This is one I've been mulling over for a long, long time. Haven't quite figured out what the state of the art is. And I've realized the best way to make a PR skill is just to steal all the best ideas that everyone's got. And it's very nice. Now, what does a good PR body look like? How would you recreate this skill on your own? Well, the first principle is some reviews are more important than others. Not every review is essential, right? And once you understand this, you realize, "Okay, that means I can focus my energy on the really important reviews. But how do we categorize them? Well, you got to think about the PR as using this AWS terminology, which is is it a one-way door or is it a two-way door? Now, most PRs that you have will be two-way doors. You can merge the PR and then always pull it back later. It's the glorious benefit of being a software engineer as opposed to a civil engineer, right? Mostly when you're a civil engineer, it's a it's a one-way door, right? If you get something wrong, that bridge is going down. But, if you have a two-way door, it means that you can easily revert the change. Now, that might be more um a bit more nuanced than you might expect. It might be that a very simple change accidentally blasts out an email to 60,000 people or something, in which case that is a one-way door. You want to review that very, very carefully. Involves expensive migrations or data loss, that is a one-way door. Review the hell out of that PR. But also, tied onto that, we need to understand radius of this PR. What can go wrong? And if things do go wrong, how bad is it? And this means I end up with a nice little sort of summary right at the bottom of all of my PRs, which is the merge danger. I can see this one is a two-way door. Its blast radius is localized, and so I can see fantastic, I don't need to pay that much attention to this. I'm just going to sort of review it a little. That's really important. Next, we need to understand what the PR is even doing, right? And I've tried lots of different ways of figuring this out, and the best thing I've come up with is using pseudo code. Now, a huge um points of gratitude here to the show me skill from the human layer skills repo by Dex Hadley. This is a phenomenal skill that just essentially dispenses with most text and shows things to you in images and diagrams instead. This makes it a lot easier to grasp actually what's changing and why it's changing. So, you get the kind of standard sort of set of like mermaid diagrams and UML for, you know, this is a sort of sequence of things that happened. You also just get these lovely simple ones like this, for instance. We can look at this and go, "Okay, we're working in a CLI. We can see a new command has been added." And we've got two new things, little flags up the top here. Just little summaries like this. It really does make a massive difference. It's hard to overstate. So, you're trying to like make understanding the why as fast as possible. And I think a third principle here is something you should be thinking about whenever you do human review. Because because we're not doing like um because our processes now are so sort of streamlined and all we're all collaborating around these same skill files, these same steering files. We're all building an environment for our agents to operate in together. You should think of the process that produces your code as just as important as the code itself. In other words, when you do a human review, you're not just reviewing the code, you're reviewing the system that creates it. And the theory here is that you never want to write the same comment twice. Right? You never want to catch the agent doing the same thing over two PRs. And so, what's the mechanism by which you can make your human review matter? Well, this is a new skill. This is called retro. Retro for retrospective. You essentially take a session that you've done. It can either be like a single um agent session, or it can be a PR plus the session, or you can just get it to look at Okay, look at all of the PRs that we've done over the last week, all of the reviews, pull them all in. Let's do a retrospective on them. And it will suggest automated checks and coding standards to make the next one better. So, this is the compounding effect where you're essentially by doing human review, you're making the quality of the next human review higher, and you're sort of saving less work from yourself next time. And Retro is a really smart skill. It adds a bunch of stuff. So, obviously, it suggests automated checks. It suggests updates to coding standards.md. It does other smart stuff, too. So, it looks at navigation pointers. How easily did the agent find its information? Can we provide a pointer inside agents.md to help it out next time? It looks at tool economy. Are there different tools that we're using in the session that can, you know, could be made more token efficient? It's amazing how many things this catches, actually, cuz those are often really hard to debug from the outside. It just looks at bloat, as well. So, are there bloated steering files? Are there bloated skills that contribute to these bad results? Can we make them more organized? So, that's the goal. It's to make human review faster. We do that by layering up automated checks. We're layering up automated review, and we make the human review as painless, as simple, and as kind of optional as we need it to. You really don't need to review every single two-way door. Every single one-way door you do. So, these are my skills. I'm here at dot dev \{slash\} skills, and we're shipping version 1.3 this week. It has been glorious hanging out with you. It's been a really nice conference. I'm going to be outside in the lobby if anyone wants to have a chat. Uh thank you so much for having me. Thank you, Paris.