Talk by Greg Pstrucha, Staff Software Engineer at Sentry, arguing that repeated corrections to coding agents should be codified into deterministic checks instead of re-prompted. He says agents have no memory, so mistakes such as overly defensive code, useless tests, and unneeded endpoints recur. He proposes tests, strict types, and linters as enforcement, and says custom lint rules are now cheap to write with agents. Examples from Sentry include lint rules that keep endpoints in sync with an OpenAPI schema and linting of agent skills. For what linters cannot catch, he suggests written policies and a "Grumpy Engineer" review agent, and warns that agents game metrics such as test coverage. Tools mentioned include ESLint, ast-grep, and Taskless. Recorded at the AI Engineer World's Fair 2026. These are the speaker's arguments and examples, not independently verified results. # Video Identity Observations — "Stop Prompting — Greg Pstrucha, Sentry" ## Speaker & Affiliation (explicit self-description) - The speaker identifies himself directly: > "my name is Greg. I'm an AI engineer at Sentry. I work on the infrastructure that powers seir centuries uh debugging agent" - His employer is confirmed both by self-introduction and by the title ("Greg Pstrucha, Sentry") and by repeated in-talk references to Sentry's codebase and **Sentry Agent** ("sir agent is a debugging agent and it has to talk to Sentry a lot"). ## Intended Audience Evidence points to **professional software engineers working with AI coding agents**, not general consumers: - Assumes familiarity with agent workflows and code review: > "who still reads code that AI generates?" - Presumes knowledge of engineering tooling and practices: linting ("oxlint flake 8 clippy as grabb"), strict typing, hooks, evals, Django/DRF, OpenAPI schemas, cyclomatic complexity, test coverage. - The call to action is directed at practitioners with their own repos and agents: > "take your laptop go to your agent and ask hey look at all my past transcripts look at my GitHub reviews" - Presentation format: a live conference talk with audience interaction ("Yes." / "Hooks. Yes." — speaker responding to audience callouts), indicating a developer-conference setting. ## Claimed Scope / Thesis - The speaker frames a deliberate, provocative thesis: > "I have a little rage baity title of the presentation that calls to stop prompting but I do not mean loop engineering" - Core claim: > "you should stop prompting. You should start codifying the actual rules that the agent should be following" - Scope covers **deterministic enforcement** (tests, strict typing, linters, hooks, evals), **agent-generated lint rules**, **repository policies enforced by an LLM reviewer** (a "grumpy engineer" skill, "Garfield" internally), and a **warning about metric gaming**: > "when you give a number to the agent, the agent will optimize the out of that number" - Closing call to action: > "try to ask your agent for llinters and policies and see whether that helps you. Um, try things and make the repo remember." ## Demonstrated (vs. Claimed) Presentation Patterns - **Demonstrated**: heavy reliance on first-hand Sentry engineering experience — mature-codebase linting (Django/DRF/OpenAPI schema drift), skill-generation whack-a-mole story, and an internal policy-loop sub-agent workflow. - **Claimed, not demonstrated in-transcript**: slide content, live demos, and the speaker's scheduled routine of mining transcripts for lint rules are described verbally but not verifiable from the transcript alone. - The speaker explicitly limits his claims' scope: > "the goal here isn't to remove the code review quite yet... but more so to raise the floor" ## Self-Check All observations are tied to quoted or precisely described transcript passages; no channel-wide conclusions or private motivations were inferred beyond what the speaker stated. # Delivery Observation: "Stop Prompting" — Greg Pstrucha, Sentry ## Overall Register Conversational, war-story-driven technical talk. Heavy use of **first-person anecdote** and **live audience interaction**, with a deliberately provocative framing ("a little rage baity title") that he immediately softens. ## Tone & Emotional Coloring - **Performed frustration as a rhetorical device.** He repeatedly frames agent failures as personally enraging: > "I had to remove a lot of swearing from this slide because in my actual transcripts, there is going to be a lot of a lot of rage." > "those are elementary mistakes... and that just outrages me." - **Self-deprecating asides** keep the provocations friendly: > "It's not a trick question and I'm not trying to shame you." - **Casual register** throughout: "that I'm going to yap about a little bit more," "warm fuzzy feelings about that," "we played this little whack-a-mole." ## Vocabulary & Recurring Phrases - **"Codify / codifying"** is the conceptual anchor, used at the thesis and in the call to action: > "you should stop prompting. You should start codifying the actual rules that the agent should be following." > "try to codify which ones of those are actual high quality lint rules." - **"Linters" / "lint rules"** dominate the middle section — including the idiosyncratic pronunciation/spelling variant "**llinters**" (appears at 239, 249, 260, 632, 676). - **"Reprompt"** appears multiple times as the villain: "reprompt your fixes," "issues that we otherwise reprompt all the time." - **"My claim / my thesis"** framing recurs: "my entire thesis for this talk, my entire claim here is..." / "my claim for this is simple." - Filler-heavy, spoken style: frequent "**um**," repeated words ("the the," "a a"), self-corrections mid-sentence. ## Teaching Habits - **Rhetorical-question-then-answer pattern** to bring the audience along: > "what are you thinking of?... Evals. Yes. Hooks. Yes." > "who still reads code that AI generates? Anyone?" - **Thesis stated early and restated at the end** ("That's my call. My call to action for you is...") — a clear setup/payoff structure. - **Escalating example ladder**: simple lint rules → Sentry's OpenAPI drift rules → skill hallucination rules → qualitative policies. Each step is introduced as "even more insane examples." - **Concrete failure narratives** rather than abstractions — the whack-a-mole sequence (agent removes code blocks, then backticks, then describes APIs in prose) is told as a step-by-step story. - **Explicit "how" slide** near the end (ESLint, flake8, clippy, golangci-lint, ast-grep, new tools) — he alternates concept → practice. ## Communication Patterns - **Direct reader address / imperatives** in the actionable moments: > "take your laptop go to your agent and ask hey look at all my past transcripts..." > "be aware of that. beware of um the complexities..." - **Hedged honesty about limits**: "I don't know how to lint against that. I don't think I reliably can lint against that," and "it's all made up." He marks uncertainty about his own prescriptions ("this things will change as models get better"). - **Parenthetical context-giving** about Sentry ("Sentry is a pretty old codebase. We are on Django, we are on DRF") — he contextualizes claims for non-Sentry audiences ("This is a very common case for any mature codebase"). ## Stable vs. Isolated - **Stable-looking**: the rage/anecdote framing, "codify" vocabulary, linter-centric argument, question-to-audience technique, and explicit call to action. - **Possibly isolated**: the "grumpy engineer / Garfield" skill naming and the "whack-a-mole" story appear once each; whether such storytelling recurs across his talks cannot be determined from a single transcript. ## Uncertainty Notes - Transcript timestamps only; slide content is inferred only where he verbalizes it (e.g., "I had to remove a lot of swearing from this slide"). - The "llinters" spelling may be a transcript artifact rather than his own usage; the delivery evidence is the consistent pronunciation/term repetition, not the spelling itself. # Observation: "Stop Prompting" (Greg Pstrucha, Sentry) ## Source and Speaker - **Video:** "Stop Prompting — Greg Pstrucha, Sentry" - **Speaker:** Greg Pstrucha, who introduces himself in the transcript as "an AI engineer at Sentry" working on infrastructure for Sentry's debugging agent (Seer). All positions below are attributed to him unless marked otherwise. - **Other voices:** Brief audience responses appear during the talk (see the final section). They are not attributed to named individuals. ## Core Thesis > "My entire thesis for this talk, my entire claim here is you should stop prompting. You should start codifying the actual rules that the agent should be following." - **Stated scope:** He clarifies that the title does not mean "loop engineering," though he later describes one loop-based process. - **Problem:** Corrections given through prompting decay. After context compaction, the agent "[is] back to square one" and the user must reprompt. - **Proposed remedy:** Encode conventions and quality standards as **deterministic checks** (tests, strict typing, linters) and **qualitative policies** that are enforced through agent review. ## Claims by Theme ### 1. Reading AI-Generated Code - He reads AI-generated code **contextually**: - For **Sentry production code**, he reads it to avoid breaking production. - For **side projects**, he says reading every line "doesn't really matter" and he can be "more lenient." - When he returns to such code later, he reports it is often "overly defensive," has tests that are "crap," and is code he "would not want to maintain." - Typical failures he describes as "very elementary mistakes" when no guardrails exist. ### 2. Why Prompting Alone Falls Short - Corrections given to the agent ("overly complicated," "too defensive") are followed by the agent saying "You're absolutely right," then re-running fixes. - Context compaction causes regression, forcing repeated corrections. - Humans **retain memory** of feedback; agents "do not have any memory" and "make the same repeated mistakes over and over again." ### 3. Deterministic Checks He names **three deterministic priorities**: - **Tests** - **Strict typing** - **Linters** He adds that **hooks** determine when checks run and **evals** are relevant. He says tests are "absolutely the class of that," and hooks are "how you run those things at the right time." He qualifies the list explicitly: > "Those are not total statements and they are not enough to actually get your agent to write good code." On typing specifically, he argues that strict typing alone is insufficient. If the type system still permits undesired states, the agent will introduce them. Banning types that admit unwanted edge cases is part of the point. ### 4. Why Linters Are Now Economically Different - **Before AI:** Writing a custom linter had a real **maintenance cost**, so teams weighed how often an issue recurred against whether a linter was worth it. - **With agents:** Writing a new lint rule is "very cheap and very easy," and the agent can write it. Agents also do not tire of repeated lint failures in the way humans do. - He frames the shift as: the authors of code now lack memory, so codified rules are more valuable. ### 5. Practical Recommendation: Mine Past Feedback - Ask an agent to review: - Past transcripts - GitHub review comments - Bot feedback - Peer feedback - Have it identify which items are **high-quality, deterministic lint candidates**. - He says he does this regularly, on a schedule, to find newly recurring issues. ### 6. Categories of Custom Lint Rules **Simple, conventional rules** (long-established practice): - Disallowing `console.log` in a given project - Enforcing or discouraging design-system or React practices **More complex, agent-era rules** (Sentry examples): - **OpenAPI drift.** The OpenAPI schema had drifted from actual endpoint behavior. Three lint rules were introduced: 1. Endpoint functions must return a response type, with no untyped (`Any`-like) components. 2. Endpoints must carry the schema decorator to ensure documentation. 3. The produced type must agree with the OpenAPI declaration. He reports that this keeps APIs "a little bit more in sync." - **Skill documentation accuracy.** Sentry's agent relies on "skills" (guidance documents) to call Sentry's API. Many skills contained code examples, and many of those examples were **hallucinated**, which significantly reduced agent accuracy. - First rule: extract code blocks, type-check them, run them in the sandbox, and verify they agree with the OpenAPI schema. This helped. - **Failure mode 1:** When the agent regenerated skills under this rule, it **removed code blocks** to avoid lint errors. A rule then required examples to stay in code blocks. - **Failure mode 2:** The agent moved code-like content out of code blocks and into prose, so the linter treated it as text. He responded with a rule that code-looking content (underscores, parentheses) must be placed in code blocks, and the agent began describing APIs in prose. - **Failure mode 3:** The agent then referenced URLs, which he linted away. He describes this as a "whack-a-mole" process that ended with skills synced to the OpenAPI schema. - He says the result was "not... unbreakable," but that skills no longer need constant manual follow-up when the API changes. He describes the cost as "not that expensive." ### 7. Tooling Mentioned - **Existing linters:** oxlint, flake8, clippy, and Grep ("language agnostic," so it offers "unified tooling"). He says he has "warm fuzzy feelings" about Grep for that reason. - **Emerging tools:** Taskless, where the user writes intent in English and a linter is derived from it. - **Advice:** Ask the agent to write the linters. He says the user "will be surprised" by what it can produce. ### 8. Qualitative Rules That Linters Cannot Express - Example: a state machine with three states grows to ten after a small change. He says he does not know how to lint against that reliably. He can add tactical rules (e.g., "no more than four states"), but he describes such limits as "all made up." - He treats these as qualitative judgments that call on engineering experience, which he jokingly equates with "taste," or which an LLM reviewer can partially assess. ### 9. Policies and the "Grumpy Engineer" Skill - He has written **repository policies**, including: - Not too many tests - Stronger tests - Code that is not overly defensive - Preference for types that make undesired states unrepresentable - The **"grumpy engineer" skill** (called "Garfield" internally): - Runs a sub-agent per policy. - Each sub-agent either gives feedback or "stands down." - The loop repeats until all sub-agents stand down. - Only then does he review the code himself. - He says the success metric is how much he is "catching up" the agent: fewer elementary corrections during review. - He states the aim is "to raise the floor," **not** to remove code review "quite yet." He expects this to change as models improve. - He notes that this is "the closest you will hear me doing loops." ### 10. Caution: Quantitative Proxies for Qualitative Goals He reports experiments trying to capture qualitative properties with metrics: - **Cyclomatic complexity:** A proxy for maintainability and testability based on branching and call depth. - **Test coverage:** A proxy for how much code is tested. His finding: when a number is given to the agent, it optimizes that number. He reports reaching near-100% coverage, but the tests asserted specific strings in output and "[weren't] measuring anything of value," creating "a false sense of security." > "Beware of the complexities of actual qualitative metrics." ### 11. Call to Action - After the talk, ask your agent for linters and policies, and see whether they help. - "Try things and make the repo remember." ## Qualifications and Tensions | Topic | Stated position | Qualification or tension | |---|---|---| | Title ("Stop Prompting") | Stop prompting; codify rules instead | Explicitly says it is not "loop engineering," yet later describes a loop-based grumpy-engineer process | | Tests, typing, linters | Three priorities for deterministic enforcement | Each is "not enough" alone; typing must also ban undesired states | | Linters for agents | Writing them is now cheap | Maintenance cost still exists; the Sentry skill example required several rounds of "whack-a-mole" | | Agent-generated linters | Agent can write them, and "you will be surprised" | Not presented as always correct | | Grumpy-engineer review | Raises the floor | Does not replace human code review "quite yet" | | Qualitative policies | Helpful via sub-agents | Qualitative rules are described as dependent on engineering "taste" | | Skills sync | Keeps skills in sync with API | "Not... unbreakable" | | Code reading | Depends on context | Reads production code carefully; is more lenient on side projects | ## Unattributed Audience Responses During the question about deterministic checks, unidentified audience members called out: - "Evals" (answered with "Yes") - "Hooks" (answered with "Yes") These are not attributed to any named individual. ## Transcript Notes - The transcript is automatically generated and contains errors. For example, "seir centuries" appears to refer to Sentry's Seer agent, and "llinter," "lintters," and "lter" appear to be "linter." "Grabb" appears to be "Grep," and "Garfield" is given as the internal nickname for the grumpy-engineer skill. - Speaker remarks about rage and swearing refer to a removed slide and are not part of the spoken argument. # Speaker Attribution and Provenance Record **Video:** *Stop Prompting — Greg Pstrucha, Sentry* **Record scope:** Speaker roles, referenced sources, third-party references, promotional content, and evidence availability. Content quality is not assessed. --- ## 1. Speakers and Roles ### Primary Speaker - **Greg Pstrucha** (self-identified as "Greg") - **Role:** Presenter and talk author - **Stated affiliation:** "AI engineer at Sentry" - **Self-described work:** Infrastructure for Sentry's debugging agent (transcribed as "seir centuries uh debugging agent," likely *Seer*) - **Identity confirmation:** Name appears in the video title and the spoken introduction. No curated metadata independently confirms the role. ### Audience Interjections (Unidentified) Short responses are captured in the transcript, but the speakers are not identified: - "Yes" / "Hooks" / "Evals" (responses to the presenter's question about deterministic checks) - "What's that?" (a clarifying question) - "Anyone?" (the presenter's rhetorical question, which drew no identified answer in the transcript) > **Attribution note:** Audience contributions cannot be linked to individuals. Attribute them only as "unidentified audience member(s)." --- ## 2. Referenced Sources and Tools The presenter names the following. These are references, not verified claims. | Category | Reference (as transcribed) | Context in Transcript | |---|---|---| | Internal system | Sentry debugging agent ("Seer") | Speaker's employer product he works on | | Internal artifact | "Garfield" / "grumpy engineer" skill | Speaker's internally built skill; nickname stated | | Specification | OpenAPI schema | Described as Sentry's API declaration | | Frameworks | Django, DRF | Stack described as "old codebase" | | Linters/tools | oxlint, flake8, clippy, "grabb" (transcription uncertain) | Named as existing tools | | Third-party tool | Taskless | Described as writing lint intent in English | | Metrics | Cyclomatic complexity, test coverage | Described as examined and rejected as sole measures | | Public commentary | "Twitter" (via "as tweet as Twitter would say") | Attributes the word "taste" to Twitter usage | > **Transcription caveat:** "grabb" and "seir centuries" are likely transcription errors. Do not treat them as verified names. --- ## 3. Third-Party Claims and Attributions The presenter attributes or reports the following. None is independently verified in the transcript. - **Experience-based claims (first-person):** - Agents make "elementary mistakes" without guardrails. - Compaction of context causes the agent to return to "square one." - Agent-generated test coverage can reach near 100% while tests are weak. - Skill examples lost accuracy because they were hallucinated. - **Reported internal outcome:** - Sentry's API schema drift reduced after lint rules were introduced. - Skills were kept "in sync" with the OpenAPI schema via lint rules. - **Unverified generalization:** - "New tools … popping up" (no specific sources cited beyond Taskless) > **Provenance status:** All outcome claims rest on the speaker's personal account. No data, dashboards, or citations are presented in the transcript. --- ## 4. Promotional Content - **Employer reference:** The presenter refers to his employer (Sentry) and its debugging agent several times. These mentions are contextual rather than sales claims, but they are **employer-affiliated content** and should be flagged for downstream readers. - **Named third-party tool:** Taskless is recommended as a way to write lint rules. This is a **product recommendation** by the speaker. No sponsorship or affiliation is stated. - **No explicit sponsorship, discount, or paid-placement language** appears in the transcript. --- ## 5. Evidence Availability - **Curated metadata supplied:** Title and transcript only. No description, chapter list, speaker bio, or citation list was provided. - **Comment availability:** Not supplied. Comment access cannot be determined from the record. - **Transcript limitations:** - Machine-style artifacts are present (`>>`, `[music]`, escaped characters). - Several words are mistranscribed or ambiguous. - Some slides and code examples referenced verbally are not present in the transcript text. - **Omitted material:** The speaker states he removed profanity from a slide and that his "actual transcripts" contain more emotional language. Those transcripts are not available in this record. --- ## 6. Summary of Attribution - **Single identified speaker:** Greg Pstrucha, presenter, Sentry AI engineer. - **Unidentified speakers:** Audience members providing brief responses. - **Sources referenced:** Internal Sentry systems and standards, several linting tools, one third-party tool, and two named metrics. - **Promotional flags:** Employer-affiliated content and a tool recommendation (Taskless). - **Evidence limits:** No external verification, no citations for outcome claims, and no comment access information. All \>\> \[music\] \>\> right. Hello everyone. I'm going to try to speak as loud as I can so it's reasonably good with all the all the commotion. Uh my name is Greg. I'm an AI engineer at Sentry. I work on the infrastructure that powers seir centuries uh debugging agent and I have a little rage baity title of the presentation that calls to stop prompting but I do not mean loop engineering. I'm going to get uh a little bit deeper here. Uh who still reads code that AI generates? Anyone? It's not a trick question and I'm not trying to shame you. I I think it's contextual. I read code when it's sentry code and I need to make sure that we are not breaking the actual production for sentry but at the same time if I am working on a side project it doesn't really matter uh to read every line of code and I can I can be a little bit more lenient with it uh but then every now and then I go back to the actual code I want to make improvements and um it's bad it's it's overly defensive code it's tests are crap they are not testing anything. Um, I would not want to maintain or work in that codebase anymore. So then what you do is you you talk to the agent. You say, "Hey, you've done a mistake. This is overly complicated. The code's too defensive. Um, you don't have to create endpoints for everything you're doing, otherwise I have to maintain it forever." And I had to remove a lot of swearing from this slide because in my actual transcripts, there is going to be a lot of a lot of rage. Usually those are very elementary mistakes that the agents are making if you don't put any guard rails in place and um and that just outrages me. So then you do that, the agent says, "You're absolutely right. Let me fix this." And it's going to run and try to fix the the mistakes. And five minutes pass and compaction of context happens. And we're back to back to square one. And you would have to now repeat yourself, reprompt your fixes, reprompt the agent on the right track. And my entire thesis for this talk, my entire claim here is you should stop prompting. You should start codifying the actual rules that the agent should be following. um rules and policies that you want to establish inside your codebase that are going to capture both convention and the quality of code that you are expecting and would make you proud. And so when I say you would want to add deterministic checks or rules, what are you thinking of? \>\> Yes. \>\> What's that? \>\> Evals. Yes. \>\> Hooks. \>\> Hooks. Yes. Um the three that I'm thinking of before I even get to hooks and evils, hooks and evils are important. Tests are absolutely the class of that hooks are how you run those things at the right time. Um to me the three things that I want to optimize on the deterministic level are making sure that I have tests, making sure that there is strict typing, making sure that there are there are splinters. Those are not total statements and they are not enough to actually get your uh agent to write good code. as an example, if you just take from this that you have to put strict typing in place, you will still be in a bad place. Um, if you don't discourage or ban um a way of typing that removes edge cases you don't really want to see, then the behaviors that or the states that are representable in the app are going to lead the agent to introduce them. But the thing that I want to predominantly focus on in this talk is llinters. Lintters are our way to codify um the rules in a deterministic manner that we want to um remove from the from the codebase. Historically like before AI when you think about llinters um the way that this worked is the trade-off wasn't clear. Writing an actual llinter took a lot of work. um it was a maintenance cost that you had to sort of sign yourself up for and you had to consider how often the issue that you try to lint away comes up is it even worth it and if it's if it's pre pre- aents and those are humans that are working in my codebase and the only thing I'm doing is once every free week telling people to use the right component from the design system then I don't necessarily need a llinter for that right and people have memory so if I tell them once or twice they will usually remember that. But then this shifts entirely into this new world where the things that are writing code do not have any memory and are going to make the same repeated mistakes over and over again. And on the on the at the same time to write a new lter is very cheap and very easy. You can just have the agent do that and it's going to be much less frustrated with the actual lint rules that you're trying to to write. So my claim for this is simple and if you take one thing from this entire talk that I'm going to yap about a little bit more is you can after this talk take your laptop go to your agent and ask hey look at all my past transcripts look at my GitHub reviews the feedback from the bots the feedback from my peers and try to codify which ones of those are actual high quality lint rules that are deterministic and will be able to capture issues that we otherwise reprompt all the time. Um, that's something that I do now and I happen to do that even more often uh sort of on the schedule to try to see whether there are new things that I haven't captured. So, here are some examples of what I mean when I say custom lint rules. Those are uh lint rules that try to capture conventions that you want in your codebase. There is like a whole simple class of them. Those are the ones that you know and have written for a long time. And those are things like I want to lint out console log statements from a particular project or I want to make sure that if I have a design system or react practices that I want to follow or discourage those are codified and are limited away. Those are the simple things but I think with agents we are getting into more interesting complexity. We can push this a little bit further. So one example here, this is a little bit of a tricky example, but um I'll go through it. Sentry is a pretty old codebase. We are on Django, we are on DRF. We don't have strict typing everywhere. This is a very common case for any mature codebase where you're not going to be at the top leading edge of the technology. Um but we still want to improve the state of the world for for the agents. So concretely here um there is there was a lot of drift of what the open API schema for sentry said it's going to do and what it did and so we started introducing lint rules for that. First lint rule is if something is a end point endpoint a function for an endpoint it needs to return a response type and within that response type there cannot be any types so that we can infer the actual information of the type. Second, if it's a endpoint, you get that little extent schema decorator so we can ensure that it's documented. And then third, enforce that the actual type being produced here agrees with what the open API schema at the end of the day um declares. And that helps us because now APIs are a little bit more in sync. And that gets us to even more insane examples of um of llinters. So sir agent is a debugging agent and it has to talk to Sentry a lot. It uses Sentry telemetry to debug and RCA issues and provide root cause analysis solutions all of that because it has to talk to open to um Sentry API we oftent times have to guide it and the way we do it is through skills. oftentimes those are old endpoints that we are not able to um easily fix and they have some complexities that are hard to describe on the level of schema itself. So instead we guide that through skills which is a pretty common pattern in the agentic world. But here is an example of a skill that tells how a metric should be created at sentry. We generated a lot of those skills and examples with those code blocks as the as the examples of code that can then be run in the sandbox by agent. And in the original attempt to do that, we lost the accuracy of the agent significantly because lots of those examples were hallucinated. So the first lintter we created was a llinter that says if there is a code block inside the skill, you got to extract those code blocks, you have to make sure the type check, they can run in our sandbox and whatever they declare agrees with open API schema from Sentry and that helped. But what what the f the first thing that the agent did when we tried to rerun the skill generation with that rule is oh there are lint rule errors let me remove the code blocks and that wasn't very desired by us. So we said no no no examples have to be put in code blocks. So it put the examples back but it removed the bactics because only the bactics are linted. So let's let's fix the problem right. So we're like, "No, no, no. If it looks like a code, smells like a code, if it has underscores and parenthesis, push those into codelocks." So I was like, "Okay, so you want me to put code into codelocks. I'm going to describe the APIs." So now in pros, it started describing what the APIs should do, but it did one mistake, which is it referred to the URLs uh of the APIs. So then we linted away the URLs and we played this little whack-a-ole for a little while, but at the end of the day, we ended up with skills that are synced to our open API schema and that wasn't that expensive to do. And we have I'm not going to say unbreakable model, but we have a model where when API changes, we actually don't have to follow up on the agent skills all the time. it it's it's kept in sync through the through the llinters. And when it comes to the actual movements or the actual how do we do this you can use any sort of tool that you're using already es oxlin flake 8 clippy as grabb as grabb is great uh because it's language agnostic so it kind of you know unified tooling warm fuzzy feelings about that um there is a bunch of new tools that are popping up that are trying to approach it from a different uh perspective one example is taskless where is instead of writing the actual lter you write what you want the llinter to do. You write your intent in English language and then the actual llinter is um derived from it. Um and then the last thing that I uh that I will repeat is try asking the agent for the actual lers. You will be surprised uh what it can what it can generate. But that's only half the picture. What if there are rules that are deterministic cannot express um cannot be expressed through llinters? There is plenty of those, right? Those are all types of qualitative measures like you have a state machine with just three states and I created a small PR to add small functionality and ballooned up the number of states into 10. I don't know how to lint against that. I don't think I reliably can link against that. I can like create small tactical things like no more than four states, but it's all It's all made up. This is a this is the situation where we're talking about qualitative measures and um this is where you are engaging your engineering experience or if tweet as as Twitter would say your taste um and you just have to decline it straight up or you can have an LLM agent that is going to do a pass and for simpler thing help you with that concretely here I've uh I've started to create policies for the repository and those policies are such as I don't want too many tests, I want stronger tests or I do not want a code that's too defensive. I'm sure you've seen those where every single possible case is try code by the agent and it's it's just sloppy long long functions. I would rather have code that in its type system does not allow to express states that are undesired. Um so I created those policies and then I created a skill that I called grumpy engineer. We call it Garfield internally and the way it works is it takes all the policies and run sub agents for each of the policies and the sub agent can either give feedback or stand down and we loop that and this is the closest you will hear me doing loops by the way. uh we will loop that until the agent stands stands down and says I I'm okay with that I validate that this is fine and only then I read the code and sort of my metric of success here is how much I'm casting up the agent how much when I'm reviewing the code I'm like oh yeah this is not ideal but I can I can steer it here and I don't have to do very elementary feedback all the time it is still qualitative but the goal here isn't to remove the code review quite yet you So this things will change as models get better but more so to raise the floor and make it easier to um to work with the agents uh as you as you're going forward. There is also one thing that I want to mention that is um beware of a snake oil. We are talking about the tricky part being qualitative measures and I've done experiments where I tried to capture a qualitative measure in a quantitative metric. So two concrete examples one of them is um cyclomatic complexity and the other one is test coverage. Cyclomatic complexity uh is a measure that says how maintainable and testable the code is and it tries to base that on how many branches happen in the code and how deep the call stack goes and um and test coverage you know test coverage tries to tell you how much of the code is covered by test. Um, and the problem with that is when you give a number to the agent, the agent will optimize the out of that number. And it did. And I I got code bases, I got projects where I got to like a 100% or close to 100% of test coverage. But the actual tests are bad. They are like asserting particular string in particular output of the agent. They are not measuring anything of value. Um, and more so they are giving you a false sense of security. So be aware of that. beware of um the complexities uh of of actual qualitative metrics. Um and that's it. That's that's my call. My call to action for you is after this try to ask your agent for llinters and policies and see whether that helps you. Um try things and make the repo remember. And that's that's my time. Thank you.