Gautam Korlam (Sonar, previously Uber's SubmitQueue and Gitar) examines how coding agents shift engineering bottlenecks into CI and the merge path. A merge queue keeps main healthy by testing combinations of changes before they land, but a process built for human arrival rates becomes strained when agents generate many pull requests at once, moving the constraint to failed builds, repeated repairs, and review/approvals. The talk covers incremental builds, caching, sharding and parallel work to reduce feedback cost (illustrated with a Rust example); durable agent context from logs and failures for repair loops; and risk-based approval policies that separate authorship from approval, tie decisions to the exact tested commit, and require recorded reasons for overrides. Reported volume and latency figures are examples from the speaker's systems, not general productivity guarantees. # Channel Presentation Observations ## Self-Description Evidence (explicit) - **Speaker identity and affiliation stated directly:** > "My name is Goautam. I am a co-founder and CTO at QAR and recently joined Sonar as a principal engineer. After the acquisition… I was at Uber for almost a decade working on build and merge infrastructure for mono repos." - **Positioning as practitioner-expert:** "a lot of experience from build and merge infrastructure that I have been working on for a decade plus at this point." - **Product-aligned mission claim:** "now at QAR as part of Solar I'm trying to make sure that code that's been checked in is verified and all the code written by agents is of high quality before it goes in." - **Direct product promotion at close:** "would love for you folks to also try out guitar and see how you can improve your software factories." ## Intended Audience - **Engineering leaders and platform/CI teams at AI-adopting organizations:** > "walk you through all the new bottlenecks we're seeing in software factories and also specifically for highly regulated industries and banks." - **Teams adopting agent-driven development:** "a lot of you probably already use like AI to review your code," "as they are going through their AI adoption journeys." - **Compliance/regulated-industry concerns explicitly addressed:** "How do we keep things more compliant and auditable while being able to move fast?"; references to HIPAA ("HIPPA"), SOC 2 ("Tora"), and Uber's London license story. ## Claimed Scope - **Topic scope (title + intro):** rebuilding CI/CD systems for AI-scale commit volume — merge queues, caching, review automation, approval policies, and auditability. - **Evidence scope includes first-party data claims:** - "at least 13% of PRs had a recorded CI failure" (on their platform) - "growing 25% month over month" in PRs reviewed - CI time reduced "from around 10 minutes" to "less than a minute for incremental steps" - **Historical credibility claims:** built SubmitQueue at Uber ("we have a paper on… this is still running today at Uber"). ## Demonstrated Presentation Patterns (observable) - **Conference-talk format:** formal greeting ("Hello everyone… welcome to my talk"), structured walkthrough of problem → techniques → case study → closing. - **Technique-oriented, product-illustrated:** generic advice (caching, sharding, "move checks left") interleaved with Qar/Guitar examples and features ("guitar supports these things out of the box," "custom rules which are plain English"). - **Anecdote-driven persuasion:** Uber London incident, shared-database flaky-test story, personal team metrics. ## Notes on Evidence Boundaries - Claims of platform growth, customer adoption ("100 plus,000 plus developers"), and internal metrics are **creator assertions**, not independently verified by the material. - No curated metadata beyond the title was supplied; the title itself ("Sonar") corroborates the affiliation claim in the transcript. # Delivery Observation — Gautam Korlam, "Rebuilding CI for AI-Scale Commit Volume" ## Tone - **Conversational and practitioner-first.** The speaker opens with an apology-free, direct framing: *"Hello everyone. Uh welcome to my talk. Uh today we're going to be talking about how we are rebuilding CI for AI scale commit volume."* - **Workmanlike, informal register.** Filler sounds are frequent (*"uh"* throughout), and there are audible **snorts** at casual asides (e.g., before *"So the problem with this approach…"* at [173] and *"it was saved with the change"* at [972]). These suggest an unpolished, extemporaneous delivery style rather than a rehearsed script. - **Measured when citing scale.** Numbers are delivered plainly rather than dramatically: *"our own uh rate of commits has almost like 10xed since we started guitar uh about 3 years ago"* [230–244]. ## Vocabulary and Recurring Phrases - **"Software factories"** — a signature framing term, introduced early ([21]) and reused across the talk ([207], [335], [1121], [1132]). - **"Bottleneck(s)"** — the talk's central metaphor, appearing in the opening (*"walk you through all the new bottlenecks"*, [10]) and repeatedly thereafter ([173], [219], [568], [1087]). - **"Move them to the left" / "shift left"** — repeated as a mantra: *"if you can move them to the left, it's way cheaper to catch them"* [556], *"making sure you can like move some checks left"* [1100]. - **"Green" as shorthand for passing CI** — *"everything was green"* [88], *"red main all the time"* [100], *"just because CI is green"* [904]. - **"Auditability and compliance"** — a persistent concern track: *"for auditability and compliance checks to happen"* [425], *"record that it was enforced for auditability"* [637]. - Frequent hedging/discourse markers: *"like"*, *"kind of"*, *"you know"*-style softeners appear on nearly every explanation (e.g., *"we could like reduce the wait time"*, [132]). ## Teaching Habits - **Problem-before-solution structure.** Each section states the pain first, then the remedy: *"So what does this mean uh for teams today? Right?"* [230] → evidence → *"how do you solve for this"* [301]. - **Rhetorical questions as transitions.** *"So how do you like solve for this like how do you keep things green"* [301]; *"So how do we make this better like what what do we do to make this problem more manageable"* [491]. - **Narrative credibility via war stories.** Uber anecdotes anchor claims: the SubmitQueue system (*"this is still running today at Uber"*, [163]) and the London license story: *"we lost like the license to operate in London because there was a glitch in the software"* [580]. - **Dogfooding evidence.** Uses his own company's metrics and incidents as proof, including a self-referential example where Qodo *"caught it on our PR before we merged it"* [787]. - **Concrete numbers as anchors.** *"20 days in this case"* [88], *"13% of PRs had a recorded CI failure"* [435], *"around like 10 minutes"* down to *"less than a minute"* [649–684], *"25% month over month"* [266]. ## Communication Patterns - **"So" as the dominant connective.** Nearly every paragraph begins with it (observable at [153], [230], [301], [335], [425], [491], [637], [904], and many more). - **Enumeration at the end of sections** — lists of record-keeping items delivered in a rolling sequence: *"which coding agent made uh the PR? Where did the PR originate from? What was the session that originated it?"* [1020–1033]. - **Direct address to the audience.** *"A lot of you probably already use like AI to review your code. Hopefully uh you're not doing every single review with with humans."* [371] - **Product mention woven into teaching.** Qodo/Guitar examples appear naturally inside lessons rather than as a separate pitch, e.g., *"guitar actually is a code reviewer. It caught it on our PR before we merged it"* [787], before a closing invitation: *"would love for you folks to also try out guitar"* [1144]. ## Stable Patterns vs. Isolated Moments - **Stable (highly repeated):** "so"-led transitions, "bottleneck"/"software factory" framing, green/red CI metaphors, problem→solution sequencing, and Uber/Qodo anecdotes as evidence. - **Less certain (single or few occurrences):** the rhetorical-question teaching device appears several times but is not uniformly distributed; the "zero trust" concept appears only once by name (*"that would not be uh zero trust"*, [882]), so its status as a signature term is uncertain. The snorts and heavy filler suggest an unrehearsed style, but this is a single-sample observation of one talk. ## Uncertainty Notes - The transcript contains transcription artifacts (e.g., "QAR"/"Guitar"/"Goautam"/"Back at Uber" [580]), so exact brand and proper-noun spellings cannot be verified from this source alone. - Audience reactions (laughter, etc.) are not visible in the transcript, so responsiveness to the room cannot be assessed. # Observed Positions — *Rebuilding CI for AI-Scale Commit Volume — Gautam Korlam, Sonar* ## Speaker Identification - **Gautam Korlam** — self-described co-founder and CTO at Qodo (Guitar/"QAR" per transcript phonetics), recently principal engineer at Sonar following an acquisition; previously ~a decade at Uber working on build and merge infrastructure for monorepos. - All positions below are attributed to Korlam as expressed in the talk; this document records stated claims, not verified facts or private beliefs. --- ## Central Thesis > "All these cues were built for human speed and once you build it from human speed what happens is they are not able to keep up once you go to agents." Korlam argues that CI and review queues are now the **centralized bottleneck in the SDLC**, and that AI-scale commit volume requires faster CI, automated review, risk-based auto-approval, and durable audit records. --- ## Historical Context: Uber and Submit Queue - Uber had "agent-scale" commit volume before agents: one giant monorepo with **thousands of changes per day** and long-running builds. - Purely serialized merging would take **up to 20 days** to keep main green; skipping the queue meant **"red main all the time"** (e.g., iOS monitors constantly red) because individually-green PRs could break when combined. - Solution: a system he calls **"summit cube"** (with a published paper) that tested likely combinations of incoming commits in parallel, predicted breakage by testing combinations, ejected PRs that might break, and only let commits into the trunk if they passed with all combinations. - Claim: the system is **still running at Uber today**, predating GitHub merge queues and similar code-host features. ## Problem Framing ### Queues as bottlenecks - Humans generate changes "a few times an hour," enabling serial testing and review; agents produce **several PRs per developer per day** and "don't always wait for the queue." - Software factories and background agents triggering on workflows add further pressure, making everything "slow down to a crawl." ### Growth claims (stated for his context) - His own team's commit rate has **~10x'd in ~3 years** with the same small team. - On his platform, PR volume grows **~25% month over month**, which he characterizes as an exponential trajectory. - He cites industry players like GitHub as reporting "staggering growth in PRs," and infers larger teams and banks face this "at a much higher degree." ### Repair overhead - Claimed platform data: **at least 13% of PRs had a recorded CI failure** (one or more), requiring a repair round trip (developer or agent reads logs, root-causes, fixes, re-pushes, iterates). - Human reviewers typically wait for green CI ("first line of defense"), so review **cannot proceed in parallel** with failed-CI repair. ### Auditability gaps - Logs expire; a bare approval stamp gives **no record of what was actually checked**. - Green CI does not imply compliance: flaky-test reruns plus force-pushes can leave the branch "pointing at another piece of code," so the tested artifact is no longer the merged artifact. --- ## Prescribed Solutions ### 1. Make CI fast - Remove expensive checks or offload them post-merge; parallelize, shard, cache. - Case study: baseline CI of **~10 minutes**, Rust build system; caching outputs, reducing compiler churn (fixes improving cachability), and sharding brought **incremental CI to under a minute** once warmed. ### 2. Make review fast and multi-pass - Claim: at Uber, **the vast majority of P90 PR-to-production time was spent in review** — a pattern he says holds for most teams "even today." - Use AI as a **first-pass reviewer**, with humans reviewing what remains; escalate deeper review only for higher-risk changes. - Example: a refactor using NextTest(?) to parallelize tests with a shared database lacked guards, risking **cross-contamination and flaky tests under busy CI**; his company's AI reviewer caught it pre-merge, suggested a fix, committed it, and added the engineer as co-author. ### 3. Shift checks left - "If you can move them to the left, it's way cheaper to catch them" — including expensive regression tests and production-failure risks; agents also have more local context to fix issues. ### 4. Risk-based approval and auto-merge policies - Auto-approvable categories (as stated): **routine/lower-risk documentation, dependency bumps (non-API-breaking), security patches, lower-risk internal tooling**. - Custom rules in **plain English** supported "out of the box" by his product; he reports rapid adoption of auto-approval/auto-merge across teams "from large teams to like 100 plus,000 plus developers." - Escalation rules: sensitive areas (e.g., **payments**) may require a human approver; the **agent that authored a change must not approve it** ("not... zero trust"), and the engineer who prompted the agent should not be the sole approver. ### 5. Compliance policies beyond CI - Uber anecdote: Uber **"lost the license to operate in London"** due to a feature-flag glitch letting drivers onboard without proper checks; the remediation was a lineage process for any change affecting London ("does this change affect London? If so, what are the controls in place"), after which they "won the license back." - Compilation passing does not mean compliance with policies like **HIPAA** ("HIPPA" per transcript) — agents won't respect policies unless they are enforced and that enforcement **recorded**. ### 6. Durable audit records - CI data expires; **PR records, agent trajectories, and per-push outcomes** should be the source of truth as artifacts expire. - Control principle stated: > "The control here is making sure that the change was approved and tested by someone who did not write it." - Example contrast: a pipeline that failed but has a **recorded reason** (known failure mode) plus an independent approver's approval passes checks; an unrecorded pass "blindly trust[s] the agent pushing code" and may hide a low-quality test. ### Records he recommends keeping - **Which coding agent** made the PR, its origin, originating session, and initiator (automated workflow trigger vs. engineer prompting). - For each CI run: the commit where CI was trusted and a **saved test report**, "not just a status." - Each finding's **resolution record**: dismissed, followed up, or fixed on the PR. - Approver identity: someone other than the bot and not prompted by the same person. - **Explicitly named overrides**: reason recorded when a build is red or coverage is skipped. - Confirmation that the merged code matches the checked code ("not like a small change on top"). --- ## Closing Argument Removing bottlenecks in sequence — faster CI, checks moved left, automated review with human escalation, auto-approval of low-risk changes, and recording every approval/merge — is what allows "a software factory to perform to its potential," because historical context is needed to iterate as factories scale. --- ## Qualifications and Tensions Preserved - **Tentative vs. firm**: platform figures (13% CI failure rate, 25% MoM growth, ~10x commits) are presented as his own platform's data; GitHub growth is cited secondhand and the inference to banks is framed as his supposition ("I'm sure... they are seeing it at a much higher degree"). - **Self-referential evidence**: several examples (Guitar catching the flaky-test PR, customer auto-merge adoption) come from his own product, and the talk ends with an explicit product pitch — recorded here as stated, not as independent validation. - **Acknowledged limits**: "not every time it's possible to like move things to the left"; record-keeping specifics "could be different for folks." - **Named transcript phonetics**: the company is rendered variously as "QAR," "Guitar," and "Gar" in the transcript; per the title and standard naming this appears to be **Qodo**, but the transcript's own variants are preserved above where quoted. # Speaker & Provenance Record — "Rebuilding CI for AI-Scale Commit Volume — Gautam Korlam, Sonar" ## Speakers & Roles | Speaker | Role | Basis | |---|---|---| | **Gautam Korlam** | Sole speaker; conference presenter. Self-identified as **co-founder and CTO at QAR (Guitar)** and **principal engineer at Sonar** following the acquisition; previously ~a decade at **Uber** working on build and merge infrastructure for monorepos | First-person introduction in transcript | No guests, interviewers, narrators, or other speakers appear; the talk is a single-presenter conference session. > "My name is Goautam. I am a co-founder and CTO at QAR and recently joined Sonar as a principal engineer. After the acquisition." --- ## Referenced Sources & Claims **Internal/company-sourced claims:** - Uber's **Submit Queue** system — a paper is referenced ("we have a paper on"), though no citation details are provided in the transcript. - Uber anecdote: losing the London operating license due to a feature-flag glitch, and the lineage/controls process introduced to regain it. - QAR platform metrics cited: commits ~10x over 3 years; ~25% month-over-month growth in PRs reviewed; **13% of PRs** had a recorded CI failure; auto-approval/automerge adoption across teams up to 100,000+ developers. - Internal CI optimization story: Rust build system, caching, compiler churn reduction, sharding — CI reduced from ~10 minutes to under a minute for incremental steps. **Third-party claims (attributed, unverifiable from transcript):** - GitHub industry data showing "staggering growth in PRs" — attributed to GitHub as corroboration. - GitHub merge queues and other code-host features referenced as context. - Regulatory frameworks named as examples: **HIPAA**, **SOX** ("Tora" per transcript), cited as compliance contexts. --- ## Promotional Segments — Flagged - **QAR/Guitar product promotion throughout**: closing direct call to action — >"would love for you folks to also try out guitar" — plus product capability claims (auto-approval policies, custom plain-English rules, code review that "caught it on our PR," agent fix suggestions with engineer as co-author) presented as examples. - The Uber fix vignette and approval-policy example double as **product demonstrations**; treat QAR performance claims as vendor claims, not independently verified. --- ## Evidence Channels & Availability - Transcript is the only supplied source; **no curated metadata, slides, links, or paper citations provided**. - The Submit Queue paper is referenced by existence only — no title, venue, or link in the transcript. - **Comment availability**: not stated in the source material. - Minor transcript quality notes: names rendered inconsistently ("Goautam," "QAR"/"Guitar"/"GAR"/"gita" all appear for the same company/speaker in the transcript); speaker identity resolved from the title metadata. Hello everyone. Uh welcome to my talk. Uh today we're going to be talking about how we are rebuilding CI for AI scale commit volume. And this comes from a lot of experience from build and merge infrastructure that I have been working on for a decade plus at this point. And kind of want to walk you through all the new bottlenecks we're seeing in software factories and also specifically for highly regulated industries and banks. How does that translate? How do we keep things more compliant and auditable while being able to move fast? So let's dive in. My name is Goautam. I am a co-founder and CTO at QAR and recently joined Sonar as a principal engineer. After the acquisition pre previous to I was at Uber for almost a decade working on build and merge infrastructure for mono repos. So everything from build, CI/CD, ids, you name it to make sure developers are productive. And now at QAR as part of Solar I'm trying to make sure that code that's been checked in is verified and all the code written by agents is of high quality before it goes in. So before I get started I want to just paint a bit of history picture. Uh at Uber we had agent scale coming volume before they were agents. So what I mean by this is we had one giant monor repo and we were doing like thousands of changes a day and with very long running builds. This meant that if you just kept everything serialized and merged them one at a time so as not to break your uh main branch you would take a really long time up to days so 20 days in this case to make sure that everything was green. And if he skipped everything and just let everyone merge to me, we would see red main all the time because changes would conflict with each other and even though the CI would be green on one of them combined, they might actually break. So we would see like our iOS monitor would be like red all the time and that was not productive for our developers. To fix this, what we did was we made a system called summit cube which we have a paper on and we tested all the likely possibilities of commits that are coming in uh and making sure that we can like reduce the wait time so we can test these things in parallel. We could also predict what could break by testing all the combinations. So once we know that a PR might break, we might remove it back from the queue and let the author know that they have to go fix it before they can push it back up after rebasing. And finally like to keep the trunk clean only if it passes with all the combination of commits coming in it would be let into go directly onto the trunk. So you want to make sure that the overhead was worth it for keeping our trunk clean. And this is still running today at Uber. I'm sure now we have like merge cues that GitHub came up with and all the other code hosts but for back then there was not a system. So we developed this. \[snorts\] So the problem with this approach is currently all these cues that we have. So that's either PCI or review are a centralized bottleneck in the SDLC. So all these cues were built for human speed and once you build it from human speed what happens is they are not able to keep up once you go to agents because humans generate changes used to generate changes like few times an hour. So we could do testing and review one at a time. Now with agents developers are writing like several PRs uh in many in a day and they don't always wait for the queue they all want to like come in at the same time which means your CI and review cues are piling up and having pressure and add to that like the patterns of software factories and background agents triggering on like various workflows suddenly you see like everything slowing down to a crawl because your bottlenecks are still going BCI in review and they're over capacity in this So what does this mean uh for teams today? Right? So just to put things into perspective uh our own uh rate of commits has almost like 10xed since we started guitar uh about 3 years ago and it's just like uh unbelievable with the same size team we're still very small we're able to like push so much more code uh and we even see this in the industry trends on guitar. So this graph shows you the number of PRs that we see that guitar reviews over time. And if you look at the growth trajectory, it's pretty exponential. Every month almost we're growing 25% month over month of just the amount of PRs that have to review fixed CI and other things. So this is also I think like corroborated by evidence that industry players like GitHub have been putting out where they're seeing like uh staggering growth in PRs and this is now coming for every single team not if our small team is seeing this I'm sure like the larger teams and banks are seeing it like at a much higher degree as they are going through their AI adoption journeys. \[snorts\] So the problem I was talking about right so if agents are opening PRs and if you make them uh check everything through the CI system uh individually they might pass but once you kind of merge them back on domain your truck might still be broken. So how do you like solve for this like how do you keep things green uh at this high rate of pace right so this is very important kind of like how we solve submit Q the changes themselves are not uh testing against each other so there needs to be some sort of change in either the CI system as well as how you architecture code and in the cues to be smarter to be able to solve this and the problem we're also seeing with everyone trying to adopt like software factories which is having agents write all the code uh from the spec and then just having humans be responsible for the spec and review is that the factory line that exists today was built for human speed as I mentioned which means that as agents push more changes uh every part of the factory line is going to be slowing down. So for example with build and test you might have lots of rebuilds whenever uh compilation breaks or you might have integration tests you might have flaky tests that are not reproducible locally and that just slows everything down. So because you have to retry them and it could take hours in many cases with a complex build. Add to the fact that now you have review and approval. So a lot of you probably already use like AI to review your code. Hopefully uh you're not doing every single review with with humans. You ideally are reviewing with AI as a first pass. Now the reviewers become the cue if you don't have that system in place. And even after you review, there might be some criteria you might want additionally to make sure that it's approved fully because what's happening now in the industry is a lot of folks are just rubber stamping them because they cannot keep up with the high rate of PR. They were relying on other signals now to be able to just get through. So the process has to change. And the other problem right now we're having is like a lot of these checks are running. There's uh a lot of records that we keep that are expiring. So like your old logs don't exist as they expire out. There is no stamp of like what was actually checked because there is like just an approval stamp. So there needs to be some record of these things for auditability and compliance checks to happen. The other issue we're seeing is that as the code volume increases the time taken for repair also is adding a ton of overhead. Uh what we see on our platform at least 13% of PRs had a recorded CI failure. This is just one or more CI failures. Which means in all these cases there is a separate round trip to go either back to the developer machine or for an agent to understand the logs root cause of failure fix it push another commit and then keep iterating till it's green. And this is just CI because most of the time if a human reviewer wants to look at a PR they're not going to take a look unless CI is green because that's the first line of defense. So that also like slows uh the review process because they can't go in parallel because usually we want to look at something once it's finalized and every lab is now paying for review. So if your review comments come in and you're doing another pass same at break again and it kind of like repeats the cycle. So how do we make this better like what what do we do to make this problem more manageable as agents are pushing down more code. So we have discovered this uh obviously back to but also in guitar steam right now. But the most obvious thing is of course like make your CI really quick and fast which means that you may want to like remove a lot of the expensive checks. Try to offload them to like post merge even and try to make the CI step itself really quick because your repairs will be much more efficient. So you can paralyze things, you can chart, you can do like uh caching. And we'll talk about some of the techniques that we have used as well. You also want to make sure the review is fast. So not just like a deep review, but you also want to have maybe multiple passes that are like giving quicker signal to your developers. So they're able to like use them as a way to understand in what scenarios they may want to go deeper for higher risk changes or maybe for like lower risk ones even like let the AI approve and merge them. And for the checks that are already running in CI, if you can move them to the left, it's way cheaper to catch them. And this actually uh is very relevant to checks that happen even past CI. For example, you might have like expensive regression tests or like things that might fail in production. Uh not every time it's possible to like move things to the left, but as much as possible, if you can move them, it means you have less bottlenecks down the line. And locally, agents have done more context anyway to fix the issue that they're facing. And you also want to have policies. This is very important for regulated industries. So a story we have at Back at Uber is we lost like the license to operate in London because there was a glitch in the software that allowed drivers to get on the platform without all the proper checks and that was because of a feature flag change. So we had to put in a process in place so that every change affecting London that was going through the SDLC would need some sort of like a a lineage of okay does this change affect London? If so, what are the controls in place and we were able to win our license back? Now, a lot of you might be facing these kind of like challenges where agents just make it much harder to understand what's happening in your codebase. So, how do you ensure your policies are actually being respected? So, you need to have a way to like enforce these rules beyond CI like just because the code compiles doesn't mean that it's following a particular specifications. It could be like HIPPA. It could be like uh Tora. There's so many other policies that these agents are not going to be aware of unless your agents are enforcing it on the other end. And also you want to make sure that you record that it was enforced for auditability. So just a quick story about how we made our CI faster as well. So we were at a baseline, we were I would say like still fast but like very annoying when you're trying to like push more code. uh we were around like 10 minutes for our CI and we wanted to like bring that down significantly. So we started think looking at how can we cache outputs uh we use rust for our build system. So we started like caching as much as possible and reusing them. And we also looked at ways to like reduce churn in the compiler. So we added some more fixes to like ensure that the compilation part is more cachable and is invalidated less often. And by adding more fixes on top of this, by sharding and by removing like these overheads, we're able to bring the time of CI almost like less than a minute for incremental steps, which means once the cold CI is warmed up, incremental builds are way way quick quicker and we can push a ton more code. The other issue is also review. So just because the CI is green, what was funny was when we were measuring our uh P90 time of like PRs making into production, the vast majority of time was spent in review back at Uber and even today to be honest like most teams spend more time reviewing code and doing other kinds of testing before they are comfortable merging the code into main. So this is something that needs to be addressed on top of CI as well because right now what's happening is agents are of course like generating code at the same time the AI agents are also like finding issues on on your uh code review system and these automated reviews are also findings that you would have to now go back and address. So either you have to pull context locally for your agent or have an agent on CI iterate and fix and humans are able to review whatever is left after the AI took its first pass. And this needs to be part of the equation so you're not wasting precious human cycles on more and more agent generated PRs. Uh here's an example for us. Uh we were refactoring our code base to have faster CI. So we were using next text to parallelize our uh tests and we had this like shared database that we were spinning up to make sure that we could set it up once and all our database tests would reuse it. But the problem was that there were some guards we didn't put in place which meant that when CI gets very busy uh there is like uh crosscontamination and test might become flaky. Guitar actually is a code reviewer. It caught it on our PR before we merged it. So we didn't have to waste time after. And GAR also fixed the issue for us because it suggested the fix, committed it, and added the engineer as a co-author. By doing this, we're able to like prevent expensive mistakes before the code made it all the way into main. The other thing that we do at Qar uh interestingly and as we see more and more of our customers start doing is we're using policies and criteria to decide when is it appropriate to actually even like approve changes. There's a lot of changes that are like routine or lower risk documentation dependency bumps of course if they're not breaking any APIs security patches uh even things like uh lower risk changes for internal tooling that can potentially be auto approved. Uh so guitar supports these things out of the box and you can also add additional custom rules which are plain English that you can use to like create the approval and merge on and we are seeing like teams uh who are like very AI native starting to adopt these more and more. We see I think on our platform the rate at which folks are adopting like the auto approval and automerge grow really quickly across teams even from large teams to like 100 plus,000 plus developers. So here's an example of approval policies. So if it's a dependency bump and doc changes, we should be approving them. Any changes under sensitive stuff like payments or o might need a human approver on top. And then of course you don't want to have the agent who made the change approve its change because that would not be uh zero trust. So in this case as agents are generating PRs the PRs are getting approved. When there is a new change that doesn't meet the criteria then we would have to escalate to another approver. And this can be done with your own agents of course but also gitas out of the box if you want to try it. And the other problem with uh just relying on CI today is that just because CI is green as I mentioned doesn't mean the code is compliant in many cases. So in this case uh the CI is passing and there is some metadata that we have about it on the rerun a flaky test uh is going to fail which means that when you force push to fix that thing now the branch is pointing at another piece of code. So the problem with this is that over time most CI data is going to be expired. So unless you have a record of what happened and what was fixed, you will not be able to like be compliant or be able to audit what actually happened. Which is why I think things like agent trajectories and like your uh outcomes after each push are very very important. So whatever information you can record on the PR or any other system of record is going to be a source of truth as your artifacts expire. So the control here is making sure that the change was approved and tested by someone who did not write it. If you don't have that information, it could be a problem. So here's an example. We have a pipeline that failed in the left and a pipeline that passed. But the difference here is that if you record properly why the failure occurred and someone else was able to approve it because it was a known failure mode and \[snorts\] it was saved with the change that made it in. This thing actually passes your checks versus something where you don't have a record of what happened but you just blindly trust the agent pushing code because it could be a test that was actually uh not high quality. There was no record of all the compliance checks that were like recorded. So you want to make sure that on the uh more sensitive uh industries you want to have all of these decisions recorded as rules or compliance checks on top of your PR in addition to code review. So the what are some of the kinds of records that we want to keep. So here's a few that we keep on which I think are just examples. It could be different for folks. So for example, which coding agent made uh the PR? Where did the PR originate from? What was the session that originated it? And who actually started it? Could it be a trigger from like automated workflow or could it be just an engineer doing prompting? And for each of these like CIS that run, what is a commit where CI was trusted and a save test report, not just a status? And making sure that each finding has like a record of how it was resolved. Was it dismissed? Was it followed up? Was it like fixed on the PR? And then making sure the approver is someone a apart from the bot that was prompted by the same person. And making sure that any overrides are explicitly named. So if there is a red build or like a control code coverage skipped, you want to make sure there's a reason for when why that check failed and why we're okay with that in that scenario. And making sure that the code that merges actually has all this and it's not like a small change on top that was merged. So all the checks are done every single time. So in this case we want to make sure that agents cannot approve their own changes. Engineer who prompted it is also not an author. And if you have these practices in place now we can actually start removing some of these bottlenecks because the CI being faster will unblock your builds. So your flow is much nicer and making sure you can like move some checks left will also make sure that your test load is going to be reduced quite a bit. On the CI side, if you can automate review for the most part and only escalate to humans for exceptional cases and potentially even auto approve lowrisk or internal changes, then you remove further bottlenecks and then you have less that has reaching humans uh for a software factory to perform to its uh potential. And of course after every single approval and merge you want to record what happened because as factories are going to ramp up if you don't have an idea of what happened in the past that context is going to be very useful on how to like iterate and make your software better over time. Uh that's it for me. Uh would love for you folks to also try out guitar and see how you can improve your software factories. And with that uh thank you so much for for your