--- schema: 1 type: video title: 'Guide, Verify, Solve — Anirban Chatterjee, Sonar' description: >- A Carnegie Mellon study sorted GitHub projects by whether an AI tool wrote the code, and found the productivity gain ran out after about three months while the static analysis warnings and the added complexity stayed. That residue is verification debt, and how much it costs scales with criticality: a short lived internal tool can live with the gap between the quality a model gives you and the quality the application needs, a large codebase with adversarial users cannot. The obvious backstop is human review, and a Wharton study suggests it leaks badly. Participants took the AI's advice 92.7% of the time when it was correct, and still followed it nearly 80% of the time when it had been instructed to lie confidently. Anirban Chatterjee's argument is that the check has to be zero trust and multi layered. Zero trust means assuming the code could have come from anywhere and verifying it by a different method than the one that wrote it, since a model grading its own output inherits its own blind spots. Multi layered means computational review running alongside reasoning based review, because no single technique catches syntax, data flow, architecture and control flow at once. Sonar's leaderboard makes the blind spots concrete: across their metrics one Claude model rates well on correctness and reliability while the other is the better choice when maintainability, security or lower complexity is what matters. The loop he proposes wraps generation on both sides, handing the agent architectural constraints and coding standards before it starts, running verification inside the inner loop so issues get fixed before they propagate into later loops, and giving the agent the tools to remediate what comes back instead of queueing it for a person who is already rubber stamping too much. Speaker info: - https://www.linkedin.com/in/anirbanc/ - https://www.sonarsource.com/the-coding-personalities-of-leading-llms/leaderboard/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/03l29gJXpCE-guide-verify-solve-the-engineering-discipline-agentic-development-demands Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/03l29gJXpCE' markdown: 'https://liquidgold.cc/watch/03l29gJXpCE.md' llms: 'https://liquidgold.cc/watch/03l29gJXpCE/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T05:46:31.912Z' language: null children: [] videoId: 03l29gJXpCE durationSeconds: 1351 publishedAt: '2026-08-09T17:45:13.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 126 endSeconds: 136 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=126' heldReason: rights - ordinal: 1 text: null startSeconds: 136 endSeconds: 224 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=136' heldReason: rights - ordinal: 2 text: null startSeconds: 136 endSeconds: 224 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=136' heldReason: rights - ordinal: 3 text: null startSeconds: 235 endSeconds: 367 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=235' heldReason: rights - ordinal: 4 text: null startSeconds: 235 endSeconds: 367 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=235' heldReason: rights - ordinal: 5 text: null startSeconds: 377 endSeconds: 474 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=377' heldReason: rights - ordinal: 6 text: null startSeconds: 485 endSeconds: 598 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=485' heldReason: rights - ordinal: 7 text: null startSeconds: 609 endSeconds: 709 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=609' heldReason: rights - ordinal: 8 text: null startSeconds: 609 endSeconds: 709 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=609' heldReason: rights - ordinal: 9 text: null startSeconds: 609 endSeconds: 709 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=609' heldReason: rights - ordinal: 10 text: null startSeconds: 721 endSeconds: 809 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=721' heldReason: rights - ordinal: 11 text: null startSeconds: 928 endSeconds: 1038 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=928' heldReason: rights - ordinal: 12 text: null startSeconds: 1038 endSeconds: 1050 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=1038' heldReason: rights - ordinal: 13 text: null startSeconds: 1050 endSeconds: 1138 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=1050' heldReason: rights - ordinal: 14 text: null startSeconds: 1050 endSeconds: 1138 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=1050' heldReason: rights - ordinal: 15 text: null startSeconds: 918 endSeconds: 928 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=918' heldReason: rights - ordinal: 16 text: null startSeconds: 1246 endSeconds: 1351 watch: 'https://liquidgold.cc/watch/03l29gJXpCE?t=1246' heldReason: rights --- # Guide, Verify, Solve — Anirban Chatterjee, Sonar A Carnegie Mellon study sorted GitHub projects by whether an AI tool wrote the code, and found the productivity gain ran out after about three months while the static analysis warnings and the added complexity stayed. That residue is verification debt, and how much it costs scales with criticality: a short lived internal tool can live with the gap between the quality a model gives you and the quality the application needs, a large codebase with adversarial users cannot. The obvious backstop is human review, and a Wharton study suggests it leaks badly. Participants took the AI's advice 92.7% of the time when it was correct, and still followed it nearly 80% of the time when it had been instructed to lie confidently. Anirban Chatterjee's argument is that the check has to be zero trust and multi layered. Zero trust means assuming the code could have come from anywhere and verifying it by a different method than the one that wrote it, since a model grading its own output inherits its own blind spots. Multi layered means computational review running alongside reasoning based review, because no single technique catches syntax, data flow, architecture and control flow at once. Sonar's leaderboard makes the blind spots concrete: across their metrics one Claude model rates well on correctness and reliability while the other is the better choice when maintainability, security or lower complexity is what matters. The loop he proposes wraps generation on both sides, handing the agent architectural constraints and coding standards before it starts, running verification inside the inner loop so issues get fixed before they propagate into later loops, and giving the agent the tools to remediate what comes back instead of queueing it for a person who is already rubber stamping too much. Speaker info: - https://www.linkedin.com/in/anirbanc/ - https://www.sonarsource.com/the-coding-personalities-of-leading-llms/leaderboard/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/03l29gJXpCE-guide-verify-solve-the-engineering-discipline-agentic-development-demands Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/03l29gJXpCE · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.