--- schema: 1 type: video title: 'Using LLMs to Secure Source Code — Eugene Yan, Anthropic' description: >- Mozilla shipped about 20 security fixes a month across Firefox in early 2025. In April it shipped 400, a 20x jump, and it credited roughly two thirds of them to a frontier model. That is the shift Eugene Yan came to describe: models are now finding and fixing real vulnerabilities at scale. Anthropic's own scan of more than a thousand open source repos surfaced 6,200 high or critical issues out of 23,000 candidates, reported 1,600 to maintainers, and saw about 100 patched upstream. Finding bugs, it turns out, is no longer the hard part. The bottleneck has moved to verifying, triaging, and patching them. The talk walks a six step workflow through one running example: a five line order lookup with a SQL injection hiding in a Python string. The two setup steps are a threat model and a sandbox. A written threat model alone pushes the true positive rate to 90%, because a model has great context of the code but poor context of the system, all the design decisions that live only in someone's head. The four loop steps read like a machine learning pipeline: discovery optimizes for recall, then a separate verification agent, kept independent and adversarial so it never sees the discovery reasoning, optimizes for precision by detonating the exploit in a fresh container. Triage protects the scarcest resource, engineer attention, and patching closes the loop so the same bug cannot return. His parting advice: start this week on open source dependencies, keep your hands on the wheel before automating, and remember that scanning was never the bottleneck. Speaker info: - https://x.com/eugeneyan - https://github.com/eugeneyan - https://eugeneyan.com Timestamps: 0:00 - Working with security teams to find and fix vulnerabilities 0:49 - Three trends in model security capability 1:16 - Cybersecurity benchmarks and the step jump in capability 1:54 - Mozilla's 20x jump in monthly security fixes 2:44 - Log4Shell, Heartbleed, and why this matters 3:22 - Anthropic's scan of a thousand open source repos 3:35 - The bottleneck shifts to verify, triage, and patch 3:48 - Why agentic harnesses changed the game 4:29 - The six step workflow 5:31 - A running example: the order service 5:45 - Step 1: the threat model and 90% true positives 7:42 - Step 2: the sandbox for isolation and reproducibility 9:24 - Step 3: discovery and the five line SQL injection 11:44 - Step 4: independent adversarial verification 13:36 - Step 5: triage and the scarcity of engineer attention 15:52 - Step 6: patching and closing the loop 17:19 - It all looks like a machine learning pipeline 17:43 - The non technical bottlenecks are harder 18:47 - Organizational bottlenecks: routing, severity, bandwidth 20:05 - Three takeaways and how to start this week "The bottleneck has now shifted to verification, triage, and patching." (3:39) "A model has great context of the code but poor context of the system." (6:06) "Things that can be solved with money are not really problems. But human attention doesn't scale." (18:14) "Scanning was never the bottleneck." (20:38) canonical: 'https://liquidgold.cc/watch/imFedndyXYQ' markdown: 'https://liquidgold.cc/watch/imFedndyXYQ.md' llms: 'https://liquidgold.cc/watch/imFedndyXYQ/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T07:31:56.461Z' language: null children: [] videoId: imFedndyXYQ durationSeconds: 1290 publishedAt: '2026-07-17T21:27:20.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 91 endSeconds: 114 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=91' heldReason: rights - ordinal: 1 text: null startSeconds: 114 endSeconds: 126 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=114' heldReason: rights - ordinal: 2 text: null startSeconds: 208 endSeconds: 219 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=208' heldReason: rights - ordinal: 3 text: null startSeconds: 219 endSeconds: 230 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=219' heldReason: rights - ordinal: 4 text: null startSeconds: 241 endSeconds: 274 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=241' heldReason: rights - ordinal: 5 text: null startSeconds: 366 endSeconds: 468 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=366' heldReason: rights - ordinal: 6 text: null startSeconds: 366 endSeconds: 468 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=366' heldReason: rights - ordinal: 7 text: null startSeconds: 480 endSeconds: 570 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=480' heldReason: rights - ordinal: 8 text: null startSeconds: 480 endSeconds: 570 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=480' heldReason: rights - ordinal: 9 text: null startSeconds: 580 endSeconds: 714 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=580' heldReason: rights - ordinal: 10 text: null startSeconds: 580 endSeconds: 714 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=580' heldReason: rights - ordinal: 11 text: null startSeconds: 714 endSeconds: 724 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=714' heldReason: rights - ordinal: 12 text: null startSeconds: 724 endSeconds: 824 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=724' heldReason: rights - ordinal: 13 text: null startSeconds: 724 endSeconds: 824 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=724' heldReason: rights - ordinal: 14 text: null startSeconds: 837 endSeconds: 958 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=837' heldReason: rights - ordinal: 15 text: null startSeconds: 837 endSeconds: 958 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=837' heldReason: rights - ordinal: 16 text: null startSeconds: 969 endSeconds: 1045 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=969' heldReason: rights - ordinal: 17 text: null startSeconds: 1056 endSeconds: 1066 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=1056' heldReason: rights - ordinal: 18 text: null startSeconds: 1131 endSeconds: 1142 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=1131' heldReason: rights - ordinal: 19 text: null startSeconds: 1210 endSeconds: 1221 watch: 'https://liquidgold.cc/watch/imFedndyXYQ?t=1210' heldReason: rights --- # Using LLMs to Secure Source Code — Eugene Yan, Anthropic Mozilla shipped about 20 security fixes a month across Firefox in early 2025. In April it shipped 400, a 20x jump, and it credited roughly two thirds of them to a frontier model. That is the shift Eugene Yan came to describe: models are now finding and fixing real vulnerabilities at scale. Anthropic's own scan of more than a thousand open source repos surfaced 6,200 high or critical issues out of 23,000 candidates, reported 1,600 to maintainers, and saw about 100 patched upstream. Finding bugs, it turns out, is no longer the hard part. The bottleneck has moved to verifying, triaging, and patching them. The talk walks a six step workflow through one running example: a five line order lookup with a SQL injection hiding in a Python string. The two setup steps are a threat model and a sandbox. A written threat model alone pushes the true positive rate to 90%, because a model has great context of the code but poor context of the system, all the design decisions that live only in someone's head. The four loop steps read like a machine learning pipeline: discovery optimizes for recall, then a separate verification agent, kept independent and adversarial so it never sees the discovery reasoning, optimizes for precision by detonating the exploit in a fresh container. Triage protects the scarcest resource, engineer attention, and patching closes the loop so the same bug cannot return. His parting advice: start this week on open source dependencies, keep your hands on the wheel before automating, and remember that scanning was never the bottleneck. Speaker info: - https://x.com/eugeneyan - https://github.com/eugeneyan - https://eugeneyan.com Timestamps: 0:00 - Working with security teams to find and fix vulnerabilities 0:49 - Three trends in model security capability 1:16 - Cybersecurity benchmarks and the step jump in capability 1:54 - Mozilla's 20x jump in monthly security fixes 2:44 - Log4Shell, Heartbleed, and why this matters 3:22 - Anthropic's scan of a thousand open source repos 3:35 - The bottleneck shifts to verify, triage, and patch 3:48 - Why agentic harnesses changed the game 4:29 - The six step workflow 5:31 - A running example: the order service 5:45 - Step 1: the threat model and 90% true positives 7:42 - Step 2: the sandbox for isolation and reproducibility 9:24 - Step 3: discovery and the five line SQL injection 11:44 - Step 4: independent adversarial verification 13:36 - Step 5: triage and the scarcity of engineer attention 15:52 - Step 6: patching and closing the loop 17:19 - It all looks like a machine learning pipeline 17:43 - The non technical bottlenecks are harder 18:47 - Organizational bottlenecks: routing, severity, bandwidth 20:05 - Three takeaways and how to start this week "The bottleneck has now shifted to verification, triage, and patching." (3:39) "A model has great context of the code but poor context of the system." (6:06) "Things that can be solved with money are not really problems. But human attention doesn't scale." (18:14) "Scanning was never the bottleneck." (20:38) Canonical: https://liquidgold.cc/watch/imFedndyXYQ · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.