--- schema: 1 type: video title: 'Benchmarking Coding Agents on New vs Legacy Codebases — Denys Linkov, Wisedocs' description: >- Wisedocs processes medical claims that arrive as PDFs over 10,000 pages long, some of them larger than video files, through a pipeline of ML models spread across ten repositories nobody enjoyed touching. Denys Linkov's team spent six months collapsing that into a monorepo, and this talk is an honest audit of whether they should have just waited for the models to get good enough to do it for them. The benchmark he keeps coming back to is a single refactor task. With o3 it took three hours of back and forth in Cursor and still shipped ten major mistakes. Rerun on newer models, Sonnet 4.6 needed one extra iteration and Opus 4.8 essentially got it in one pass, at roughly a fifth of the original effort. The counterweight is what happens when you hand a current model the whole job. GPT 5.5 extra high declared the refactor done in 10 minutes 22 seconds and wrote 2,000 lines, which turned out to be scaffolding with the actual models missing, something it admitted in its own output by noting it had not added the deployment or bootstrap command yet. That gap is why Linkov reads the METR task length curve at 80% or 90% success instead of the usual 50%. Launching an hour long agent run on coin flip odds mostly buys you a wasted hour and a broken attention span. His verdict is that doing the refactor beat deferring it, and the evidence is as much social as technical: commit velocity rose and never flattened, work that used to take months ships in under a week, and developers across the company now volunteer into the repo even outside their own area, which was never true of the ten it replaced. Speaker info: - https://x.com/denyslinkov - https://www.linkedin.com/in/denyslinkov/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/7vn4WpqNpck-benchmarking-coding-agents-on-new-vs-legacy-codebases-denys-linkov-wisedocs Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/7vn4WpqNpck' markdown: 'https://liquidgold.cc/watch/7vn4WpqNpck.md' llms: 'https://liquidgold.cc/watch/7vn4WpqNpck/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T05:41:19.567Z' language: null children: [] videoId: 7vn4WpqNpck durationSeconds: 1088 publishedAt: '2026-08-08T19:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 22 endSeconds: 110 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=22' heldReason: rights - ordinal: 1 text: null startSeconds: 22 endSeconds: 110 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=22' heldReason: rights - ordinal: 2 text: null startSeconds: 110 endSeconds: 121 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=110' heldReason: rights - ordinal: 3 text: null startSeconds: 121 endSeconds: 211 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=121' heldReason: rights - ordinal: 4 text: null startSeconds: 221 endSeconds: 309 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=221' heldReason: rights - ordinal: 5 text: null startSeconds: 221 endSeconds: 309 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=221' heldReason: rights - ordinal: 6 text: null startSeconds: 221 endSeconds: 309 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=221' heldReason: rights - ordinal: 7 text: null startSeconds: 320 endSeconds: 401 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=320' heldReason: rights - ordinal: 8 text: null startSeconds: 320 endSeconds: 401 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=320' heldReason: rights - ordinal: 9 text: null startSeconds: 411 endSeconds: 512 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=411' heldReason: rights - ordinal: 10 text: null startSeconds: 522 endSeconds: 611 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=522' heldReason: rights - ordinal: 11 text: null startSeconds: 611 endSeconds: 621 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=611' heldReason: rights - ordinal: 12 text: null startSeconds: 621 endSeconds: 708 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=621' heldReason: rights - ordinal: 13 text: null startSeconds: 708 endSeconds: 719 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=708' heldReason: rights - ordinal: 14 text: null startSeconds: 719 endSeconds: 849 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=719' heldReason: rights - ordinal: 15 text: null startSeconds: 719 endSeconds: 849 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=719' heldReason: rights - ordinal: 16 text: null startSeconds: 849 endSeconds: 859 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=849' heldReason: rights - ordinal: 17 text: null startSeconds: 859 endSeconds: 959 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=859' heldReason: rights - ordinal: 18 text: null startSeconds: 969 endSeconds: 1088 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=969' heldReason: rights - ordinal: 19 text: null startSeconds: 969 endSeconds: 1088 watch: 'https://liquidgold.cc/watch/7vn4WpqNpck?t=969' heldReason: rights --- # Benchmarking Coding Agents on New vs Legacy Codebases — Denys Linkov, Wisedocs Wisedocs processes medical claims that arrive as PDFs over 10,000 pages long, some of them larger than video files, through a pipeline of ML models spread across ten repositories nobody enjoyed touching. Denys Linkov's team spent six months collapsing that into a monorepo, and this talk is an honest audit of whether they should have just waited for the models to get good enough to do it for them. The benchmark he keeps coming back to is a single refactor task. With o3 it took three hours of back and forth in Cursor and still shipped ten major mistakes. Rerun on newer models, Sonnet 4.6 needed one extra iteration and Opus 4.8 essentially got it in one pass, at roughly a fifth of the original effort. The counterweight is what happens when you hand a current model the whole job. GPT 5.5 extra high declared the refactor done in 10 minutes 22 seconds and wrote 2,000 lines, which turned out to be scaffolding with the actual models missing, something it admitted in its own output by noting it had not added the deployment or bootstrap command yet. That gap is why Linkov reads the METR task length curve at 80% or 90% success instead of the usual 50%. Launching an hour long agent run on coin flip odds mostly buys you a wasted hour and a broken attention span. His verdict is that doing the refactor beat deferring it, and the evidence is as much social as technical: commit velocity rose and never flattened, work that used to take months ships in under a week, and developers across the company now volunteer into the repo even outside their own area, which was never true of the ten it replaced. Speaker info: - https://x.com/denyslinkov - https://www.linkedin.com/in/denyslinkov/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/7vn4WpqNpck-benchmarking-coding-agents-on-new-vs-legacy-codebases-denys-linkov-wisedocs Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/7vn4WpqNpck · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.