--- schema: 1 type: video title: 'Benchmarking semantic code retrieval on Claude Code — Kuba Rogut, Turbopuffer' description: >- By default, Claude Code wastes one in every three file reads. Add windowed grep and that drops to one in five. Add semantic search on top and it drops to one in eight, with file precision climbing from 65% to 87%. Kuba Rogut from Turbopuffer ran a 50-task benchmark against ContextBench to measure not whether the agent solved the problem but whether it found the right files, lines, and symbols along the way. The benchmark tested three conditions: raw Claude Code, windowed reads capped at 50 lines, and windowed reads plus a semantic search tool backed by Turbopuffer. Semantic search won on behavior adjacent tasks where files share no keywords. Grep won on import tracing where the keyword is right there. Cursor's production numbers show a 24% relative improvement in answer accuracy from semantic retrieval, plus a 2.6% increase in code retention in large codebases. Kuba's explanation for why his gains were smaller: Cursor's model knows when and why to call semantic search. Claude Code just has it as another tool in the list. Speaker info: - https://ca.linkedin.com/in/kubarogut - https://rogutkuba.com/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/zKk7sDMGDEQ-benchmarking-semantic-code-retrieval-on-claude-code Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/zKk7sDMGDEQ' markdown: 'https://liquidgold.cc/watch/zKk7sDMGDEQ.md' llms: 'https://liquidgold.cc/watch/zKk7sDMGDEQ/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T09:57:02.562Z' language: null children: [] videoId: zKk7sDMGDEQ durationSeconds: 968 publishedAt: '2026-06-03T15:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 25 endSeconds: 157 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=25' heldReason: rights - ordinal: 1 text: null startSeconds: 25 endSeconds: 157 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=25' heldReason: rights - ordinal: 2 text: null startSeconds: 157 endSeconds: 168 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=157' heldReason: rights - ordinal: 3 text: null startSeconds: 168 endSeconds: 295 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=168' heldReason: rights - ordinal: 4 text: null startSeconds: 168 endSeconds: 295 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=168' heldReason: rights - ordinal: 5 text: null startSeconds: 306 endSeconds: 427 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=306' heldReason: rights - ordinal: 6 text: null startSeconds: 306 endSeconds: 427 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=306' heldReason: rights - ordinal: 7 text: null startSeconds: 427 endSeconds: 437 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=427' heldReason: rights - ordinal: 8 text: null startSeconds: 437 endSeconds: 526 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=437' heldReason: rights - ordinal: 9 text: null startSeconds: 538 endSeconds: 626 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=538' heldReason: rights - ordinal: 10 text: null startSeconds: 538 endSeconds: 626 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=538' heldReason: rights - ordinal: 11 text: null startSeconds: 736 endSeconds: 746 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=736' heldReason: rights - ordinal: 12 text: null startSeconds: 746 endSeconds: 834 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=746' heldReason: rights - ordinal: 13 text: null startSeconds: 845 endSeconds: 968 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=845' heldReason: rights - ordinal: 14 text: null startSeconds: 845 endSeconds: 968 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=845' heldReason: rights - ordinal: 15 text: null startSeconds: 636 endSeconds: 736 watch: 'https://liquidgold.cc/watch/zKk7sDMGDEQ?t=636' heldReason: rights --- # Benchmarking semantic code retrieval on Claude Code — Kuba Rogut, Turbopuffer By default, Claude Code wastes one in every three file reads. Add windowed grep and that drops to one in five. Add semantic search on top and it drops to one in eight, with file precision climbing from 65% to 87%. Kuba Rogut from Turbopuffer ran a 50-task benchmark against ContextBench to measure not whether the agent solved the problem but whether it found the right files, lines, and symbols along the way. The benchmark tested three conditions: raw Claude Code, windowed reads capped at 50 lines, and windowed reads plus a semantic search tool backed by Turbopuffer. Semantic search won on behavior adjacent tasks where files share no keywords. Grep won on import tracing where the keyword is right there. Cursor's production numbers show a 24% relative improvement in answer accuracy from semantic retrieval, plus a 2.6% increase in code retention in large codebases. Kuba's explanation for why his gains were smaller: Cursor's model knows when and why to call semantic search. Claude Code just has it as another tool in the list. Speaker info: - https://ca.linkedin.com/in/kubarogut - https://rogutkuba.com/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/zKk7sDMGDEQ-benchmarking-semantic-code-retrieval-on-claude-code Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/zKk7sDMGDEQ · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.