--- schema: 1 type: video title: 'Stop Chunking Like It''s 2022 — Yuval Belfer, AI21 Labs' description: >- Ask a corpus of Seinfeld transcripts for the name of Jerry's favorite church and a 100 token chunk returns it at rank one, while every larger window buries it below rank fifty. Ask the same corpus who Jerry calls his nemesis and pure evil and the small chunks fail completely, because the answer is spread across a scene rather than sitting in a sentence. Same data, same index, opposite requirements. Yuval Belfer uses that pair to make a claim most retrieval teams have quietly assumed away: there is no correct chunk size, because the correct size is a property of the query, and you pick it at indexing time when you do not yet have any queries. That is the trap in one sentence. At indexing you control the window and know nothing about the questions. At retrieval you have the question and the window is already frozen. To size the cost, his team duplicated several datasets at six different chunk sizes and ran an oracle experiment, choosing per query the size that happened to work best. The gap between that oracle and any single fixed choice ran 20 to 40 percent of recall, which is what an arbitrary 512 has been quietly costing. Their fix refuses the premise rather than tuning it. Index the corpus at every window size, query all of them, and because chunks of different sizes cannot be compared, return whole documents so the rankings become commensurable, then merge them by reciprocal rank fusion. It is a short script rather than a model. The honest accounting is at the end: two to five times the memory, and almost no added latency. Speaker info: - https://x.com/yuvalinthedeep - https://linkedin.com/in/yuval-belfer Timestamps: 0:00 - A talk about nothing, and why chunking 1:43 - Indexing is boring, retrieval tuning is fun 3:23 - A World Cup directory that cannot answer the query 4:17 - Chunking as lossy compression 6:13 - Six copies of the same dataset 7:07 - Two Seinfeld questions, opposite answers 8:12 - The oracle experiment 9:59 - An information problem at both ends 11:02 - Multiscale indexing and rank fusion 13:50 - Results across four datasets 15:24 - What it costs in memory --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/r9OwPx_HoV0-youtube-53783b315e0664562cfb Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/stop-chunking-like-it-s-2022-yuval-belfer-ai21-labs-c06ec25f2e4a markdown: >- https://liquidgold.cc/channel/aidotengineer/video/stop-chunking-like-it-s-2022-yuval-belfer-ai21-labs-c06ec25f2e4a.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T03:24:57.427Z' language: null children: [] videoId: r9OwPx_HoV0 durationSeconds: 1081 publishedAt: '2026-09-16T17:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 26 endSeconds: 106 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=26' heldReason: rights - ordinal: 1 text: null startSeconds: 119 endSeconds: 211 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=119' heldReason: rights - ordinal: 2 text: null startSeconds: 119 endSeconds: 211 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=119' heldReason: rights - ordinal: 3 text: null startSeconds: 222 endSeconds: 265 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=222' heldReason: rights - ordinal: 4 text: null startSeconds: 275 endSeconds: 376 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=275' heldReason: rights - ordinal: 5 text: null startSeconds: 275 endSeconds: 376 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=275' heldReason: rights - ordinal: 6 text: null startSeconds: 433 endSeconds: 444 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=433' heldReason: rights - ordinal: 7 text: null startSeconds: 503 endSeconds: 514 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=503' heldReason: rights - ordinal: 8 text: null startSeconds: 514 endSeconds: 602 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=514' heldReason: rights - ordinal: 9 text: null startSeconds: 614 endSeconds: 668 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=614' heldReason: rights - ordinal: 10 text: null startSeconds: 668 endSeconds: 679 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=668' heldReason: rights - ordinal: 11 text: null startSeconds: 668 endSeconds: 679 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=668' heldReason: rights - ordinal: 12 text: null startSeconds: 679 endSeconds: 836 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=679' heldReason: rights - ordinal: 13 text: null startSeconds: 679 endSeconds: 836 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=679' heldReason: rights - ordinal: 14 text: null startSeconds: 846 endSeconds: 925 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=846' heldReason: rights - ordinal: 15 text: null startSeconds: 925 endSeconds: 936 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=925' heldReason: rights - ordinal: 16 text: null startSeconds: 936 endSeconds: 1081 watch: 'https://liquidgold.cc/watch/r9OwPx_HoV0?t=936' heldReason: rights --- # Stop Chunking Like It's 2022 — Yuval Belfer, AI21 Labs Ask a corpus of Seinfeld transcripts for the name of Jerry's favorite church and a 100 token chunk returns it at rank one, while every larger window buries it below rank fifty. Ask the same corpus who Jerry calls his nemesis and pure evil and the small chunks fail completely, because the answer is spread across a scene rather than sitting in a sentence. Same data, same index, opposite requirements. Yuval Belfer uses that pair to make a claim most retrieval teams have quietly assumed away: there is no correct chunk size, because the correct size is a property of the query, and you pick it at indexing time when you do not yet have any queries. That is the trap in one sentence. At indexing you control the window and know nothing about the questions. At retrieval you have the question and the window is already frozen. To size the cost, his team duplicated several datasets at six different chunk sizes and ran an oracle experiment, choosing per query the size that happened to work best. The gap between that oracle and any single fixed choice ran 20 to 40 percent of recall, which is what an arbitrary 512 has been quietly costing. Their fix refuses the premise rather than tuning it. Index the corpus at every window size, query all of them, and because chunks of different sizes cannot be compared, return whole documents so the rankings become commensurable, then merge them by reciprocal rank fusion. It is a short script rather than a model. The honest accounting is at the end: two to five times the memory, and almost no added latency. Speaker info: - https://x.com/yuvalinthedeep - https://linkedin.com/in/yuval-belfer Timestamps: 0:00 - A talk about nothing, and why chunking 1:43 - Indexing is boring, retrieval tuning is fun 3:23 - A World Cup directory that cannot answer the query 4:17 - Chunking as lossy compression 6:13 - Six copies of the same dataset 7:07 - Two Seinfeld questions, opposite answers 8:12 - The oracle experiment 9:59 - An information problem at both ends 11:02 - Multiscale indexing and rank fusion 13:50 - Results across four datasets 15:24 - What it costs in memory --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/r9OwPx_HoV0-youtube-53783b315e0664562cfb Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/stop-chunking-like-it-s-2022-yuval-belfer-ai21-labs-c06ec25f2e4a · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.