--- schema: 1 type: video title: >- The unreasonable effectiveness of BM25 for agentic search — Jo Kristian Bergum, Hornet.dev description: >- BM25 stands for Best Match 25, and the number is not a version. A group of researchers ran a long series of scoring experiments decades ago, the twenty fifth one worked best, and the name simply stuck. Jo Kristian Bergum has spent more than twenty years on search and retrieval, and his claim is that this thirty year old lexical function is making a comeback without having changed at all. What changed is the user. A model already knows entities, companies, dates, postal codes and product identifiers, so it can write queries that are far longer and far more specific than anything a person would type, and it can fire off a dozen in a row. The old AOL query logs showed people searching in two or three words, and human query logs still look about the same today. An agent is a fundamentally more powerful user of a dumb tool. The sharpest evidence comes from a deep research benchmark of 830 riddle like questions over roughly one hundred thousand web documents. Stuff the answer bearing documents directly into the context window and accuracy is high, even for older models, which means reasoning was never the bottleneck. Hand the same model a search tool instead and accuracy drops, because now it depends on query formulation and on the retriever. Bergum compares a context window to a floppy disc, about 1.4 megabytes then and roughly 350,000 tokens now before quality degrades, so something still has to decide what goes in. He closes on a pattern he likes: dump retrieved documents into a file system workspace and let the model use grep and the other primitives it is already trained on. Speaker info: - https://x.com/jobergum - https://www.linkedin.com/in/jo-bergum - https://hornet.dev/ Timestamps: 0:00 - A thirty year old scoring function makes a comeback 2:19 - Best Match 25, and where the name came from 3:37 - The function did not change, the user did 4:57 - A benchmark of 830 riddles 6:01 - Context windows are floppy discs 7:05 - Reasoning is not the bottleneck 8:37 - How a model formulates queries 9:44 - Which BM25 do you mean? 12:06 - Retrieved documents as a file system 14:42 - Classical evaluation is dead 16:32 - Four claims to take away --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/fZH97QHHYjY-youtube-02abd6378435a976ca23 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/the-unreasonable-effectiveness-of-bm25-for-agentic-4291a26a9f58 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/the-unreasonable-effectiveness-of-bm25-for-agentic-4291a26a9f58.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T03:36:51.725Z' language: null children: [] videoId: fZH97QHHYjY durationSeconds: 1109 publishedAt: '2026-09-16T13:30:32.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 23 endSeconds: 139 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=23' heldReason: rights - ordinal: 1 text: null startSeconds: 139 endSeconds: 150 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=139' heldReason: rights - ordinal: 2 text: null startSeconds: 150 endSeconds: 225 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=150' heldReason: rights - ordinal: 3 text: null startSeconds: 235 endSeconds: 304 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=235' heldReason: rights - ordinal: 4 text: null startSeconds: 304 endSeconds: 315 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=304' heldReason: rights - ordinal: 5 text: null startSeconds: 315 endSeconds: 361 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=315' heldReason: rights - ordinal: 6 text: null startSeconds: 361 endSeconds: 373 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=361' heldReason: rights - ordinal: 7 text: null startSeconds: 373 endSeconds: 425 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=373' heldReason: rights - ordinal: 8 text: null startSeconds: 437 endSeconds: 517 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=437' heldReason: rights - ordinal: 9 text: null startSeconds: 437 endSeconds: 517 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=437' heldReason: rights - ordinal: 10 text: null startSeconds: 527 endSeconds: 592 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=527' heldReason: rights - ordinal: 11 text: null startSeconds: 527 endSeconds: 592 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=527' heldReason: rights - ordinal: 12 text: null startSeconds: 603 endSeconds: 732 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=603' heldReason: rights - ordinal: 13 text: null startSeconds: 603 endSeconds: 732 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=603' heldReason: rights - ordinal: 14 text: null startSeconds: 603 endSeconds: 732 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=603' heldReason: rights - ordinal: 15 text: null startSeconds: 744 endSeconds: 882 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=744' heldReason: rights - ordinal: 16 text: null startSeconds: 744 endSeconds: 882 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=744' heldReason: rights - ordinal: 17 text: null startSeconds: 744 endSeconds: 882 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=744' heldReason: rights - ordinal: 18 text: null startSeconds: 893 endSeconds: 1000 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=893' heldReason: rights - ordinal: 19 text: null startSeconds: 893 endSeconds: 1000 watch: 'https://liquidgold.cc/watch/fZH97QHHYjY?t=893' heldReason: rights --- # The unreasonable effectiveness of BM25 for agentic search — Jo Kristian Bergum, Hornet.dev BM25 stands for Best Match 25, and the number is not a version. A group of researchers ran a long series of scoring experiments decades ago, the twenty fifth one worked best, and the name simply stuck. Jo Kristian Bergum has spent more than twenty years on search and retrieval, and his claim is that this thirty year old lexical function is making a comeback without having changed at all. What changed is the user. A model already knows entities, companies, dates, postal codes and product identifiers, so it can write queries that are far longer and far more specific than anything a person would type, and it can fire off a dozen in a row. The old AOL query logs showed people searching in two or three words, and human query logs still look about the same today. An agent is a fundamentally more powerful user of a dumb tool. The sharpest evidence comes from a deep research benchmark of 830 riddle like questions over roughly one hundred thousand web documents. Stuff the answer bearing documents directly into the context window and accuracy is high, even for older models, which means reasoning was never the bottleneck. Hand the same model a search tool instead and accuracy drops, because now it depends on query formulation and on the retriever. Bergum compares a context window to a floppy disc, about 1.4 megabytes then and roughly 350,000 tokens now before quality degrades, so something still has to decide what goes in. He closes on a pattern he likes: dump retrieved documents into a file system workspace and let the model use grep and the other primitives it is already trained on. Speaker info: - https://x.com/jobergum - https://www.linkedin.com/in/jo-bergum - https://hornet.dev/ Timestamps: 0:00 - A thirty year old scoring function makes a comeback 2:19 - Best Match 25, and where the name came from 3:37 - The function did not change, the user did 4:57 - A benchmark of 830 riddles 6:01 - Context windows are floppy discs 7:05 - Reasoning is not the bottleneck 8:37 - How a model formulates queries 9:44 - Which BM25 do you mean? 12:06 - Retrieved documents as a file system 14:42 - Classical evaluation is dead 16:32 - Four claims to take away --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/fZH97QHHYjY-youtube-02abd6378435a976ca23 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/the-unreasonable-effectiveness-of-bm25-for-agentic-4291a26a9f58 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.