--- schema: 1 type: video title: >- Connect AI to Billions of Legal Documents — Simon Eskildsen, turbopuffer & Jacob Lauritzen, Legora description: >- Legora's search latency once went from a 100 millisecond P99 to twenty seconds, and the cause was a packing problem invisible in the schema. Legal work arrives as projects, a project gets worked hard for a while and then is never opened again, and when the team sharded document chunks across four thousand partitions the dead projects landed in the same partitions as the live ones. Every query pulled an enormous partition into memory, evicted the last, and thrashed the cache. Jacob Lauritzen walks the whole migration honestly, from one search cluster for everybody, then one per region once clients on three continents each demanded their processing stay home, then into the general purpose database the team already ran, which is where it broke. What worked was making the project the unit of storage, one namespace each, so an idle one rests in object storage costing nothing. Simon Eskildsen then explains why that shape suits regulated customers so well. If the namespace is the atom, every namespace can carry its own encryption key and its own bucket, which is what banks and large firms actually mean when they demand physical isolation and the ability to revoke a key and make their data unreadable. The best detail is an experiment: rather than build encryption into the SSD cache, they disabled the cache outright to see how bad it would be, found the performance acceptable, and left it off. The back half is a crash course in storage engineering: why round trips dominate when object storage answers in 200 milliseconds, why clustering into a tree beats navigating a graph, and why full text search at web scale costs more than vector search. Speaker info: - https://x.com/Sirupsen - https://www.linkedin.com/in/sirupsen/ - https://sirupsen.com - https://sirupsen.com/napkin Timestamps: 0:00 - What Legora does, and two kinds of legal search 2:32 - One search cluster, then one per region 4:00 - What enterprises mean by physical isolation 4:48 - Moving search into the database they already ran 5:38 - How hot and cold projects thrashed the cache 6:40 - A namespace per project, and what it fixed 7:56 - Writing straight to object storage 9:58 - Why a namespace is the unit of encryption 12:07 - Legal research, and ten billion vectors 14:25 - Puffing data through the memory hierarchy 15:30 - Why a tree beats a graph on object storage 17:45 - How full text search actually works --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/V-isu4eTHgw-youtube-bfc00573674178488871 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/connect-ai-to-billions-of-legal-documents-simon-esk-e0bb3e6fec42 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/connect-ai-to-billions-of-legal-documents-simon-esk-e0bb3e6fec42.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T03:25:06.378Z' language: null children: [] videoId: V-isu4eTHgw durationSeconds: 1236 publishedAt: '2026-09-16T16:00:19.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 23 endSeconds: 158 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=23' heldReason: rights - ordinal: 1 text: null startSeconds: 23 endSeconds: 158 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=23' heldReason: rights - ordinal: 2 text: null startSeconds: 170 endSeconds: 244 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=170' heldReason: rights - ordinal: 3 text: null startSeconds: 256 endSeconds: 289 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=256' heldReason: rights - ordinal: 4 text: null startSeconds: 301 endSeconds: 345 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=301' heldReason: rights - ordinal: 5 text: null startSeconds: 356 endSeconds: 400 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=356' heldReason: rights - ordinal: 6 text: null startSeconds: 400 endSeconds: 411 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=400' heldReason: rights - ordinal: 7 text: null startSeconds: 484 endSeconds: 494 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=484' heldReason: rights - ordinal: 8 text: null startSeconds: 494 endSeconds: 607 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=494' heldReason: rights - ordinal: 9 text: null startSeconds: 619 endSeconds: 732 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=619' heldReason: rights - ordinal: 10 text: null startSeconds: 743 endSeconds: 874 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=743' heldReason: rights - ordinal: 11 text: null startSeconds: 743 endSeconds: 874 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=743' heldReason: rights - ordinal: 12 text: null startSeconds: 886 endSeconds: 930 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=886' heldReason: rights - ordinal: 13 text: null startSeconds: 941 endSeconds: 1074 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=941' heldReason: rights - ordinal: 14 text: null startSeconds: 1085 endSeconds: 1236 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=1085' heldReason: rights - ordinal: 15 text: null startSeconds: 1085 endSeconds: 1236 watch: 'https://liquidgold.cc/watch/V-isu4eTHgw?t=1085' heldReason: rights --- # Connect AI to Billions of Legal Documents — Simon Eskildsen, turbopuffer & Jacob Lauritzen, Legora Legora's search latency once went from a 100 millisecond P99 to twenty seconds, and the cause was a packing problem invisible in the schema. Legal work arrives as projects, a project gets worked hard for a while and then is never opened again, and when the team sharded document chunks across four thousand partitions the dead projects landed in the same partitions as the live ones. Every query pulled an enormous partition into memory, evicted the last, and thrashed the cache. Jacob Lauritzen walks the whole migration honestly, from one search cluster for everybody, then one per region once clients on three continents each demanded their processing stay home, then into the general purpose database the team already ran, which is where it broke. What worked was making the project the unit of storage, one namespace each, so an idle one rests in object storage costing nothing. Simon Eskildsen then explains why that shape suits regulated customers so well. If the namespace is the atom, every namespace can carry its own encryption key and its own bucket, which is what banks and large firms actually mean when they demand physical isolation and the ability to revoke a key and make their data unreadable. The best detail is an experiment: rather than build encryption into the SSD cache, they disabled the cache outright to see how bad it would be, found the performance acceptable, and left it off. The back half is a crash course in storage engineering: why round trips dominate when object storage answers in 200 milliseconds, why clustering into a tree beats navigating a graph, and why full text search at web scale costs more than vector search. Speaker info: - https://x.com/Sirupsen - https://www.linkedin.com/in/sirupsen/ - https://sirupsen.com - https://sirupsen.com/napkin Timestamps: 0:00 - What Legora does, and two kinds of legal search 2:32 - One search cluster, then one per region 4:00 - What enterprises mean by physical isolation 4:48 - Moving search into the database they already ran 5:38 - How hot and cold projects thrashed the cache 6:40 - A namespace per project, and what it fixed 7:56 - Writing straight to object storage 9:58 - Why a namespace is the unit of encryption 12:07 - Legal research, and ten billion vectors 14:25 - Puffing data through the memory hierarchy 15:30 - Why a tree beats a graph on object storage 17:45 - How full text search actually works --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/V-isu4eTHgw-youtube-bfc00573674178488871 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/connect-ai-to-billions-of-legal-documents-simon-esk-e0bb3e6fec42 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.