--- schema: 1 type: video title: >- Distill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash description: >- Search for gluten-free pasta and a ranker trained on engagement may show you regular spaghetti, because it sells well. Raghav Saboo, Staff ML Engineer and Tech Lead for Search & Personalization at DoorDash, argues that the real bottleneck in marketplace discovery is semantic understanding, not engagement. As DoorDash grows into grocery, retail, pets and gifting across billions of store-level items, it uses LLMs to understand what items mean and what shoppers actually intend. His core pattern: do the expensive LLM reasoning once, offline, and then distill it into small, fast models for serving. He covers four building blocks. The first is LLM-generated relevance labels, which improved retrieval NDCG by 2.3%. The second is semantic IDs, a learned taxonomy of the catalog that raised ranking MRR by 4–5% and powers query reformulation. The third is consumer memory at three timescales, stored as text, vectors and a graph. The fourth is steerable, LLM-generated personalized collections, which lifted order rate by nearly 1% in the pets category. Speaker info: Substack: https://buildshipai.substack.com/ Timestamps: 0:00 Intro 0:47 Discovery's real bottleneck is semantic understanding 1:42 Beyond restaurants: broad shopping missions 2:52 The four primitives 3:17 Primitive 1: LLM supervision 3:27 Why engagement ranks the wrong pasta 4:11 Human labels vs. behavioral signals 4:46 Building a golden labeled dataset 5:46 A fine-tuned LLM labeler for the whole catalog 6:11 Reason offline, serve cheaply 6:21 Two-stage contrastive retrieval 8:00 Adding a relevance tower to rankers 9:00 Distill, don't replace 9:30 Primitive 2: catalog semantics with semantic IDs 11:00 A learned taxonomy 11:50 Cross-category, cold start and tail coverage 12:55 Results: 4–5% MRR gains in ranking 13:20 Query reformulation 14:05 Primitive 3: consumer memory 15:15 Three timescales of memory 15:55 Memory blocks: text, vectors and graphs 17:04 Context graphs 17:55 Where memory shows up 18:20 Primitive 4: steerable content generation 19:05 Personalized collections on store pages 20:33 Results in the pets category 20:48 Three takeaways --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/ACPEpji5NV4-youtube-2d91cb29b1a65a6468a3 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/distill-the-llm-don-t-serve-it-search-personalizati-bdc08f8f58fd markdown: >- https://liquidgold.cc/channel/aidotengineer/video/distill-the-llm-don-t-serve-it-search-personalizati-bdc08f8f58fd.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:20:50.636Z' language: null children: [] videoId: ACPEpji5NV4 durationSeconds: 1331 publishedAt: '2026-09-25T13:30:37.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 199 endSeconds: 211 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=199' heldReason: rights - ordinal: 1 text: null startSeconds: 221 endSeconds: 257 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=221' heldReason: rights - ordinal: 2 text: null startSeconds: 257 endSeconds: 269 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=257' heldReason: rights - ordinal: 3 text: null startSeconds: 302 endSeconds: 348 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=302' heldReason: rights - ordinal: 4 text: null startSeconds: 348 endSeconds: 359 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=348' heldReason: rights - ordinal: 5 text: null startSeconds: 359 endSeconds: 382 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=359' heldReason: rights - ordinal: 6 text: null startSeconds: 393 endSeconds: 483 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=393' heldReason: rights - ordinal: 7 text: null startSeconds: 495 endSeconds: 551 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=495' heldReason: rights - ordinal: 8 text: null startSeconds: 584 endSeconds: 665 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=584' heldReason: rights - ordinal: 9 text: null startSeconds: 678 endSeconds: 713 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=678' heldReason: rights - ordinal: 10 text: null startSeconds: 723 endSeconds: 776 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=723' heldReason: rights - ordinal: 11 text: null startSeconds: 786 endSeconds: 809 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=786' heldReason: rights - ordinal: 12 text: null startSeconds: 820 endSeconds: 854 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=820' heldReason: rights - ordinal: 13 text: null startSeconds: 865 endSeconds: 926 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=865' heldReason: rights - ordinal: 14 text: null startSeconds: 937 endSeconds: 961 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=937' heldReason: rights - ordinal: 15 text: null startSeconds: 973 endSeconds: 1028 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=973' heldReason: rights - ordinal: 16 text: null startSeconds: 1038 endSeconds: 1085 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=1038' heldReason: rights - ordinal: 17 text: null startSeconds: 1117 endSeconds: 1151 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=1117' heldReason: rights - ordinal: 18 text: null startSeconds: 1163 endSeconds: 1244 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=1163' heldReason: rights - ordinal: 19 text: null startSeconds: 1244 endSeconds: 1256 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=1244' heldReason: rights - ordinal: 20 text: null startSeconds: 1267 endSeconds: 1331 watch: 'https://liquidgold.cc/watch/ACPEpji5NV4?t=1267' heldReason: rights --- # Distill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash Search for gluten-free pasta and a ranker trained on engagement may show you regular spaghetti, because it sells well. Raghav Saboo, Staff ML Engineer and Tech Lead for Search & Personalization at DoorDash, argues that the real bottleneck in marketplace discovery is semantic understanding, not engagement. As DoorDash grows into grocery, retail, pets and gifting across billions of store-level items, it uses LLMs to understand what items mean and what shoppers actually intend. His core pattern: do the expensive LLM reasoning once, offline, and then distill it into small, fast models for serving. He covers four building blocks. The first is LLM-generated relevance labels, which improved retrieval NDCG by 2.3%. The second is semantic IDs, a learned taxonomy of the catalog that raised ranking MRR by 4–5% and powers query reformulation. The third is consumer memory at three timescales, stored as text, vectors and a graph. The fourth is steerable, LLM-generated personalized collections, which lifted order rate by nearly 1% in the pets category. Speaker info: Substack: https://buildshipai.substack.com/ Timestamps: 0:00 Intro 0:47 Discovery's real bottleneck is semantic understanding 1:42 Beyond restaurants: broad shopping missions 2:52 The four primitives 3:17 Primitive 1: LLM supervision 3:27 Why engagement ranks the wrong pasta 4:11 Human labels vs. behavioral signals 4:46 Building a golden labeled dataset 5:46 A fine-tuned LLM labeler for the whole catalog 6:11 Reason offline, serve cheaply 6:21 Two-stage contrastive retrieval 8:00 Adding a relevance tower to rankers 9:00 Distill, don't replace 9:30 Primitive 2: catalog semantics with semantic IDs 11:00 A learned taxonomy 11:50 Cross-category, cold start and tail coverage 12:55 Results: 4–5% MRR gains in ranking 13:20 Query reformulation 14:05 Primitive 3: consumer memory 15:15 Three timescales of memory 15:55 Memory blocks: text, vectors and graphs 17:04 Context graphs 17:55 Where memory shows up 18:20 Primitive 4: steerable content generation 19:05 Personalized collections on store pages 20:33 Results in the pets category 20:48 Three takeaways --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/ACPEpji5NV4-youtube-2d91cb29b1a65a6468a3 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/distill-the-llm-don-t-serve-it-search-personalizati-bdc08f8f58fd · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks --- #### Chunk 0: Document excerpt (0:12 to 2:57) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 1: Document excerpt (2:57 to 4:51) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 2: Document excerpt (4:51 to 6:22) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 3: Document excerpt (6:22 to 8:03) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 4: Document excerpt (8:03 to 9:23) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 5: Document excerpt (9:23 to 11:53) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 6: Document excerpt (11:53 to 13:40) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 7: Document excerpt (13:40 to 15:37) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 8: Document excerpt (15:37 to 17:08) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 9: Document excerpt (17:08 to 18:26) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 10: Document excerpt (18:26 to 19:23) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 11: Document excerpt (19:23 to 22:11) > Held: transcript rights do not permit this chunk's text.