--- schema: 1 type: video title: >- Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean description: >- Two terminals run the same prompt, build me a spinning wheel app. On the left every request goes to a single premium model. On the right they go through a router that picks a model per task. Both finish at about the same time with comparable output, and by then the router's session has cost 8 cents against 25. The gap widens with every prompt after that. Archana Kamath and Tyler Gillam use it to argue that picking a model by climbing a leaderboard is the wrong instinct, because there is no single best model, only the right one for a given request. What makes a model right is a mix no public leaderboard encodes: the task itself, the system prompt and tools around it, the cost you are willing to spend, the latency the use case needs, and what the end user actually wants. Their router takes those as preferences you declare, in natural language or as decision tree rules, then honors them per request. It runs on a purpose built mixture of experts model that decides in under 200 milliseconds, costs nothing extra, and is open sourced along with the proxy in front of it. Gillam then shows the part that separates it from a vibe check, an evaluation scoring the router at 90% correctness against 95% for the single premium model while using far fewer tokens and returning faster. Routing is the foundation layer, with evaluation, caching and personalization built on top. Speaker info: - https://www.linkedin.com/in/tdgillam Timestamps: 0:00 - Why the one model habit is breaking 2:42 - There is no single best model 4:21 - A router you can customize and evaluate 6:57 - Configuring tasks, model pools and failover 7:48 - Side by side in the playground 9:29 - Proving it with an evaluation 10:18 - Two coding agents, and the session cost gap 13:49 - Under 200ms, open sourced, no code changes 14:43 - Evaluation, caching, personalization canonical: 'https://liquidgold.cc/watch/FvxY8oPoI8o' markdown: 'https://liquidgold.cc/watch/FvxY8oPoI8o.md' llms: 'https://liquidgold.cc/watch/FvxY8oPoI8o/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T04:50:53.260Z' language: null children: [] videoId: FvxY8oPoI8o durationSeconds: 954 publishedAt: '2026-08-22T15:30:18.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 23 endSeconds: 162 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=23' heldReason: rights - ordinal: 1 text: null startSeconds: 23 endSeconds: 162 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=23' heldReason: rights - ordinal: 2 text: null startSeconds: 23 endSeconds: 162 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=23' heldReason: rights - ordinal: 3 text: null startSeconds: 174 endSeconds: 264 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=174' heldReason: rights - ordinal: 4 text: null startSeconds: 174 endSeconds: 264 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=174' heldReason: rights - ordinal: 5 text: null startSeconds: 275 endSeconds: 419 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=275' heldReason: rights - ordinal: 6 text: null startSeconds: 275 endSeconds: 419 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=275' heldReason: rights - ordinal: 7 text: null startSeconds: 275 endSeconds: 419 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=275' heldReason: rights - ordinal: 8 text: null startSeconds: 275 endSeconds: 419 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=275' heldReason: rights - ordinal: 9 text: null startSeconds: 429 endSeconds: 474 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=429' heldReason: rights - ordinal: 10 text: null startSeconds: 429 endSeconds: 474 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=429' heldReason: rights - ordinal: 11 text: null startSeconds: 474 endSeconds: 485 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=474' heldReason: rights - ordinal: 12 text: null startSeconds: 574 endSeconds: 585 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=574' heldReason: rights - ordinal: 13 text: null startSeconds: 630 endSeconds: 836 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=630' heldReason: rights - ordinal: 14 text: null startSeconds: 630 endSeconds: 836 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=630' heldReason: rights - ordinal: 15 text: null startSeconds: 630 endSeconds: 836 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=630' heldReason: rights - ordinal: 16 text: null startSeconds: 848 endSeconds: 889 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=848' heldReason: rights - ordinal: 17 text: null startSeconds: 889 endSeconds: 900 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=889' heldReason: rights - ordinal: 18 text: null startSeconds: 900 endSeconds: 954 watch: 'https://liquidgold.cc/watch/FvxY8oPoI8o?t=900' heldReason: rights --- # Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean Two terminals run the same prompt, build me a spinning wheel app. On the left every request goes to a single premium model. On the right they go through a router that picks a model per task. Both finish at about the same time with comparable output, and by then the router's session has cost 8 cents against 25. The gap widens with every prompt after that. Archana Kamath and Tyler Gillam use it to argue that picking a model by climbing a leaderboard is the wrong instinct, because there is no single best model, only the right one for a given request. What makes a model right is a mix no public leaderboard encodes: the task itself, the system prompt and tools around it, the cost you are willing to spend, the latency the use case needs, and what the end user actually wants. Their router takes those as preferences you declare, in natural language or as decision tree rules, then honors them per request. It runs on a purpose built mixture of experts model that decides in under 200 milliseconds, costs nothing extra, and is open sourced along with the proxy in front of it. Gillam then shows the part that separates it from a vibe check, an evaluation scoring the router at 90% correctness against 95% for the single premium model while using far fewer tokens and returning faster. Routing is the foundation layer, with evaluation, caching and personalization built on top. Speaker info: - https://www.linkedin.com/in/tdgillam Timestamps: 0:00 - Why the one model habit is breaking 2:42 - There is no single best model 4:21 - A router you can customize and evaluate 6:57 - Configuring tasks, model pools and failover 7:48 - Side by side in the playground 9:29 - Proving it with an evaluation 10:18 - Two coding agents, and the session cost gap 13:49 - Under 200ms, open sourced, no code changes 14:43 - Evaluation, caching, personalization Canonical: https://liquidgold.cc/watch/FvxY8oPoI8o · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.