--- schema: 1 type: video title: 'The State of Model Routing — NVIDIA, Cognition, OpenRouter' description: >- Run terminal bench on Opus and on Haiku and Opus scores about three times better at a tenth of the cost, even though Haiku is far cheaper per token. Alex Atallah's point is that a small model pushed outside its training distribution thrashes, calling tools in loops until it costs more than the expensive model ever would. That inverts the obvious version of model routing, where you send each task to whichever model benchmarks best on it. Walden Yan calls that approach fragile for exactly the reason agents make it worse: a session starts as a question about a codebase, becomes a feature request, then becomes live debugging, and the model you picked at the start is stranded. Cognition's answer keeps a frontier model planning and delegates the implementation, which cut the cost of Fable level intelligence by 40% while going deeper, because a cheaper model can afford to spin off three sub agents to explore a codebase. They also avoid sub agents in favor of one sidekick with a continuous running context, so the KV cache stays warm and cached tokens cost roughly ten times less. Compaction, Yan argues, is worth doing for intelligence rather than cost, since compacting forces a cache miss and model quality falls off a cliff well before the advertised million token window. The most telling story is OpenRouter's: its auto router sat almost unused for two years until openclaw began sending heartbeats every ten minutes, creating one popular app with two completely different intelligence needs. Speaker info: Nader Khalil, moderator (NVIDIA): - https://x.com/naderlikeladder - https://nader.coffee Walden Yan (Cognition): - https://x.com/walden_yan - https://www.linkedin.com/in/waldenyan Alex Atallah (OpenRouter): - https://x.com/alexatallah - https://openrouter.ai Tanay Varshney (NVIDIA): - https://www.linkedin.com/in/tanayvarshney Carter Abdallah (NVIDIA): - https://x.com/Baxate - https://www.linkedin.com/in/carter-abdallah Timestamps: 0:00 - Welcome and the multimodel premise 1:16 - Panel introductions 3:24 - How Devin Fusion beats the frontier models 4:25 - Let the frontier model plan and delegate the work 6:31 - Jagged capabilities: no one model wins everything 9:42 - Why naive task based routing is fragile 11:48 - Sharing context without paying for it twice 13:56 - Should the orchestrator be the big or the small model 16:01 - In distribution versus out of distribution 19:12 - Training models to collaborate 20:12 - Flex Run and flexible model sizes 22:24 - Lossy context and the systems you fall back to 26:41 - How a heartbeat created the auto router 29:43 - Routing between local and cloud 31:51 - Compaction versus routing 32:55 - How a small model signals it is out of its depth 35:00 - Cache duration and self hosting economics 40:12 - Are prompts portable across models 43:20 - Is the router a product or plumbing --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/QHBjufYK8TA-the-state-of-model-routing-nvidia-cognition-openrouter Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/the-state-of-model-routing-nvidia-cognition-openrou-fdfa702d5609 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/the-state-of-model-routing-nvidia-cognition-openrou-fdfa702d5609.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T05:47:13.346Z' language: null children: [] videoId: QHBjufYK8TA durationSeconds: 2897 publishedAt: '2026-08-06T17:07:24.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 12 endSeconds: 77 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=12' heldReason: rights - ordinal: 1 text: null startSeconds: 87 endSeconds: 211 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=87' heldReason: rights - ordinal: 2 text: null startSeconds: 223 endSeconds: 270 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=223' heldReason: rights - ordinal: 3 text: null startSeconds: 280 endSeconds: 399 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=280' heldReason: rights - ordinal: 4 text: null startSeconds: 399 endSeconds: 411 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=399' heldReason: rights - ordinal: 5 text: null startSeconds: 411 endSeconds: 586 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=411' heldReason: rights - ordinal: 6 text: null startSeconds: 586 endSeconds: 597 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=586' heldReason: rights - ordinal: 7 text: null startSeconds: 597 endSeconds: 717 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=597' heldReason: rights - ordinal: 8 text: null startSeconds: 717 endSeconds: 727 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=717' heldReason: rights - ordinal: 9 text: null startSeconds: 845 endSeconds: 856 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=845' heldReason: rights - ordinal: 10 text: null startSeconds: 856 endSeconds: 970 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=856' heldReason: rights - ordinal: 11 text: null startSeconds: 980 endSeconds: 1155 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=980' heldReason: rights - ordinal: 12 text: null startSeconds: 1155 endSeconds: 1167 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=1155' heldReason: rights - ordinal: 13 text: null startSeconds: 1222 endSeconds: 1234 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=1222' heldReason: rights - ordinal: 14 text: null startSeconds: 1359 endSeconds: 1608 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=1359' heldReason: rights - ordinal: 15 text: null startSeconds: 1619 endSeconds: 1788 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=1619' heldReason: rights - ordinal: 16 text: null startSeconds: 1799 endSeconds: 1912 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=1799' heldReason: rights - ordinal: 17 text: null startSeconds: 1912 endSeconds: 1923 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=1912' heldReason: rights - ordinal: 18 text: null startSeconds: 1923 endSeconds: 1978 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=1923' heldReason: rights - ordinal: 19 text: null startSeconds: 1989 endSeconds: 2101 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=1989' heldReason: rights - ordinal: 20 text: null startSeconds: 2112 endSeconds: 2195 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=2112' heldReason: rights - ordinal: 21 text: null startSeconds: 2326 endSeconds: 2336 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=2326' heldReason: rights - ordinal: 22 text: null startSeconds: 2414 endSeconds: 2424 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=2414' heldReason: rights - ordinal: 23 text: null startSeconds: 2424 endSeconds: 2600 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=2424' heldReason: rights - ordinal: 24 text: null startSeconds: 2600 endSeconds: 2612 watch: 'https://liquidgold.cc/watch/QHBjufYK8TA?t=2600' heldReason: rights --- # The State of Model Routing — NVIDIA, Cognition, OpenRouter Run terminal bench on Opus and on Haiku and Opus scores about three times better at a tenth of the cost, even though Haiku is far cheaper per token. Alex Atallah's point is that a small model pushed outside its training distribution thrashes, calling tools in loops until it costs more than the expensive model ever would. That inverts the obvious version of model routing, where you send each task to whichever model benchmarks best on it. Walden Yan calls that approach fragile for exactly the reason agents make it worse: a session starts as a question about a codebase, becomes a feature request, then becomes live debugging, and the model you picked at the start is stranded. Cognition's answer keeps a frontier model planning and delegates the implementation, which cut the cost of Fable level intelligence by 40% while going deeper, because a cheaper model can afford to spin off three sub agents to explore a codebase. They also avoid sub agents in favor of one sidekick with a continuous running context, so the KV cache stays warm and cached tokens cost roughly ten times less. Compaction, Yan argues, is worth doing for intelligence rather than cost, since compacting forces a cache miss and model quality falls off a cliff well before the advertised million token window. The most telling story is OpenRouter's: its auto router sat almost unused for two years until openclaw began sending heartbeats every ten minutes, creating one popular app with two completely different intelligence needs. Speaker info: Nader Khalil, moderator (NVIDIA): - https://x.com/naderlikeladder - https://nader.coffee Walden Yan (Cognition): - https://x.com/walden_yan - https://www.linkedin.com/in/waldenyan Alex Atallah (OpenRouter): - https://x.com/alexatallah - https://openrouter.ai Tanay Varshney (NVIDIA): - https://www.linkedin.com/in/tanayvarshney Carter Abdallah (NVIDIA): - https://x.com/Baxate - https://www.linkedin.com/in/carter-abdallah Timestamps: 0:00 - Welcome and the multimodel premise 1:16 - Panel introductions 3:24 - How Devin Fusion beats the frontier models 4:25 - Let the frontier model plan and delegate the work 6:31 - Jagged capabilities: no one model wins everything 9:42 - Why naive task based routing is fragile 11:48 - Sharing context without paying for it twice 13:56 - Should the orchestrator be the big or the small model 16:01 - In distribution versus out of distribution 19:12 - Training models to collaborate 20:12 - Flex Run and flexible model sizes 22:24 - Lossy context and the systems you fall back to 26:41 - How a heartbeat created the auto router 29:43 - Routing between local and cloud 31:51 - Compaction versus routing 32:55 - How a small model signals it is out of its depth 35:00 - Cache duration and self hosting economics 40:12 - Are prompts portable across models 43:20 - Is the router a product or plumbing --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/QHBjufYK8TA-the-state-of-model-routing-nvidia-cognition-openrouter Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/the-state-of-model-routing-nvidia-cognition-openrou-fdfa702d5609 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.