--- schema: 1 type: video title: 'Same Model, Different Speed: Why Your Inference Provider Matters — FriendliAI' description: >- Open-weight models are good enough now. The question is who can run them fast and reliably. Yunmo Koo, founding engineer at FriendliAI, argues that open-weight models like GLM 5.2 and MiniMax M3 now compete with the best proprietary models. He shows GLM 5.2 and Claude Opus 4.8 building similar tower defense games from one prompt, with GLM 5.2 more than 5.6x cheaper. Then he runs the same model and the same coding agent on two providers: FriendliAI builds a Candy Crush-style game in about two minutes, the other takes more than three. He explains where the gap comes from: prefix caching with cache-aware routing and a GPU, CPU and NVMe cache hierarchy for long, repetitive agent inputs, plus hybrid speculative decoding that adapts to live traffic. He also covers why reliability matters for agents (one failed call can sink a whole run) and shares Kilo Code's results of up to 7x faster responses with fewer errors. In this talk: • Why open-weight models are now good enough for many workloads • Why the same model feels different on different providers • Prefix caching, cache-aware routing and a tiered KV cache • Hybrid speculative decoding and why reliability is part of speed SPEAKER Yunmo Koo, Founding Engineer, FriendliAI LinkedIn: https://www.linkedin.com/in/yunmokoo/ GitHub: https://github.com/kooyunmo LINKS FriendliAI: https://friendli.ai Kilo Code case study: https://friendli.ai/customers/kilo CHAPTERS 0:00 Intro 0:40 Open vs closed models 1:09 Open weights crossed the threshold 2:24 GLM 5.2 vs Opus: a tower defense game 2:49 5.6x cheaper 3:09 Provider quality 3:53 Same model, different provider 4:33 It's the whole stack 5:28 Long, repetitive agent inputs 6:27 Cache-aware routing 6:47 A cache hierarchy 7:37 Speculative decoding 9:06 Hybrid, self-improving speculation 9:41 Reliability is part of speed 11:06 Test your own workload 11:45 Kilo Code: up to 7x faster Recorded at the AI Engineer World's Fair 2026 in San Francisco. Subscribe for more talks from the engineers building with AI. AI Engineer: https://ai.engineer YouTube: https://www.youtube.com/@aiDotEngineer X: https://x.com/aiDotEngineer LinkedIn: https://www.linkedin.com/company/aidotengineer/ #LLMInference #OpenSourceAI #AIEngineer canonical: 'https://liquidgold.cc/watch/_naLJOqVOEI' markdown: 'https://liquidgold.cc/watch/_naLJOqVOEI.md' llms: 'https://liquidgold.cc/watch/_naLJOqVOEI/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:19:11.063Z' language: null children: [] videoId: _naLJOqVOEI durationSeconds: 784 publishedAt: '2026-10-11T19:30:05.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 72 endSeconds: 84 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=72' heldReason: rights - ordinal: 1 text: null startSeconds: 84 endSeconds: 151 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=84' heldReason: rights - ordinal: 2 text: null startSeconds: 162 endSeconds: 175 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=162' heldReason: rights - ordinal: 3 text: null startSeconds: 197 endSeconds: 209 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=197' heldReason: rights - ordinal: 4 text: null startSeconds: 209 endSeconds: 241 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=209' heldReason: rights - ordinal: 5 text: null startSeconds: 241 endSeconds: 252 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=241' heldReason: rights - ordinal: 6 text: null startSeconds: 274 endSeconds: 284 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=274' heldReason: rights - ordinal: 7 text: null startSeconds: 341 endSeconds: 392 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=341' heldReason: rights - ordinal: 8 text: null startSeconds: 392 endSeconds: 403 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=392' heldReason: rights - ordinal: 9 text: null startSeconds: 424 endSeconds: 460 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=424' heldReason: rights - ordinal: 10 text: null startSeconds: 471 endSeconds: 554 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=471' heldReason: rights - ordinal: 11 text: null startSeconds: 554 endSeconds: 564 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=554' heldReason: rights - ordinal: 12 text: null startSeconds: 588 endSeconds: 601 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=588' heldReason: rights - ordinal: 13 text: null startSeconds: 668 endSeconds: 679 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=668' heldReason: rights - ordinal: 14 text: null startSeconds: 714 endSeconds: 725 watch: 'https://liquidgold.cc/watch/_naLJOqVOEI?t=714' heldReason: rights --- # Same Model, Different Speed: Why Your Inference Provider Matters — FriendliAI Open-weight models are good enough now. The question is who can run them fast and reliably. Yunmo Koo, founding engineer at FriendliAI, argues that open-weight models like GLM 5.2 and MiniMax M3 now compete with the best proprietary models. He shows GLM 5.2 and Claude Opus 4.8 building similar tower defense games from one prompt, with GLM 5.2 more than 5.6x cheaper. Then he runs the same model and the same coding agent on two providers: FriendliAI builds a Candy Crush-style game in about two minutes, the other takes more than three. He explains where the gap comes from: prefix caching with cache-aware routing and a GPU, CPU and NVMe cache hierarchy for long, repetitive agent inputs, plus hybrid speculative decoding that adapts to live traffic. He also covers why reliability matters for agents (one failed call can sink a whole run) and shares Kilo Code's results of up to 7x faster responses with fewer errors. In this talk: • Why open-weight models are now good enough for many workloads • Why the same model feels different on different providers • Prefix caching, cache-aware routing and a tiered KV cache • Hybrid speculative decoding and why reliability is part of speed SPEAKER Yunmo Koo, Founding Engineer, FriendliAI LinkedIn: https://www.linkedin.com/in/yunmokoo/ GitHub: https://github.com/kooyunmo LINKS FriendliAI: https://friendli.ai Kilo Code case study: https://friendli.ai/customers/kilo CHAPTERS 0:00 Intro 0:40 Open vs closed models 1:09 Open weights crossed the threshold 2:24 GLM 5.2 vs Opus: a tower defense game 2:49 5.6x cheaper 3:09 Provider quality 3:53 Same model, different provider 4:33 It's the whole stack 5:28 Long, repetitive agent inputs 6:27 Cache-aware routing 6:47 A cache hierarchy 7:37 Speculative decoding 9:06 Hybrid, self-improving speculation 9:41 Reliability is part of speed 11:06 Test your own workload 11:45 Kilo Code: up to 7x faster Recorded at the AI Engineer World's Fair 2026 in San Francisco. Subscribe for more talks from the engineers building with AI. AI Engineer: https://ai.engineer YouTube: https://www.youtube.com/@aiDotEngineer X: https://x.com/aiDotEngineer LinkedIn: https://www.linkedin.com/company/aidotengineer/ #LLMInference #OpenSourceAI #AIEngineer Canonical: https://liquidgold.cc/watch/_naLJOqVOEI · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks --- #### Chunk 0: Document excerpt (0:13 to 1:12) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 1: Document excerpt (1:12 to 3:17) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 2: Document excerpt (3:17 to 4:34) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 3: Document excerpt (4:34 to 5:28) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 4: Document excerpt (5:28 to 6:32) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 5: Document excerpt (6:32 to 6:54) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 6: Document excerpt (6:54 to 7:40) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 7: Document excerpt (7:40 to 9:14) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 8: Document excerpt (9:14 to 9:48) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 9: Document excerpt (9:48 to 11:08) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 10: Document excerpt (11:08 to 11:54) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 11: Document excerpt (11:54 to 13:04) > Held: transcript rights do not permit this chunk's text.