--- schema: 1 type: video title: 'Tolan: Voice-First AI Companion — Paula Dozsa, Tolan' description: >- Tolan's response latency once drifted from two seconds to about two and a half. That half second tanked essentially every metric in the product, and users wrote in to say their companion had become slow. Paula Dozsa is an engineer on the iOS app for Tolan, a voice first companion shaped as a small alien, and the team has logged more than four million hours of spoken conversation. Her argument is that voice does not just add a modality to an LLM app, it invalidates the assumptions underneath one. Text chat has slow turns and stable context. People read, they wait, they stay on topic. Voice has fast turns and volatile context. People talk while cooking or walking or falling asleep, they abandon a subject mid sentence and come back to it, they interrupt. Her team stopped trying to reduce interruptions and started reducing the wrong ones, building turn detection that reads speech patterns and paying sixty milliseconds of latency to halve the worst early cutoffs. The rest is unusually specific engineering. Every pipeline stage is measured separately, because knowing it feels slow tells you nothing. Turns are routed per turn by a cheap classifier reading the emotional stakes, so a user's first ever conversation gets the strongest model and idle chatter does not, and a third of turns can ride a small model with no measurable retention cost. Memory is a retrieval system rather than a transcript, compressed nightly to merge duplicates and resolve contradictions. And context gets reassembled from parts every single turn, because reusing it to keep a cache warm means being confidently wrong the moment someone changes the subject. Speaker info: - https://x.com/paularambles - https://www.linkedin.com/in/paulacodes/ Timestamps: 0:00 - The companion humans keep imagining 2:45 - Why voice breaks how we build LLM apps 3:38 - The half second that broke everything 5:20 - Measuring every stage of the pipeline 6:12 - Routing by stakes, not by cost 7:52 - Memory as retrieval, compressed nightly 8:43 - Rebuilding context instead of reusing it 9:31 - Why the character is an alien 10:23 - Using agents to build the app 12:05 - A new character in an afternoon --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/xLUQOqjudtA-tolan-voice-first-ai-companion Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/tolan-voice-first-ai-companion-paula-dozsa-tolan-789eda9a6d60 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/tolan-voice-first-ai-companion-paula-dozsa-tolan-789eda9a6d60.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T03:36:57.519Z' language: null children: [] videoId: xLUQOqjudtA durationSeconds: 908 publishedAt: '2026-09-15T14:30:14.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 23 endSeconds: 169 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=23' heldReason: rights - ordinal: 1 text: null startSeconds: 179 endSeconds: 224 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=179' heldReason: rights - ordinal: 2 text: null startSeconds: 179 endSeconds: 224 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=179' heldReason: rights - ordinal: 3 text: null startSeconds: 224 endSeconds: 236 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=224' heldReason: rights - ordinal: 4 text: null startSeconds: 236 endSeconds: 326 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=236' heldReason: rights - ordinal: 5 text: null startSeconds: 236 endSeconds: 326 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=236' heldReason: rights - ordinal: 6 text: null startSeconds: 236 endSeconds: 326 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=236' heldReason: rights - ordinal: 7 text: null startSeconds: 326 endSeconds: 339 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=326' heldReason: rights - ordinal: 8 text: null startSeconds: 339 endSeconds: 372 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=339' heldReason: rights - ordinal: 9 text: null startSeconds: 339 endSeconds: 372 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=339' heldReason: rights - ordinal: 10 text: null startSeconds: 372 endSeconds: 384 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=372' heldReason: rights - ordinal: 11 text: null startSeconds: 384 endSeconds: 472 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=384' heldReason: rights - ordinal: 12 text: null startSeconds: 384 endSeconds: 472 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=384' heldReason: rights - ordinal: 13 text: null startSeconds: 384 endSeconds: 472 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=384' heldReason: rights - ordinal: 14 text: null startSeconds: 472 endSeconds: 483 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=472' heldReason: rights - ordinal: 15 text: null startSeconds: 472 endSeconds: 483 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=472' heldReason: rights - ordinal: 16 text: null startSeconds: 483 endSeconds: 526 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=483' heldReason: rights - ordinal: 17 text: null startSeconds: 526 endSeconds: 537 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=526' heldReason: rights - ordinal: 18 text: null startSeconds: 578 endSeconds: 590 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=578' heldReason: rights - ordinal: 19 text: null startSeconds: 590 endSeconds: 624 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=590' heldReason: rights - ordinal: 20 text: null startSeconds: 635 endSeconds: 726 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=635' heldReason: rights - ordinal: 21 text: null startSeconds: 635 endSeconds: 726 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=635' heldReason: rights - ordinal: 22 text: null startSeconds: 635 endSeconds: 726 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=635' heldReason: rights - ordinal: 23 text: null startSeconds: 726 endSeconds: 737 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=726' heldReason: rights - ordinal: 24 text: null startSeconds: 737 endSeconds: 908 watch: 'https://liquidgold.cc/watch/xLUQOqjudtA?t=737' heldReason: rights --- # Tolan: Voice-First AI Companion — Paula Dozsa, Tolan Tolan's response latency once drifted from two seconds to about two and a half. That half second tanked essentially every metric in the product, and users wrote in to say their companion had become slow. Paula Dozsa is an engineer on the iOS app for Tolan, a voice first companion shaped as a small alien, and the team has logged more than four million hours of spoken conversation. Her argument is that voice does not just add a modality to an LLM app, it invalidates the assumptions underneath one. Text chat has slow turns and stable context. People read, they wait, they stay on topic. Voice has fast turns and volatile context. People talk while cooking or walking or falling asleep, they abandon a subject mid sentence and come back to it, they interrupt. Her team stopped trying to reduce interruptions and started reducing the wrong ones, building turn detection that reads speech patterns and paying sixty milliseconds of latency to halve the worst early cutoffs. The rest is unusually specific engineering. Every pipeline stage is measured separately, because knowing it feels slow tells you nothing. Turns are routed per turn by a cheap classifier reading the emotional stakes, so a user's first ever conversation gets the strongest model and idle chatter does not, and a third of turns can ride a small model with no measurable retention cost. Memory is a retrieval system rather than a transcript, compressed nightly to merge duplicates and resolve contradictions. And context gets reassembled from parts every single turn, because reusing it to keep a cache warm means being confidently wrong the moment someone changes the subject. Speaker info: - https://x.com/paularambles - https://www.linkedin.com/in/paulacodes/ Timestamps: 0:00 - The companion humans keep imagining 2:45 - Why voice breaks how we build LLM apps 3:38 - The half second that broke everything 5:20 - Measuring every stage of the pipeline 6:12 - Routing by stakes, not by cost 7:52 - Memory as retrieval, compressed nightly 8:43 - Rebuilding context instead of reusing it 9:31 - Why the character is an alien 10:23 - Using agents to build the app 12:05 - A new character in an afternoon --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/xLUQOqjudtA-tolan-voice-first-ai-companion Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/tolan-voice-first-ai-companion-paula-dozsa-tolan-789eda9a6d60 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.