--- schema: 1 type: video title: 'Latency Is a Budget. Humanlike Is the Goal. — Jesse Hall, LiveKit' description: >- Your voice agent's dashboard says it's fast. Your callers say it feels robotic. They're both right. Jesse Hall, staff developer advocate at LiveKit, explains how to build voice agents that feel human. He shows why leaderboards can't pick your models (scores measure one model, but you ship a pipeline, and errors compound from speech-to-text to the LLM to text-to-speech) and why measured latency isn't what your callers hear. He covers buying time like a person does while tools run, plays a hotel-receptionist demo that handles interruptions and a caller changing his mind, explains LiveKit's audio-based turn detection (4.5% of users cut off vs 9.9% for the best alternative), and lays out the window that matters: a second and a half end to end still works, and around 600 milliseconds starts to feel human. He ends with LiveKit's open-source benchmark you can run against your own agent, a sneak peek of a voice-tuned Gemma model on LiveKit Inference, and a checklist. In this talk: • The leaderboard trap, and why pipelines fail differently than models • Measured vs perceived latency • Turn detection from audio, and buying time during tool calls • A latency budget, an open-source benchmark for your agent, and a checklist SPEAKER Jesse Hall, Staff Developer Advocate, LiveKit X: https://twitter.com/codeSTACKr LinkedIn: https://linkedin.com/in/codestackr LINKS LiveKit: https://livekit.io LiveKit Agents docs: https://docs.livekit.io/agents/ CHAPTERS 0:00 Intro 0:42 You've talked to one of these 1:22 Latency is a budget 2:02 Picking models 2:42 What LiveKit is 3:26 Realtime models vs pipelines 4:16 How pipeline errors compound 4:51 The leaderboard trap 6:06 Measured vs perceived latency 7:35 Buying time like a human 8:21 Demo: hotel receptionist 10:26 Audio-based turn detection 11:41 The 600 ms target 13:54 Benchmark your own agent 15:09 A faster voice LLM 15:53 The checklist Recorded at the AI Engineer World's Fair 2026 in San Francisco. Subscribe for more talks from the engineers building with AI. AI Engineer: https://ai.engineer YouTube: https://www.youtube.com/@aiDotEngineer X: https://x.com/aiDotEngineer LinkedIn: https://www.linkedin.com/company/aidotengineer/ #VoiceAI #LiveKit #AIEngineer canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/latency-is-a-budget-humanlike-is-the-goal-jesse-hal-0d8b3f73d51e markdown: >- https://liquidgold.cc/channel/aidotengineer/video/latency-is-a-budget-humanlike-is-the-goal-jesse-hal-0d8b3f73d51e.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:19:20.920Z' language: null children: [] videoId: j4AmIK6OvHc durationSeconds: 1021 publishedAt: '2026-10-11T01:30:13.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 90 endSeconds: 100 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=90' heldReason: rights - ordinal: 1 text: null startSeconds: 224 endSeconds: 259 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=224' heldReason: rights - ordinal: 2 text: null startSeconds: 259 endSeconds: 270 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=259' heldReason: rights - ordinal: 3 text: null startSeconds: 371 endSeconds: 383 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=371' heldReason: rights - ordinal: 4 text: null startSeconds: 461 endSeconds: 471 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=461' heldReason: rights - ordinal: 5 text: null startSeconds: 634 endSeconds: 647 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=634' heldReason: rights - ordinal: 6 text: null startSeconds: 702 endSeconds: 713 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=702' heldReason: rights - ordinal: 7 text: null startSeconds: 838 endSeconds: 850 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=838' heldReason: rights - ordinal: 8 text: null startSeconds: 962 endSeconds: 1021 watch: 'https://liquidgold.cc/watch/j4AmIK6OvHc?t=962' heldReason: rights --- # Latency Is a Budget. Humanlike Is the Goal. — Jesse Hall, LiveKit Your voice agent's dashboard says it's fast. Your callers say it feels robotic. They're both right. Jesse Hall, staff developer advocate at LiveKit, explains how to build voice agents that feel human. He shows why leaderboards can't pick your models (scores measure one model, but you ship a pipeline, and errors compound from speech-to-text to the LLM to text-to-speech) and why measured latency isn't what your callers hear. He covers buying time like a person does while tools run, plays a hotel-receptionist demo that handles interruptions and a caller changing his mind, explains LiveKit's audio-based turn detection (4.5% of users cut off vs 9.9% for the best alternative), and lays out the window that matters: a second and a half end to end still works, and around 600 milliseconds starts to feel human. He ends with LiveKit's open-source benchmark you can run against your own agent, a sneak peek of a voice-tuned Gemma model on LiveKit Inference, and a checklist. In this talk: • The leaderboard trap, and why pipelines fail differently than models • Measured vs perceived latency • Turn detection from audio, and buying time during tool calls • A latency budget, an open-source benchmark for your agent, and a checklist SPEAKER Jesse Hall, Staff Developer Advocate, LiveKit X: https://twitter.com/codeSTACKr LinkedIn: https://linkedin.com/in/codestackr LINKS LiveKit: https://livekit.io LiveKit Agents docs: https://docs.livekit.io/agents/ CHAPTERS 0:00 Intro 0:42 You've talked to one of these 1:22 Latency is a budget 2:02 Picking models 2:42 What LiveKit is 3:26 Realtime models vs pipelines 4:16 How pipeline errors compound 4:51 The leaderboard trap 6:06 Measured vs perceived latency 7:35 Buying time like a human 8:21 Demo: hotel receptionist 10:26 Audio-based turn detection 11:41 The 600 ms target 13:54 Benchmark your own agent 15:09 A faster voice LLM 15:53 The checklist Recorded at the AI Engineer World's Fair 2026 in San Francisco. Subscribe for more talks from the engineers building with AI. AI Engineer: https://ai.engineer YouTube: https://www.youtube.com/@aiDotEngineer X: https://x.com/aiDotEngineer LinkedIn: https://www.linkedin.com/company/aidotengineer/ #VoiceAI #LiveKit #AIEngineer Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/latency-is-a-budget-humanlike-is-the-goal-jesse-hal-0d8b3f73d51e · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.