--- schema: 1 type: video title: >- Designing Voice Agents for Real Conversations - Chintan Agrawal & Daniel Wirjo, AWS description: >- Chat agents get seconds to respond. Voice agents get 200 milliseconds, and if they get it wrong, the user doesn't retry, they hang up. The gap between "impressive voice demo" and "agent people actually want to talk to" is entirely in the engineering: latency budgets, barge-in handling, turn-taking, and the silence detection problem nobody warns you about. In this workshop you'll build a production voice agent using Pipecat (open-source real-time AI framework) that handles the three things most voice implementations get wrong. Interruption: user starts talking mid-response. Do you stop, fade, or finish? The answer depends on context, and we'll build the decision logic. Silence: is the user done talking or just thinking? Get this wrong and you either interrupt them or leave dead air. Latency: you have 200ms total for STT to LLM to TTS. We'll build the streaming pipeline that hits that budget consistently. The thing nobody tells you about voice agents: the hardest problem isn't the AI, it's the audio engineering. WebSocket orchestration, audio chunking, backpressure when the LLM is slower than real-time, echo cancellation when the agent's own output feeds back into the mic. We'll solve all of it. Stack: Pipecat + Daily (WebRTC) + any STT/TTS (Deepgram, AssemblyAI, Whisper) + any LLM. Bring headphones. You'll be talking to your agent by the end. Speakers: - Chintan Agrawal (Amazon Web Service): Chintan Agrawal and Daniel Wirjo are Solutions Architects at AWS, focused on AI and SaaS startups. They are passionate about helping customers unlock value from their data and accelerate innovation on AWS. By partnering closely with founders and engineering leaders, they enable organizations to build scalable solutions, drive business growth, and realize the full potential of cloud and AI technologies. LinkedIn: https://www.linkedin.com/in/chintan-agrawal-87a866135/ GitHub: https://github.com/wirjo/pipecat-turn-detection-demo canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/designing-voice-agents-for-real-conversations-chint-20b3986c2533 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/designing-voice-agents-for-real-conversations-chint-20b3986c2533.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T07:18:50.625Z' language: null children: [] videoId: hMlLw1LeIK8 durationSeconds: 1977 publishedAt: '2026-07-20T06:24:55.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 113 endSeconds: 192 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=113' heldReason: rights - ordinal: 1 text: null startSeconds: 203 endSeconds: 291 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=203' heldReason: rights - ordinal: 2 text: null startSeconds: 291 endSeconds: 302 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=291' heldReason: rights - ordinal: 3 text: null startSeconds: 302 endSeconds: 390 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=302' heldReason: rights - ordinal: 4 text: null startSeconds: 390 endSeconds: 400 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=390' heldReason: rights - ordinal: 5 text: null startSeconds: 400 endSeconds: 489 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=400' heldReason: rights - ordinal: 6 text: null startSeconds: 500 endSeconds: 609 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=500' heldReason: rights - ordinal: 7 text: null startSeconds: 500 endSeconds: 609 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=500' heldReason: rights - ordinal: 8 text: null startSeconds: 620 endSeconds: 754 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=620' heldReason: rights - ordinal: 9 text: null startSeconds: 620 endSeconds: 754 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=620' heldReason: rights - ordinal: 10 text: null startSeconds: 620 endSeconds: 754 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=620' heldReason: rights - ordinal: 11 text: null startSeconds: 765 endSeconds: 886 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=765' heldReason: rights - ordinal: 12 text: null startSeconds: 897 endSeconds: 1023 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=897' heldReason: rights - ordinal: 13 text: null startSeconds: 897 endSeconds: 1023 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=897' heldReason: rights - ordinal: 14 text: null startSeconds: 1033 endSeconds: 1141 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=1033' heldReason: rights - ordinal: 15 text: null startSeconds: 1141 endSeconds: 1153 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=1141' heldReason: rights - ordinal: 16 text: null startSeconds: 1153 endSeconds: 1288 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=1153' heldReason: rights - ordinal: 17 text: null startSeconds: 1153 endSeconds: 1288 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=1153' heldReason: rights - ordinal: 18 text: null startSeconds: 1422 endSeconds: 1541 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=1422' heldReason: rights - ordinal: 19 text: null startSeconds: 1554 endSeconds: 1676 watch: 'https://liquidgold.cc/watch/hMlLw1LeIK8?t=1554' heldReason: rights --- # Designing Voice Agents for Real Conversations - Chintan Agrawal & Daniel Wirjo, AWS Chat agents get seconds to respond. Voice agents get 200 milliseconds, and if they get it wrong, the user doesn't retry, they hang up. The gap between "impressive voice demo" and "agent people actually want to talk to" is entirely in the engineering: latency budgets, barge-in handling, turn-taking, and the silence detection problem nobody warns you about. In this workshop you'll build a production voice agent using Pipecat (open-source real-time AI framework) that handles the three things most voice implementations get wrong. Interruption: user starts talking mid-response. Do you stop, fade, or finish? The answer depends on context, and we'll build the decision logic. Silence: is the user done talking or just thinking? Get this wrong and you either interrupt them or leave dead air. Latency: you have 200ms total for STT to LLM to TTS. We'll build the streaming pipeline that hits that budget consistently. The thing nobody tells you about voice agents: the hardest problem isn't the AI, it's the audio engineering. WebSocket orchestration, audio chunking, backpressure when the LLM is slower than real-time, echo cancellation when the agent's own output feeds back into the mic. We'll solve all of it. Stack: Pipecat + Daily (WebRTC) + any STT/TTS (Deepgram, AssemblyAI, Whisper) + any LLM. Bring headphones. You'll be talking to your agent by the end. Speakers: - Chintan Agrawal (Amazon Web Service): Chintan Agrawal and Daniel Wirjo are Solutions Architects at AWS, focused on AI and SaaS startups. They are passionate about helping customers unlock value from their data and accelerate innovation on AWS. By partnering closely with founders and engineering leaders, they enable organizations to build scalable solutions, drive business growth, and realize the full potential of cloud and AI technologies. LinkedIn: https://www.linkedin.com/in/chintan-agrawal-87a866135/ GitHub: https://github.com/wirjo/pipecat-turn-detection-demo Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/designing-voice-agents-for-real-conversations-chint-20b3986c2533 ยท Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.