A hands-on workshop by AWS Solutions Architects Chintan Agrawal and Daniel Wirjo on building production-ready voice agents with Pipecat, an open-source real-time AI framework. The session covers the core engineering challenges that separate voice demos from usable agents: barge-in and interruption handling with context-based decision logic, silence and turn-taking detection, and meeting a roughly 200ms end-to-end latency budget across a streaming STT-to-LLM-to-TTS pipeline. It also addresses audio engineering issues such as WebSocket orchestration, audio chunking, backpressure when the LLM runs slower than real-time, and echo cancellation. The demonstrated stack combines Pipecat, Daily (WebRTC), and pluggable STT/TTS (e.g., Deepgram, AssemblyAI, Whisper) and LLM services. # Channel Identity Observations ## Presenter Identity & Affiliation - **Chintan Agrawal** and **Daniel Wirjo** introduce themselves as **solutions architects on the AWS APJ startup team**: > "I'm Chintan and I have my colleague Daniel with me. We are solutions architect on the AWS APJ startup team." - Daniel confirms the same affiliation: *"I'm also a solutions architect with the AWS startups team."* ## Intended Audience - **Developers and builders of voice agent pipelines** — evidenced by: - Daniel's framing: *"hopefully you can kind of incorporate some of the learnings into your own voice agent pipelines."* - Walkthrough of Python source code, pip-installable models, and debug logs. - References to deployment choices (local machine, GPU, cloud, "Picovoice Cloud for managed hosting… or the AWS guidance reference architecture for enterprise deployments"). - Sub-audiences explicitly named: - **Startups** (via the AWS APJ startup team framing and startup-focused recommendations). - **Enterprise/compliance-conscious builders** — *"really useful when you either want full control of your voice pipeline just from a compliance perspective or if you want to fine-tune and customize your own turn detection model."* ## Claimed Scope of Content - Topic explicitly stated at the outset: > "today we are going to talk about something we've been working on for a while. Which is turn taking in voice agents." - The talk claims coverage of: - The **200 ms constraint** ("the physics of why voice is hard"). - **Pipeline architecture** ("what are the components, where turn taking actually lives"). - **Three levels of solving** the problem: Silero VAD, STT-provider turn detection, and a local smart-turn model. - **Latency budgets and production issues**: "the latency, the budget and production issues that we've seen." - Format: **conceptual overview + hands-on demo** — *"Daniel is going to… make it more concrete by running all the three demo for you."* ## Self-Descriptions of Expertise/Experience - Claim of first-hand production observation (repeated "we've seen" / "we've seen customers"): - *"a lot of production system even today we see are running agents just by using this."* - *"We see a lot of customers use that."* - *"a few production issues that we've seen… false interruptions significantly increase your escalation rate."* - Claim of original benchmarking work: > "We benchmark current models specifically for voice, and the target we kept it as under 700 ms time to first token." - Benchmarks dated *"recording in June 2026"* (also restated as "June 26"). ## Demonstrated Presentation Patterns (Observable) - **Technical depth**: specific metrics cited throughout (e.g., Silero VAD as a "300,000 parameter model… 2 megabyte model"; Smart Turn "58.9% recall and 68.4 percent precision"; latency figures like "Nemotron-3 Ultra gives us a 529 P50 latency. GPT-4.1 at about 536"). - **Third-party attributions**: cites external sources — "Salesforce… published their results in March 26th," "Meta's published paper earlier this year," and "a breakdown from Quintela, who was a co-founder of Daily.co." - **Ecosystem-specific tooling demonstrated**: live demos using **Picovoice/PipeCat**, Silero VAD, Cartesia STT, Sonic TTS, and Anthropic Claude — with Daniel describing a repository "we've created": *"this Picovoice Cat 10 [turn] detection demo, which is just a simple repository that we've created."* - **Demo-driven structure**: three sequential live examples (silence detection, STT turn detection, local smart-turn model), each shown with real-time debug logs. - **Vendor-neutral-but-AWS-anchored posture**: discusses multiple providers (Cartesia, Deepgram, OpenAI, Meta, Picovoice) while steering to AWS deployment options at the close. ## Claims vs. Demonstrations — Distinction Notes - **Creator claims**: affiliation, having "been working on this for a while," running benchmarks, and observing customer production patterns (not independently verifiable from the material). - **Demonstrated**: working code walkthroughs, live voice agent interactions, and debug-log interpretation — all directly observable in the transcript. # Delivery Observations: "Designing Voice Agents for Real Conversations" ## Overall Structure - **Two-presenter format**: Chintan covers conceptual/architectural content; Daniel delivers the hands-on demos. Handoff is explicit: *"Okay, over to you, Dan."* - **Signposted roadmap**: The first speaker previews the agenda upfront: *"we'll start with the 200 millisecond constraint which is basically the physics of why voice is hard. Then we look into the pipeline architecture..."* - **Three-level framing** organizes the technical content: *"Going from simple silence detection all the way up to running your own turn detection model."* - **Closing pattern**: Both segments end with summary statements — *"So, to summarize your three levels..."* and *"So that's in a nutshell how turn detection works."* ## Tone - **Conversational and informal**, with casual fillers throughout: *"you know,"* *"like,"* *"uh,"* *"um."* - Colloquial, vivid phrasing when describing behavior: the agent *"is going to be snapping,"* it might become *"super patient,"* users *"just hang up on you,"* and the agent *"knows when to shut up."* - Daniel's tone is softer and hedging: *"hopefully um you can kind of um incorporate some of the learnings,"* *"So yeah, let's get started."* ## Teaching Habits & Recurring Techniques ### 1. Contrast as an explanatory device (repeated) - Left-vs-right demos: *"on the left when the user says I want to fly... the agent just does not notice. ... On the right side, the same exact interaction is happening, but the agent is able to capture the interruption in under 200 milliseconds."* - *"Both the scenarios, the LLM was identical. It was the same model. It was the same prompt, but the difference in the user experience is so different."* - Interruption classification by color: *"the red is the it's a correction... the amber could be just some sort of a filler background noise... Blue means like a signal for cough."* ### 2. Concrete numbers, repeated precisely Latency figures recur across the talk: **200 ms**, **755 ms**, **1.1–1.3 s**, **529/536 ms P50**, **58.9% recall / 68.4% precision**, *"32 milliseconds"*, *"50 ms"*. Numbers are usually paired with interpretation: *"that's like almost 4x slower than how humans naturally take turns."* ### 3. Everyday-conversation analogies - *"think about a normal conversation when someone says, 'Yeah.' Like, if they send an acknowledgement while you're speaking, you don't stop, right?"* ### 4. "Trade-off" framing - VAD silence threshold: *"If you keep it very low, the agent is going to be snapping... But if you keep it very high, the agent might become super patient."* - Level two: *"the trade-off is with the transparency. Because when it is working, it works great. But when it misfires... you like have no way to figure out."* ### 5. Narrated live demos Daniel narrates each action and links it back to the concepts: *"As you can see, I haven't really finished my uh sentence yet, but the voice agent is already responding."* and *"As you can see, this line here... says that um the end of turn state in is complete."* ## Recurring Phrases | Phrase | Speaker | Function | |---|---|---| | *"So, as you can see"* | Mostly Daniel (10+ times) | Demo narration anchor | | *"So what I might do is..."* | Daniel | Transitions between demo steps | | *"Like"* | Both, very frequent | Discourse filler | | *"So, coming to..."* / *"So let's..."* | Chintan | Topic transitions | | *"It depends"* / *"depends entirely on what you're building"* | Chintan | Qualified advice | ## Communication Patterns - **Direct address to the audience's building context**: *"If you're working on a budget of 755 milliseconds and every millisecond count, you also need to understand what's actually in the pipeline."* - **Hedged, honest caveats about limitations** rather than overselling: *"there's like no universally right answer,"* *"Meta... they haven't released the code, so you really cannot deploy it,"* *"we're nowhere near the human speed of interaction."* - **Practice-oriented recommendations**: *"in practice, you would probably combine both the turn detection model and the silence detection."* - **Signposting transitions in demos**: *"So, let's just clear that and I'll refresh that. Okay..."* ## Delivery Irregularities (Notable) - A **stutter/restart sequence** appears mid-transcript around the LLM benchmark slide: *"But also, like what matters more in voice uh than P95. So, which LLMs are actually fast enough for this? We benchmark current models..."* — the segment repeats itself several times before settling, suggesting a live stumble or edit point. - Occasional **name inconsistencies** in the transcript ("Chintan" vs. "Jinson," "Daniel Wirjo" vs. "Daniel Weijia"), likely transcription artifacts — this is uncertain and may not reflect actual speech. - At least one **self-correction**: *"it's going to look at the prosody and it's going to look at prosody and intonation"* ## Uncertainty Notes - Filler frequency ("like," "um") is high for both speakers, but exact counts are approximate; transcript punctuation may not fully reflect spoken rhythm. - The repeated demo phrasing (*"as you can see"*) is clearly a stable Daniel pattern; whether Chintan shares it cannot be strongly established since he does not run live demos. # Observation Record: Designing Voice Agents for Real Conversations **Speakers:** Chintan Agrawal (AWS APJ startup team, solutions architect) and Daniel Wirjo (AWS startups team, solutions architect) ## Core Thesis (Chintan) - Turn-taking in voice agents — how the system knows when a user has finished speaking — is an **audio engineering problem, not an LLM problem**. - A perfect model and prompt can still feel broken if turn-taking is off; identical LLM/prompt can yield very different UX depending purely on the audio pipeline. ## The 200ms Constraint - Humans switch conversation turns in ~**200 milliseconds**. - At **800ms** things feel off; at **1.5 seconds** users hang up. - Unlike text chat agents (which may get 5 seconds), voice agents don't have that luxury. - Cited: **Salesforce** published pipeline results (March 26) with best measured response time of **755ms** — ~4x slower than human turn-taking. ## Pipeline Architecture - Main components: STT → LLM → TTS. - **Voice Activity Detection (VAD)** sits at the front and controls turn-taking. - **Smart turn detection**: watches VAD signal + audio features; decides respond-now vs. wait. - **Interruption handler**: on barge-in, flushes downstream (TTS stops, LLM generation cancelled) within ~**50ms**. ## Level 1 — Silero VAD - ~300,000 parameter, 2MB model; STFT features, 4 conv layers, LSTM, sigmoid speech probability. - The **minimum silence milliseconds** parameter defines the UX: too low → agent cuts people off; too high → dead air. - No universal answer: sales agent ~200ms; domains needing user thinking time ~1,000–1,200ms. - **Limitations:** VAD can't distinguish breath-catch vs. finished thought; can't handle barge-in; same silence may be completed sentence, incomplete thought, thinking pause, or backchannel — VAD treats all identically. ## Level 2 — STT-Embedded Turn Detection - Cartesia (turn detection in STT websocket) and Deepgram ("endpointing") emit turn-end events. - P50 latency: ~**300ms** (Cartesia), ~**250ms** (Deepgram Nova 3). - Advantage: uses full audio signal + linguistic context — more information than VAD. - **Trade-off — transparency:** when it misfires, no logs or visibility; decision made on someone else's server, "you just got to live with that." ## Level 3 — VAD + Smart Turn (Local Model) - Silero VAD runs locally as base signal; Smart Turn model runs during silence. - **Smart Turn V3.2:** 58.9% recall, 68.4% precision; BSD-2 licensed, 8MB, pip-installable. - Roughly 6/10 sentence-endings caught quickly; remaining 4 fall back to the VAD timer — user is never stuck waiting, just slower. - Comparison: Meta paper (~March) reported **87.7% recall** but code unreleased — not deployable. - **Interruption classification:** user speech detected in 32ms, pipeline flushed in 15ms (handled by PipeCat). Chintan argues the agent should distinguish: correction (red — stop), filler/background (amber — continue quieter), cough/noise (blue — ignore and finish). Most current systems stop every time; this is improving. - Code-wise all three levels share an identical pipeline; only the analyzer configuration differs. ## Latency Budget (attributed to Quindela/co-founder of Daily.co, creator of PipeCat, per Chintan) - API setup: ~40ms (making an API call), 52ms network/jitter buffer (physics), ~300ms transcription + endpointing, **500–650ms LLM time-to-first-byte** (dominant bottleneck), 120–85ms TTS network out/playback. - Total: roughly **1,100–1,300ms** typical; **800–1,300ms** for most API-calling developers. - Quindos team demonstrated ~**500ms** voice-to-voice by co-locating models on one GPU cluster — an achievable floor with infrastructure investment. - Key insight: **STT + LLM consume ~2/3 of the budget** — the only two meaningful levers. ## LLM Benchmarks (June 2026, per Chintan; target <700ms TTFT) - Nemotron-3 Ultra: 529ms P50. - GPT-4.1: 536ms P50 — but spikes to **1.7s at P95**. - Claude 3: over **4 seconds** at P95. - P95 tail matters most in voice: one slow response ruins the flow; can't average it out. - **Multi-turn drift:** after 15–20 turns, models may ignore parts of the system prompt, get verbose, go off-script — requiring context pruning or session resets. ## Production Issues (Chintan) - **False interruptions increase escalation rates** — users cut off incorrectly are more likely to request human support. - 755ms remains the best measured voice-to-voice for cascaded pipelines; nowhere near human speed, so turn-taking must do heavy lifting. - 58.9% recall is the best deployable turn detection today; space expected to improve. - Infrastructure is hard: five systems scaling and failing differently. Suggested: **Picovoice Cloud** (managed hosting) or **AWS guidance reference architecture** for enterprise deployments. ## Demo (Daniel) - Uses a Picovoice/PipeCat demo repo with three examples; recommends the open Smart Turn model for full control (compliance) or fine-tuning on own data. - Pipeline: Cartesia STT, Anthropic Claude model for LLM, Sonic TTS ("British lady" voice); Silero configurable via confidence-to-trigger, silence duration, minimum audio volume. - **Example 1 (Silero):** with "I'm thinking of going to um…" the agent cut in early because the default stop seconds (300ms) triggered end-of-turn. - **Example 2 (STT turn detection):** Cartesia emitted turn events and correctly waited on the incomplete sentence until a complete one ("Sydney?"). Daniel qualifies: in practice you'd combine with silence detection — he was muted/silent for a while and the agent didn't respond; VAD is also useful for barge-in. - **Example 3 (Smart Turn):** incomplete sentence showed "end of turn: incomplete" but completed via silence fallback; a full sentence ("I'm actually thinking of going to Sydney") was classified by the local Smart Turn model as complete with high probability, and the agent responded immediately. ## Preserved Tensions / Qualifications - Chintan: "no universally right answer" for the silence parameter — domain-dependent. - Level 2 works well *when* it works; opacity on misfire is an accepted trade-off. - Daniel's demo shows the limitation of each approach in real time (Silero cutting in early; STT detection missing prolonged silence), explicitly recommending combined approaches in production. - Transcript contains garbled/repeated passages (e.g., repeated LLM benchmark framing, inconsistent names: "Cartesiant/Cadence," "Quintela/Quindos/Quindela," "Jinson" vs. Chintan) — recorded as expressed without correction. # Speaker Attribution Record ## Video **Title:** Designing Voice Agents for Real Conversations - Chintan Agrawal & Daniel Wirjo, AWS **Video ID:** `01a15ca8-74e0-7775-b6ec-20b3986c2533` ## Identified Speakers | Speaker | Role | Evidence | |---|---|---| | **Chintan Agrawal** | **Co-presenter / creator-side speaker** — Solutions Architect, AWS APJ Startup team | Self-introduction: "I'm Chintan and I have my colleague Daniel with me. We are solutions architect on the AWS APJ startup team." | | **Daniel Wirjo** (spoken as "Daniel Weijia"; also addressed as "Jinson" and "Dan" by Chintan) | **Co-presenter / demonstrator** — Solutions Architect, AWS Startups team | Self-introduction: "my name is Daniel Weijia. I'm also a solutions architect with the AWS startups team." Runs all three demos. | | **Demo voice agent** | **Non-human speaker** — pre-recorded/running demo assistant ("travel assistant") | Greeting: "Hey there, I'm your travel assistant..." within Daniel's demo segments. | > Note: Chintan addresses Daniel as both "Dan" and "Jinson" — a name inconsistency in the transcript; identity remains the same co-presenter. ## Segment Boundaries - **Chintan:** Conceptual overview — 200 ms constraint, pipeline architecture, three levels of turn detection, latency benchmarks, production issues, closing handoff: "Okay, over to you, Dan." - **Daniel:** Hands-on demo of three turn-detection examples (Silero VAD, STT-native turn detection, Smart Turn model), including live voice agent output. ## Referenced Sources & Third-Party Claims (per speaker) ### Attributed to Chintan - **Salesforce** — published pipeline pivot results "March 26th"; best measured response time 755 ms. - **Cartesiant (Cartesia)** — turn detection in STT websocket; P50 latency ~300 ms. - **Deepgram** — endpointing; ~250 ms for Deepgram Nova 3. - **Smart Turn V3.2** — 58.9% recall, 68.4% precision; BSD-2 licensed; 8 MB. - **Meta** — published paper (~March) reporting 87.7% recall; code not released. - **Quintela (Quindos team)** — co-founder of Daily.co, creator of "Pipe Band" (likely Pipecat); latency breakdown production measurements; ~500 ms total voice-to-voice floor. - **Model benchmarks (June 2026, per speaker):** Nemotron-3 Ultra 529 ms P50; GPT-4.1 536 ms P50, 1.7 s P95; Claude 3 "over 4 seconds" P95. - **Picovoice Cloud** and **AWS guidance reference architecture** — see promotion flag below. ### Attributed to Daniel - Demo stack: **Pipe Cat (Pipecat)** framework, **Cartesia** STT model, **Sonic** TTS, **Anthropic Claude** LLM, **Silero VAD**. - Demo repository: "Picovoice Cat 10 detection demo" (transcript garbling; likely a turn-detection demo repo). ## Promotional Segments — Flagged - **Closing infrastructure recommendation (Chintan):** > "You can explore Picovoice Cloud for managed hosting where your focus is on agent logic or the AWS guidance reference architecture for enterprise deployments..." — Promotes AWS's own reference architecture and a third-party paid service; flagged as **promotional**. - **References to Smart Turn, Pipecat, and Picovoice** recur throughout and in the demo repo name; treated as **product-referential**, with the closing recommendation being the clearest promotional instance. ## Provenance Notes - **Curated metadata:** Not supplied; speaker identities drawn solely from transcript self-introductions. - **Comment availability:** Not stated in source. - **Uncertainty:** Benchmark figures are speaker-reported and dated by the speaker as "June 2026"; several product names appear garbled in the transcript ("Cartesiant/Katisha Inc./Cadence," "Pipe Band/Pipe Cat," "Picovoice Cat 10"). These are recorded as transcribed, not corrected. - No timestamps available in the transcript. Hey everyone. I'm Chintan and I have my colleague Daniel with me. We are solutions architect on the AWS APJ startup team. So today we are going to talk about something we've been working on for a while. Which is turn taking in voice agents. What I mean by that is how does the system actually know that you have finished speaking and it's time for the agent to respond? These are all audio engineering problems. They are not LLM problems because you can have the perfect model, perfect track but the experience still might feel broken if the turn taking is off. So we are going to walk you through the concepts and three different approaches to solving this. And Daniel is going to you know make it more concrete by running all the three demo for you. So to kick things off uh we'll start with the 200 millisecond constraint which is basically the physics of why voice is hard. Then we look into the pipeline architecture, what are the components, where turn taking actually lives. We'll then look at three level of solving this problem. Going from simple silence detection all the way up to running your own turn detection model. And then we'll also talk about the latency, the budget and production issues that we've seen. So but before any of that, I just want to show you the problem statement more clearly because you have the same user. You have the same sentence on the both side but the user outcome is very different because on the left when the user says I want to fly and midway they try to correct themselves, the agent just does not notice. Because it keeps going on for almost 2 seconds while the person is sitting there trying to get a word in. On the right side, the same exact interaction is happening, but the agent is able to capture the interruption in under 200 milliseconds. And immediately backs off so the user can now speak. Both the scenarios, the LLM was identical. It was the same model. It was the same prompt, but the difference in the user experience is so different. And the difference is purely because of the audio pipeline. And how fast does it notices like someone else is talking and it knows when to shut up. Like that's the problem we're trying to solve today. So, we'll cover this 200 millisecond constraint because 200 milliseconds is how fast humans switch turns with each other in a conversation. And the implications are pretty brutal because at 800 milliseconds, things start to feel off. While at 1.5 seconds, your user just hang up on you because you have like your HR chat agents might get 5 seconds to respond and nobody will care. But with the voice agents, you don't get that luxury. We've seen recently like Salesforce was able to pivot the pipeline and they published their results in March 26th. But even there, best measured response time was 755 milliseconds. So, that's like almost 4x slower than how humans naturally take turns when talking. So, the question here becomes how do we close this gap and where we cannot close it like at least how do we get the turn taking right? So, the user experience does not feel as bad as the raw numbers might suggest here. So, if you're working with a budget of 755 milliseconds and every millisecond count, you also need to understand what's actually in the pipeline. Like where does the time go and where does this turn taking fit? So, the pipeline itself you probably might know the main components, the STT, the LLM, the TTS. You have the audio in, text out, text in, audio out. But the piece that's often missing like from people while people are thinking about this is this voice activity detection. Commonly called as VAD. It's a tiny component that sits right at the front and it is the one that controls turn taking. Its job is to basically detect whether the user has stopped talking or not. And the two other thing that you set alongside main pipeline are also very critical for turn taking. So, you have the smart turn detection which watches the VAD signal. Plus the audio features and then it has to make that decision that whether should it respond now or should it wait because the user might still be talking. And the interruption handler is triggered when someone barges in. So, what it'll do is it'll propagate a flush of downstream like the TTS would stop and the LLM generation need to be canceled so that the pipeline is ready for the new input within about, you know, 50 ms. So, if you remember I was talking about like how there are different levels to, you know, building this turn taking in voice agents. So, we'll start with a level one. Level one is your Silero VAD. So, this is a simple component which you fully own and like a lot of production system even today we see are running agents just by using this. So, coming to Silero VAD, it's a small 300,000 parameter model. It takes in like a short term Fourier transform. It has a and it takes raw audio and it'll convert it into a spectral features. It has four convolutional layers to pick up the pattern and an LSTM that gives it memory so across the frame so it's just not looking at one chunk in isolation. And then it has a sigmoid that gives a probability of speech and it's like a very small 2 megabyte model. And the one parameter that minimum silence millisecond is basically the entire user experience of level one. If you keep it very low, the agent is going to be snapping like it will cut people off while they're still, you know, thinking. But if you keep it very high, the agent might become super patient and it will never interrupt but you might get to uh get like a dead air where people will wonder if the agent is still connected or not. So there's like no universally right answer. It depends on the domain. If you're working on it from a perspective of a sales agent, you probably want it to be like 200 milliseconds. If you're in a domain where you need some to give some time to your user to respond, it could be like 1 uh thousand to 1,200 milliseconds. It depends entirely on what you're building. So VAD works well like that's why a lot of people are using it and it's generally good at what it does that is to return fast response to the basic question of whether there's someone talking right now. But uh there are things like it was never designed to handle because there'll be situations where just knowing that is there a silence will not be enough because you also need to answer these questions of how long to wait like uh the silence duration. If someone is pausing for 300 milliseconds or 400 milliseconds, VAD is seeing the same exact thing. It has no idea like whether the person is catching up their breath or whether they've completely finished their thought. So like they cannot distinguish that. And the second scenario is like, what if they, you know, the user talks over the agent, like, it barges in. It does not know how to handle it. And the same silence can have a different intent. And it can be like completely different things. It could be like a completed sentence, it would be an incomplete thought, it could be like a thinking pause. Or maybe it could be something as basic as like a backchannel acknowledgement. Uh the situation of the silence is same, but the intent again was very different in all the scenarios, but VAD considers all four equally, and it literally cannot tell them apart. So, those are the limitation of level one. And with the level two, basically what you get is that you let the STT service tell you like when the turn is over. So, Cartesiant does turn detection right inside their STT websocket. So, when they stream your audio in, the server would handle both the transcription and the turn detection together, and it will emit event when it thinks like the turn is done. Similarly, Deepgram does the same thing with their what they call as their I think at the endpointing. And their P50 latency for Cartesiant is about 300, and it's about 250 for Deepgram Nova 3. Both of these are also working really well. Like, we see a lot of customers use that. Because they are using like full audio signals plus some linguistic context to make these decisions. So, which is like much more information than your VAD will ever have. However, the trade-off is with the transparency. Because when it is working, it works great. But when it misfires, or when it cuts someone off at the wrong moment, then you like have no way to figure out because there's no law that will say uh it was done because of this reason, because of the what was what it saw. And that decision was made inside someone else's server, and you just got to live with that, basically. So, that brings us to level three. So, in level three, you keeps a little VAD running locally for basic signal that is there a speech. And you add Smart Turn on top of it, which is a small model that runs during silence. So, Smart Turn V3.2 is the latest model, like, and uh let me share some numbers with you. And it has about 58.9% recall and 68.4 percent precision. So, what this means is that about six out of 10 time when someone finishes a sentence, Smart Turn will be able to catch it quickly. But the other four times, it might not be confident enough, but that's also fine because you still have your VAD timer, which is running underneath as a safety net. If the Smart Turn doesn't fire, your stop second will kick in, like, at the configured latency anyway. So, it could be like 300 ms. So, in that scenario, you are never stuck waiting. You just, like, get the fast respond when the model is confident. And you get a slightly slower but a safer respond when it's not. The other number on this list is from, like, a Meta's published paper earlier this year, um around March. There, they had reported a higher recall at 87.7%, but they haven't released the code, so you really cannot deploy it. Smart Turn is BSD-2 licensed, and it's a 8-MB small model. You can pip install it today as well. So, yeah, like, that works out better for a lot of our customers. We decide when to start talking. But like, what about the other direction? What happens when the user interrupts and while you are already, you know, the agent is talking? So, on the left side, you can see what happens mechanically. Like, user opens you mouth, the windy will pick it up in 32 milliseconds and then then like 15 milliseconds, the entire pipeline is flushed. TTS stops, LLM cancels, everything is clean. Like, the user never hears the agent. You know, that part is taken care by Pipe Cat. But, the important thing is like, should it have stopped there on because think about a normal conversation when someone says, "Yeah." Like, if they send an acknowledgement while you're speaking, you don't stop, right? You know, they're just agreeing, but if you they say like, "Okay, wait, no. Like, that was wrong." You have to stop immediately. Same thing here. Like, the red is the it's a correction. Stop on everything. They amber could be just some sort of a filler background noise. But, you want to go ahead and keep going, maybe in a bit quieter. Blue means like a signal for cough for a background noise. So, you can just ignore it and finish your sentence. So, at level three because you own this classification, you can start distinguishing between a real interruption and someone just, you know, going um. So, today most of these systems stop every time, but this is a piece that's improving. Like, we are seeing a lot of incremental improvement in this. So, by now that you've seen all three levels and if you look at these three Python files, pipeline is almost identical in all of them. The only thing that differs is how you answer all of these questions. When is the user done talking? In file one, you pass in Celero VAD analyzer and that says the user had been silent for 300 milliseconds, so you can assume they're done. In file two, you swap it with the Cadence turns STT service class and now the service is going to tell you when the turn is over. And in file three, you keep your Celero VAD analyzer, but you also add like a local smart turn analyzer version three, which is sort of a small model that runs during your silence and it's going to look at the prosody and it's going to look at prosody and intonation to decide whether that pause meant I'm done or I'm still thinking. So, you have the code-wise the same pipeline, but the configuration changes, which you know, leads to completely different behavior. So, to summarize your three levels, where our level one was Celero VAD, so you basically own the silence detection completely. Level two, you let the STT provider handle it for you. And in most cases, it it will speak smarter, but then you cannot see what happened inside it. Level three is your VAD plus smart turn, so where we basically own everything, like you have full portability. And now coming to the latency piece, because we've been throwing around these numbers, 755 milliseconds, 1.3 seconds, but let me show you where all the time actually goes in. So, this is a breakdown from Quintela, who was a co-founder of Daily.co and is the person who created Pipe Band. And these are some of the numbers based on their production measurement. So, you have the numbers for making an which is about 40 ms. Then you have network and jitter buffer of 52. So, these are physics. Like you can't really change them much. Then you have your transcription plus end pointing. That's about 300 ms. Again, that's your STT doing its thing. LLM time to first byte, and this is generally the dominant bottleneck because in a typical API setup, you're looking at 500 to 650 ms depending on what model you're calling and from which cloud provider, which region. And then you have your network out and playback TTS. That's it's going to take about 120 to 90 85 ms. So, you're totally looking at roughly 1,100 to 1,300 ms in a standard setup calling cloud APIs. Now, the Quindos team had demonstrated about 500 ms total voice-to-voice by co-locating all models in the same GPU cluster. Because they were able to eliminate the network hops within the service. And it's sort of an achievable floor because if you're willing to invest in infrastructure, but for most of the developers today calling APIs, you're somewhere in the range of 800 to 1,300 ms. And the key insight to note here is that the STT, LLM, together eat about 2/3 of the is latency budget. So, these are the really the only two levers that we have to move this latency metric needle in a meaningful manner. So, coming to like which LLMs are actually fast enough for this? We benchmark current models specifically for voice, and the target we kept it as under 700 ms time to first token. Because anything slower will push our total response past where the user is going to start noticing. So, for now, like recording in June 2026, what we've observed Nemotron-3 Ultra gives us a 529 P50 latency. GPT-4.1 at about 536. But also, like what matters more in voice uh than P95. So, which LLMs are actually fast enough for this? We benchmark current models, specially uh for voice, and So, coming to the So, coming to the LLMs models, which are actually fast enough for the We benchmark current models well for voice. Uh so, these benchmarks are from June 26, and the target that we kept was under 700 ms for time to first open, because anything slower would push our total response time past the limit, like where users is going to start noticing. So, for now, the Nemotron-3 Ultra at 529 P50, we observed GPT-4.1 at 536, again P50. But what matters more in voice than maybe anywhere else is that the P95 tail. Because GPT-4.1 was great at P50, but it spikes to 1.7 at P95. And it was even worse for Claude 3 that it hit over 4 seconds and because in a conversation you cannot average this out because one slow response and your entire flow is gone. And there's one more dimension that people often miss, which is the multi-turn trip because after 15 or 20 turn, sometime model starts ignoring parts of the system prompt. They might get to verbose, they go off spirit. Because in that's fatal, you can't just dump a wall of text on someone if the instruction following degrades across turn. Then you have to do some sort of a context pruning or session resets. So a few production issues that we've seen that false interruptions significantly increase your escalation rate. Because when agents cut people off incorrectly, then users are more likely to request for a human in the loop uh support. And a 755 ms that's still I think the best measured voice-to-voice for a cascaded pipeline like the ones which we propose. We're nowhere near the human speed of interaction, which means the return taking also needs to do a lot of heavy lifting to make the experience feel better than the raw numbers it would suggest. And at 58.9% recall on smart turn, that's actually the best deployable turn detection you can get today and because although it still leaves like four out of 10 turns to be filled by a uh VAD timer, but this space is going to improve a lot over the next year few years and uh we'll have better model. And on infrastructure running all of this in production is also hard because five systems that all scale differently and can also fail differently. You can explore Picovoice Cloud for managed hosting where your focus is on agent logic or the AWS guidance reference architecture for enterprise deployments with all the enterprise guardrails. So yeah, like uh Uh that's it from me. Everything uh we've discussed or mentioned is linked on the screen that I pulled benchmark. Okay, over to you, Dan. \>\> Thanks, Jinson. Hi, everyone. Uh my name is Daniel Weijia. I'm also a solutions architect with the AWS startups team. So I think um Jinson walked you through a great overview of some of the key challenges when building voice agents. So what I'm going to do in this session is just to walk you through a hands-on demo um just to make the concepts a bit more concrete. Um hopefully um you can kind of um incorporate some of the learnings into your own voice agent pipelines as well. So what we'll be going through is just this Picovoice Cat 10 detection demo, which is just a simple repository that we've created. Um we'll talk through three examples. So one with um silence detection, another with um built-in 10 detection in the speech-to-text model. And then finally, um we'll go through the open smart 10 model as well. So this is really useful when you either want full control of your um voice pipeline just for from a compliance perspective or if you want to also fine-tune and customize your own 10 detection model based your own based on your own data. So yeah, let's get started. So let's just go through the source code of the first example. As you can see here, um using Silero is pretty easy in Pipe Cat. All you have to do is import the relevant libraries, and you can configure the parameters as well. Um I haven't actually uh configured any of the parameters, so these are just the default, but you can change the parameters like the confidence to trigger, the silence duration before the turn ends, and the minimum audio volume as well. And in terms of the voice voice pipeline, it's pretty standard. So, we've used um an STT model, the LLM, and also a TTS model here. We've just defined it as the Cartesia model for STT. Same with the Sonic model for TTS, and as you can see, we've used the voice that sounds like a a British lady. And for the LLM in the voice agent's brain, we've just used the Anthropic Claude like cool model. So, we'll see um what that looks like, and I'll spin up this example right now. So, So, if you haven't used Pipe Cat before, one of the nice things is that it can spin up a local prototyping environment. Really useful for playing around with your voice agent pipeline. So, let me just see if this now works. Okay. So, as you can see, it will come up with a um local uh development environment. So, what I might do is just connect to the voice agent, but before I do that, I might just mute just in case um I'm interrupting the voice agent. It should come up with like an opening greeting, and then I'll test it with some uh interactions to see how the turn detection is working. \>\> Hey there, I'm your travel assistant and I'm excited to help you plan an amazing trip. So, where are you thinking of going? Anywhere in the world you've got your eye on? \>\> As you can see, the assistant's gone ahead to do the opening greeting and on the right here in the terminal, I've just got um some real-time logs just so we can see what's happening. So, what I might do now is just I'm going to pretend that I'm uh thinking about my travel destination and see how the voice agent behaves. So, I'm thinking of going to um \>\> I'm all ears. Where are you thinking of heading? \>\> As you can see, I haven't really finished my uh sentence yet, but the voice agent is already responding. So, you can see how this works in the debug logs and as you can see, there's an end of turn complete here and the reason for that is due to the stop seconds parameter, which is um set to um 300 milliseconds, it looks like here. So, that's why it's gone ahead and ended the turn. Um what I might do is I might just exit this local server and try the second example. So, let's just clear that and I'll refresh that. Okay, so I'm going to try the second example here. And in this example, we'll use a um speech-to-text model which has turn detection. So, we'll see how the voice agent behaves. Okay. I'm going to try uh the same example, so let me just mute myself here. \>\> Hey there. I'm your travel assistant and I'm excited to help you plan an amazing trip. So, where are you thinking of heading? Are you dreaming of a beach getaway, a city adventure, mountains, or maybe somewhere completely different? \>\> All right, let's see how the model behaves this time. Hi, I'm thinking of going to um As you can see, the model is pretty smart. It now can detect the turn, so let's see if we can find out what's going on in the debug logs. So, um let's see if it emits uh some of those turn taking events here. So, as you can see, um the Katisha Inc. model does emit some turn events here. And as you can see, it hasn't really completed the turn. It hasn't sort of um responded back with any response. But, maybe I can unmute myself and then resume and get to the destination. So, Sydney? \>\> Oh, Sydney is fantastic. You're going to have such a great time there. Are you thinking about when you'd like to go? And how long are you planning to stay? That'll help me figure out the best flights and hotels for you. \>\> So, as you can see, once the model detects that it's a full complete sentence, is able to then respond. Now, in practice, you would also incorporate um some silence detection there because as you can see, I was quite silent. Um I was muted for a while and you'd probably want the voice agent to respond in practice. Um having like voice uh activity detection or silence detection is also useful for um interrupting the voice models um um or what you call barge in as well. So, definitely this is just a a quick demo, but in practice, you would probably combine both the turn detection model and the silence detection. Um just for the final demo, I might quickly show the smart turn model as well and see what that looks like. Um it will behave very similar to to this. It's just that um the model will be running on your uh either your local machine or um on a GPU uh environment as well if you want to deploy that into your into the cloud. So, let me just maybe clear this just to avoid confusion. And then I'll use um I'll refresh this as well. And I'll start now the smart turn example. I'll refresh this to see if we can Okay, there you go. It's starting it. And I'm just going to do the same example. So, I'll mute myself again. \>\> Hey there. I'm your travel assistant and I'm excited to help you plan an amazing trip. So, where are you thinking about going? Are you dreaming of a beach getaway, exploring a new city, or maybe something else entirely? \>\> I'm thinking of going to um \>\> I'm all ears. Where are you thinking of going? \>\> Now, let's take a look at the um both the silence detection, which is incorporated into this example, and also the turn detection model. Hopefully, there are some um debug logs that we can see. As you can see, um it's not detecting that I've completed the turn. The end of turn state is incomplete, but it did complete it um due to the silence that I've um I've um incurred there. Um so, that there was some silence, and therefore it sort of kept going with the with the um next turn. What I might do is see if I can do a complete sentence and see if, you know, um it can be detected through the local smart turn model. I'm actually thinking of um going to Sydney. \>\> Oh, Sydney's fantastic. I'd love to help you plan that trip. So, tell me, what time of year are you thinking about going, and how long are you planning to stay? Also, are you traveling solo, with a partner, or with family? \>\> Okay, fantastic. So, the voice agent was able to respond as soon as I uh completed the sentence, but let's see if that was actually intentional, or was that um something that um you know, can be detected through the smart turn model. So, let's have a look here to see if we can find the smart turn um debug logs. As you can see, this line here, I'm sorry about the legibility of the highlight, but effectively this here says that um the end of turn state in is complete, and the smart turn model is actually classifying that there is a high probability that that um the sentence is complete and therefore it's okay to go to the next turn. So that's in a nutshell how turn detection works. Hope that was helpful.