Talk by Joel Allou and Ornella Bahidika (Microsoft) arguing that AI voice agents don't need frontier models because latency, not intelligence, is the binding constraint. Covers why time to first token drives model choice for voice, how deterministic scaffolding (memory of the student, lesson planning, turn decisions) can be moved out of the model, leaving it to handle only conversation, demonstrated in an AI voice tutor built on a small, fast model, plus where small models still fall short. # Channel Identity Observations — Ace Voice Agent Video ## Observable Self-Description - **Presenters identify themselves and their product directly**: *"Hi, I'm Ornella, and that's Joel, and we built Ace, a live AI voice tutor."* - The talk closes with a repeat of this identity: *"this is Joel and Ornella and we are building A[ce]... and if you have any questions, let us know."* - **Employer affiliation is in the title**: "Joel Allou & Ornella Bahidika, **Microsoft**" — indicating the speakers present as Microsoft-affiliated builders. ## Claimed Scope - The video positions itself as **practitioner knowledge-sharing about voice AI architecture**, specifically: - Latency budgets for voice agents (*"The AI model need to start talking in about 950 milliseconds"*). - Offloading reasoning from LLMs into external scaffolding (*"let's extract all of the thinking away from the model... all of the thinking is extracted into a state machine"*). - Cost/latency tradeoffs between frontier and small models (comparing unnamed-in-reality models referenced as *"Claude 4.7"* / *"Opus 4.7"* and *"Haiku 4.5"*). - They present a **prescriptive engineering rule**: *"Pick the fastest model that your latency budget allows and then spend the rest of your time actually building the scaffolding."* - Applicability is claimed beyond their own product: *"that's true for voice applications... true for real-time applications where latency is of priority and that's true for anything that is high volume."* ## Intended Audience - **Developers/engineers building voice or real-time AI applications** — evidenced by: - Technical demo structure (side-by-side videos with/without scaffolding, latency measurements: *"the answer comes in about 900 milliseconds"*). - Direct engineering guidance framed as tradeoffs (*"this isn't necessarily free. It has a cost"* — small models *"tend to drift... and really needs strict rules"*). - Practical terminology: state machine, scaffolding, latency budget, cost-effectiveness. ## Demonstrated Presentation Patterns - **Two-person tag-team format**: Ornella covers theory/motivation; Joel demos (*"that's the theory. Joel, go on and show them what it actually feel like"*, *"maybe I can add some color"*). - **Product-as-example teaching style**: their own product Ace serves as the case study, with live latency comparisons. - **Balanced framing**: they voluntarily disclose downsides of their approach (*"I have to be honest because this isn't necessarily free"*). - **Candid, conversational tone** with rhetorical hooks (*"Quick gut check. That silence on a voice call..."*). ## Claims vs. Demonstration Distinction - **Creator claims**: small models beat frontier models for voice latency; scaffolding costs are paid "once and in code." - **Demonstrated (but unverified)**: two demo videos are described but not independently verifiable from the transcript; latency figures (~900ms) are asserted, not evidenced in-text. ## No Metadata Provided - No curated metadata fields were supplied; all observations derive from title and transcript only. # Delivery Observation: "Your Voice Agent Doesn't Need a Frontier Model" — Joel Allou & Ornella Bahidika ## Overall Format - **Two-speaker demo talk** with a clear handoff: Ornella opens with the framing/thesis, then explicitly passes the floor: > "And that's the theory. Joel, go on and show them what it actually feel like." - Joel takes the second half, including a **live demo walkthrough** of before/after videos, then closes for both speakers: > "So this is Joel and Ornella and we are building A is again." ## Tone - **Conversational, pitch-style delivery.** Ornella uses casual hooks and direct address: > "Quick gut check." and > "I want to tell you more why that's not a compromise." - Joel is candid and informal, including a **verbal tic moment**: > "So [snorts] in our case, right?" He also uses hedging interjections like > "Yeah, if maybe I can add some color..." ## Vocabulary & Recurring Phrases - **Repeated core concept: "scaffolding"** — appears multiple times in Joel's section: > "if it doesn't have any scaffolding, tend to drift" ... > "So the scaffolding piece is the price" ... > "spend the rest of your time actually building the scaffolding." - **Repetition of "thinking/reasoning" as the thing removed:** > "let's extract all of the thinking away from the model" ... "by removing all of the thinking, all of the logic, all of the reasoning from the model." - **A repeated formula of listing actions the model doesn't do** (Ornella): > "It doesn't decide when happen what's happen in the lesson... It doesn't track what the student knows. It doesn't explain what's next." *(Note: transcript is garbled here; exact phrasing is uncertain.)* - **Concrete technical naming:** both speakers name specific models and metrics — "Claude 4.7," "Opus 4.7," "Haiku 4.5," "about 950 milliseconds," "about 900 milliseconds." ## Teaching Habits & Structural Patterns - **Concrete rule as a takeaway.** Joel distills advice into an imperative: > "So here's a rule. Pick the fastest model that your latency budget allows and then spend the rest of your time actually building the scaffolding." - **Contrast/comparison demonstration.** Joel shows a before/after pair of demos: > "The first video here is without the implementation we've done" vs. > "Same question, but now you see that the answer comes in about 900 milliseconds." - **Honest trade-off framing.** Joel prefaces the limitation with a transparency marker: > "But I have to be honest because this isn't necessarily free. It has a cost, right?" - **Metaphor-rich framing** from Ornella: > "When a voice agent pause for even a second, your brain says it's dead" and > "A frontier model that think for a full second has already lost the room." - **Causal "so" chaining.** Joel frequently connects steps with "So": > "So, if you think about the models of today..." / "So what we are doing is saying..." / "So all of the thinking is extracted into a state machine." This is a stable pattern throughout his section. ## Uncertainty Notes - The transcript appears to be **auto-generated and garbled in places** (e.g., "a leader of every instant stay," "building A is again"), so exact wording and some phrase counts are unreliable. - Ornella's segment is short, so patterns like her rule-of-three negation structure are plausible but only observable **once** — insufficient evidence to call it a stable habit. # Position Record: "Your Voice Agent Doesn't Need a Frontier Model" **Speakers:** Ornella Bahidika and Joel Allou (Microsoft), builders of **Ace**, a live AI voice tutor. ## Central Claim A small model is not a compromise for voice agents — latency, not intelligence, is the binding constraint. ## Ornella's Positions ### Latency as the core budget - Silence on a voice call is perceived as breakage: *"When a voice agent pauses for even a second, your brain says it's dead."* - **"Our budget was never IQ, it's milliseconds."** The model must start talking in **about 950 milliseconds**; a frontier model that thinks for a full second "has already lost the room, no matter how good the answer is." ### Narrow the model's job - The small model does **not** decide lesson flow, track student knowledge, or explain what's next — a surrounding system handles those and hands the model **a summary every turn**. - What remains is the one thing the model is good at: **talking**. ## Joel's Positions ### Extract the thinking from the model - Frontier models (he cites **Claude 4.7 / Opus 4.7, Anthropic**) reason well, but reasoning takes seconds — "precisely the problem" for voice. - Solution: move all thinking into a **state machine** coordinating lesson steps, plus an **"intelligent layer"** deriving student mastery. What happens next, display decisions, and answer logic all occur **outside the model**; the model only speaks the output. ### Demonstrated comparison (as presented) - **Without implementation:** Opus 4.7 — reasons for a couple of seconds before answering. - **With implementation:** **Haiku 4.5** — same question answered in **about 900 milliseconds**. - Conclusion: removing logic from the model into code enables smaller, **cost-effective** models better suited to real-time voice. ### Acknowledged cost (qualification) - *"This isn't necessarily free."* Haiku 4.5 **without scaffolding tends to drift** and needs strict rules to stay organized. - Trade-off framing: **"the scaffolding piece is the price"** — but it's paid **once, in code**, not every turn. ### Stated rule > "Pick the fastest model that your latency budget allows and then spend the rest of your time actually building the scaffolding." ### Scope of the claim Applies, per Joel, to: - Voice applications (like Ace) - Real-time applications where latency is a priority - **High-volume** applications In all such cases: *"the model is the smallest part of the system."* ## Unresolved / Caveats - The cost of scaffolding is only qualitatively described ("pay it once") — no figures given. - The comparison relies on Joel's two demo videos; specifics of the question or environment are not stated in the transcript. # Speaker Attribution Record ## Video **Title:** Your Voice Agent Doesn't Need a Frontier Model - Joel Allou & Ornella Bahidika, Microsoft **Video ID:** `01a15ca8-72a0-7f96-8d60-30cb084b4f34` ## Identified Speakers & Roles | Speaker | Role | Evidence | |---|---|---| | **Ornella Bahidika** | Co-creator / co-presenter (introduces herself: "Hi, I'm Ornella... we built Ace") | Opens transcript; presents latency rationale; hands off to Joel mid-talk | | **Joel Allou** | Co-creator / co-presenter, handles live demo ("Joel, go on and show them what it actually feels like") | Second speaking segment onward, including demos and closing rule/advice | - Both speakers are identified in the title as **Microsoft** personnel. - No narrator, guest, or other third-party speakers present. ## Referenced Third-Party Sources & Models - **Claude 4.7 (Anthropic)** — cited by Joel as an example frontier reasoning model ("from Anthropic"). - **Opus 4.7** — used in the first demo clip without their implementation. - **Haiku 4.5** — described as "a much smaller model" used with their scaffolding implementation. ## Third-Party Claims - Joel attributes reasoning capability to Anthropic's frontier model (paraphrased: it "is really good at reasoning" but too slow for voice). - No other third-party claims or external citations appear. ## Promotional Segments - The entire talk is effectively a **showcase of the speakers' own product, "Ace"**, a live AI voice tutor they built; demos of their implementation are product demonstrations. No explicit paid sponsorship or separate ad-read segment is present. ## Provenance & Availability Limits - **Curated metadata:** None supplied beyond title. - **Comment availability:** Not indicated. - **Transcript quality note:** Transcript contains apparent transcription artifacts (e.g., "Claude 4.7," "Opus 4.7" may reflect mis-transcribed model names; product name rendered variously as "Ace" and "A is"). Uncertainty remains explicit. - **Timestamps:** None present in transcript. Hi, I'm Ornella, and that's Joel, and we built Ace, a live AI voice tutor. It run on a small model on purpose, and I want to tell you more why that's not a compromise. Quick gut check. That silence on a voice call, that the difference between a tutor and a broken up. When a voice agent pause for even a second, your brain says it's dead. So, when the answer fit a leader of every instant stay, which for the smallest, biggest model. In voice, that instant is actually a backward. Because our budget was never IQ, it's millisecond. The AI model need to start talking in about 950 milliseconds. A frontier model that think for a full second has already lost the room, no matter how good the answer is. So, we made the model small and took the hardest part jobs away from it. It doesn't decide when happen what's happen in the lesson. It had It doesn't track what the student knows. It doesn't explain what's next. We have a system in place to do that, and they hand the model a summary every turn. What's left for the model is one thing it's really good at, talking. And that's the theory. Joel, go on and show them what it actually feel like. \>\> Yeah, if maybe I can add some color to what Ornella was mentioning. So, if you think about the models of today, especially the frontier model, let's take Claude 4.7, which is uh from Anthropic. The model is really good at reasoning. You can give it a problem, in this case a lesson, and it can reason through it, it can reason through what the student asking and it can come up with the answer. But that is actually precisely the problem. Because the reasoning can take couple of seconds and those seconds are really valuable when you are building voice applications. So what we are doing is saying, "Hey, let's extract all of the thinking away from the model so that the model focuses on only what matters, which is speaking in our case." So all of the thinking is extracted into a state machine. So for Ace, we have thought about all the scenarios that are needed for a lesson. We have built a state machine that is able to coordinate each step to the next and we also added intelligent layer on top to derive some of the mastery that a student might need for the lesson to be complete. So everything when it comes to what happens next, when it comes to what needs to be displayed, when it comes to how to actually answer a question, it's all done outside of the model. And we simply feed that output to the model to speak out. And so let's go ahead and look at an example and see how that works in real time. So the first video here is without the implementation we've done. So it's a simple Opus 4.7. We ask a very simple question and as you can see the model is thinking, it's reasoning and it takes couple of seconds to return the answer back to the user. Hey, what In this video, we've added all everything we just talked about on Haiku 4.5, which is a much smaller model. Same question, but now you see that the answer comes in about 900 milliseconds. And so that's the beauty of building around the model. So by removing all of the thinking, all of the logic, all of the reasoning from the model and actually putting it within the code, we actually saves a lot of time and allows us to use smaller models which are cost effective and actually better at real-time voice applications. And as you can see, this feels almost instant and again that's because all of the smart parts have already happened prior to the model actually speaking. But I have to be honest because this isn't necessarily free. It has a cost, right? A small model like the Haiku 4.5, if it doesn't have any scaffolding, tend to drift on long structure and really needs strict rules in order to be able to stay organized. So the scaffolding piece is the price. But the good thing is you pay it once and in code, right? Not on every single turn. So here's a rule. Pick the fastest model that your latency budget allows and then spend the rest of your time actually building the scaffolding. So \[snorts\] in our case, right? Maybe you build a state machine, you build a reasoning process, you think about scenarios, what happens if this happens, how should your model handle it? Everything that comes with the logic, everything that comes with the harnessing, you do that outside of the model and then allowing the model to focus on that one thing that is really good at. And so that's true for voice applications like A is, that's true for real-time applications where latency is of priority and that's really true for anything that is high volume, right? In those cases, the model is the smallest part of the system. So this is Joel and Ornella and we are building A is again and if you have any questions, let us know. Thank you. \>\> Thank you.