Talk by Allen Pike, co-founder of Forestwalk Labs, on voice-in, visuals-out interactions as an emerging AI UX paradigm for LLM-powered products. He discusses why this approach is useful, what it takes to delight users, and lessons from building products with highly responsive AI agents, focusing on techniques for achieving low latency. Based on the title and description; speaker bio notes he runs the Infer AI engineering meetup and hosts the It Shipped That Way podcast. # Video Identity Observations: Allen Pike, Forestwalk Labs ## Speaker & Channel Self-Description - **Explicit self-introduction**: The speaker opens with *"I'm Alan Pike"* (transcript spelling; title says "Allen Pike") and identifies his company as **Forestwalk Labs** (*"what we've done at Forest Walk is we have built an agent that is in our calls"*). - **Stated affiliation/role evidence**: He references a **co-founder** (*"I mentioned to my co-founder in call on a call that I'd seen a bug"*), indicating he presents from a founder's perspective at a small company. - The video title labels this as a **talk/presentation** ("The Agony and the Ecstasy"), and the transcript confirms a live presentation format (*"today I'm going to be sharing some of what we've learned"*). ## Demonstrated Scope & Subject Matter - **Focus area — real-time AI interaction design**: The talk centers on **"voice in, visuals out"** AI experiences, latency engineering, and LLM inference optimization. - **Practitioner experience sharing**: The claimed scope is lessons learned from building, not theory — *"we've been building for that over the last few months and it's a lot of fun and we've learned a lot doing that."* - **Technical depth demonstrated**: The talk covers specific engineering topics with concrete claims: - Latency thresholds (100ms perceived instantaneity, ~1000ms attention limit, 200ms conversational limit) - Model selection tradeoffs (GPT-5 mini showing "5,000 milliseconds... sometimes 10,000 millisecond latency" vs. **"a Haiku class model"**) - **Prefix caching** (up to "90% cheaper, faster inference") - **Product demonstration (anecdotal)**: He describes a Forestwalk voice agent that filed a Linear issue during a co-founder call "within a second" — evidence the company builds real-time voice agents. ## Intended Audience - **AI builders and engineers**: The closing invitation — *"I would love to hear from anybody who has been exploring and experimenting, whether it's with real-time or any way that people are pushing the boundaries"* — directly addresses fellow experimenters. - **Call to build**: The sign-off, *"I hope some of this inspires some of you to go out and build something great,"* confirms a maker/developer audience. ## Presentation Style (Observable) - **Structured listicle core**: *"I want to finish up just by sharing three of the things that we found to be really really important"* — fast model, short inference intervals, stable caching. - **Cites external authorities**: References **Andrej Karpathy** and demoed architectures from **Thinking Machines and Neolab** as framing evidence, showing engagement with current AI discourse. - **Conversational, informal register**: Transcribed with fillers ("um," "you know"), suggesting an unscripted conference/demo-day style talk rather than polished produced content. ## Claims vs. Demonstrated Patterns - **Creator claim (about his company's product)**: That Forestwalk's voice agent responds within a second in real calls — supported only by the speaker's own anecdote, not shown live in the transcript. - **Demonstrated pattern**: Deep firsthand operational knowledge of model latencies and caching, consistent with an actively building team. # Delivery Observation: Allen Pike — "Voice In, Visuals Out" ## Overall Tone & Register - **Conversational, informal talk-track**: casual openers and connectors throughout — *“All right. I'm Alan Pike.”*, *“Here we go.”*, *“So this is Andrej Karpathy.”* - **Colloquial vocabulary** dominates: *“slow and dumb, which is like not a great combination”*, *“a lot of people are down on voices as an input”*, *“blown your budget”*, *“lean heavily into”*. - Rhetorical tag *“right?”* used repeatedly to check in with the audience: - *“The visuals outside is pretty intuitive, right?”* - *“…it keeps being like awkward and confused, right?”* - *“…we're off to something else mentally, right?”* ## Structural / Teaching Habits - **Explicit numbered structure**: telegraphs and executes a three-part list. - Setup: *“I want to finish up just by sharing three of the things that we've found…”* - *“So the first one is that in order to respond in a way that feels seamless, you have to use a really fast model.”* - *“…the second thing that's really critical… is to have short intervals of how quickly we're sending out for inference.”* - *“And then finally… you need to have a stable caching regimen.”* - **Concrete anecdote before abstractions**: a first-person story grounds the talk — *“the other day I mentioned to my co-founder in call on a call that I'd seen a bug with the Slack integration…”* - **Authority-by-reference**: cites named figures/products (Karpathy, Siri, ChatGPT voice, GPT-5 mini, Haiku, Thinking Machines, Linear, Slack) rather than abstract claims. - **Quantified claims**: recurring use of specific numbers — 100 ms, 1000 ms, 200 ms, P95 latencies of *“5,000 milliseconds, 7,000… sometimes 10,000 millisecond”*, 90% caching savings, 90/10 context split. ## Recurring Language Patterns - **The word “delightful/delight”** appears as a repeated value judgment: - *“We can create really delightful experiences with it.”* - *“making the experience feel really delightful”* - *“create that that delightful experience”* - **“Seamless”** as the recurring quality bar: - *“respond in a way that feels seamless”*, *“It feels much more seamless”*, *“feels like it has met… your attention span.”* - **First-person plural framing** for team work: *“what we've done at Forest Walk”*, *“we've been building for that over the last few months”*, *“these are some of the the techniques we've found really useful.”* - **Memorable coinage/labeling**: *“the tyranny of latency”*, *“a Haiku class model”*, *“stable caching regimen”*. - **Repeated “and so” as a transition device**, appearing frequently at paragraph starts: *“And so a lot of people are down on…”*, *“And so we're always in that play…”*, *“And so we've been building for that…”*. ## Filler & Disfluency Markers - Frequent **“um/uh”** and stutters, consistent with live, unscripted delivery: *“visualizations that come back… communicate the the responses”*, *“Um Haiku is much better um in terms of of that that P95 latency.”* - Trailing *“uh”* mid-list: *“maybe you take turns speech to text, uh and then…”* - Occasional self-correction: *“whether it's with real-time or any way that people are pushing the boundaries…”* follows a slightly tangled closing sentence. ## Closing Habits - Ends with **invitation to converse** plus encouragement, a stable two-beat pattern: - *“I would love to hear from anybody who um has been exploring and experimenting… I'd love to chat, share what we've been learning.”* - *“I hope some of this inspires some of you to go out and uh build something great. Thanks.”* ## Uncertainty Notes - Transcript is untimestamped beyond ordering; pacing, pauses, and vocal emphasis (e.g., the contrast between two intonations of *“okay”*) are implied by text but not directly observable here. - No curated metadata was supplied, so no additional delivery context beyond the transcript is available. # Position Record: "Voice In, Visuals Out: The Agony and the Ecstasy" — Allen Pike, Forestwalk Labs ## Framing Claim - **Speaker: Allen Pike** (self-identified at start) - References an argument made "last month" by **Andrej Karpathy**: voice is the human-preferred *input* for AI, but visuals are the preferred *output*. Pike notes Karpathy "knows a thing or two about this stuff." - Pike observes current practice diverges: "We've been typing to it. It's been typing back maybe with some markdown." - Claims recent breakthroughs make **audio-in / visuals-out** experiences feasible and delightful. ## Arguments ### Visuals Out (the intuitive half) - A third of the brain is dedicated to visual processing; "We love looking at things." - Models can now generate **rich HTML**, tool calling, interactive controls, and illustrations/images, lifting the ceiling on visual output. ### Voice In (the controversial half) - Most existing voice experiences are "slow and dumb" (e.g., Siri failing, awkward ChatGPT voice mode), so "a lot of people are down on voice as an input." - **Counter-argument**: speaking is the ultimate human communication channel — higher words-per-minute than typing, plus prosody carries meaning ("a huge difference" between different ways of saying "okay"). Important communication pushes people to calls or in-person speech for high bandwidth. - **Forestwalk example**: Pike mentions a Slack integration bug on a call; his co-founder confirmed seeing it; he said "let's file that as a Linear issue"; a voice agent in the call filed it within a second. He argues this feels natural when dialed in — the AI acts on intent without needing to respond in voice. ### Latency ("the tyranny of latency") - Known since the '60s: ~**100 ms** feels instant for computer reaction. - Up to **1000 ms** is the limit before people lose their train of thought. - Seamless voice-in/voice-out conversation requires **≤200 ms** — very hard given network, speech-to-text, and inference in the chain. - Notes **Thinking Machines and Neolab** demoed a thoughtful architecture time-slicing into 200 ms chunks with continuous inference. - **Pike's position**: we don't need novel architectures — voice-in/visuals-out benefits from a more forgiving visual response envelope (~1 second). ## Three Techniques (Forestwalk's lessons) 1. **Fast model + latency-prioritizing inference platform** - GPT-5 mini: despite being cheaper, they saw 5,000 ms latencies, 7,000–10,000 ms P95 — "we weren't consistently or even ever" fast enough. - **Haiku-class** (or small open-source) models are much better on P95 latency. - For heavier work, the fast model hands off asynchronously to a larger "thinking" model, then interleaves results. - Requires short context. 2. **Short inference intervals** - Traditional silence-detection wastes the latency budget; instead, send inference eagerly every **1–2 seconds** while the user is still speaking. 3. **Stable caching regimen** - Lean heavily on **prefix caching** (up to 90% cheaper/faster when the context prefix is identical). - Keep the first ~90% of the context window identical across requests; vary only the final 10%; minimize output tokens. ## Closing - Pike invites others experimenting with real-time or boundary-pushing model experiences to share learnings; hopes to inspire others to build. ## Qualifications / Unresolved Tensions - Pike labels the voice-input half of Karpathy's claim as "the more controversial half," while endorsing it conditionally via his bandwidth argument. - He hedges on always hitting 100 ms: "Sometimes we can achieve that, but with networking and everything it can be challenging depending on what work needs to get done." - Latency figures are stated as aspiration ranges, not guarantees. # Speaker & Provenance Observation Record **Video:** Voice In, Visuals Out: The Agony and the Ecstasy — Allen Pike, Forestwalk Labs **Video ID:** `01a15ca8-dc79-7e07-b2f4-881cf488a429` ## Speakers & Roles - **Allen Pike** — **Creator / presenter / sole speaker**. Self-introduces: *"I'm Alan Pike"* (spelling per title: Allen Pike). Speaks on behalf of **Forestwalk Labs**, where he and an unnamed **co-founder** (female, referenced in an anecdote) work. - **No narrator, guests, or other speakers** appear in the transcript. ## Referenced Third Parties & Claims - **Andrej Karpathy** — cited as arguing "last month" that voice is the human-preferred AI input and visuals the preferred output. - **Thinking Machines / "Neolab"** — cited as demoing a low-latency voice architecture "a few weeks ago" with 200 ms time-sliced inference. - **GPT-5 mini** — reported by the speaker as exhibiting 5,000–10,000 ms P95 latencies in their practice. - **Haiku** — described by speaker as better for P95 latency. - **Slack / Linear** — referenced in a Forestwalk Labs internal anecdote (bug report filed via voice agent). - **Siri, ChatGPT voice mode** — referenced as examples of unsatisfying voice interfaces (third-party claims by the speaker). ## Referenced Concepts (provenance noted, not evaluated) - **100 ms / 200 ms / 1000 ms latency thresholds** — speaker attributes the 100 ms figure to research "since the '60s"; no citation given. - **Prefix caching** — described as up to 90% cheaper/faster; speaker's claim. ## Promotional Segments - **Forestwalk Labs self-promotion**: the talk's framing ("what we've done at Forest Walk... built an agent") and the closing call-to-action ("I'd love to chat... build something great") promote the speaker's company and its work. ## Availability & Metadata Limits - **No curated metadata supplied** — no comment availability, publication details, or verified speaker identity beyond the self-introduction. - **Uncertainty**: speaker's name spelling varies between transcript ("Alan Pike") and title ("Allen Pike"); co-founder unnamed; no timestamps in transcript. All right. I'm Alan Pike. And today I'm going to be sharing some of what we've learned building voice in visuals out experiences using AI. Here we go. So this is Andrej Karpathy. And he made an argument last month that voice is the human preferred input for AIs. But that we prefer visuals as the output. And he knows a thing or two about this stuff. But that's not how we have been for the most part building or using AI. We've been typing to it. It's been typing back maybe with some markdown. But there's some real breakthroughs over the last few months where both of this visuals in and audio or the audio in and visuals out experience is now um really feasible. We can create really delightful experiences with it. The visuals outside is pretty intuitive, right? Of course a third of our brain is dedicated to processing visual information. We love looking at things. And models have recently got to the point where they can generate rich HTML, tool calling. So we can have these experiences where there's visualizations that come back, explain things, help us understand, communicate the the responses from these models. They can give us interactive um controls that allows to explore and understand and modify and change and and and direct the models. And they can even respond with beautiful illustrations and images, right? And so the the ceiling on the visuals out piece has really lifted and we have a lot more capability of what we can do in terms of responding from these models. The more controversial half of Karpathy's argument though is this idea of voice as the preferred input. We've long sort of idealized and fantasized about speaking to an AI and having this real-time conversation where it understands what we need and it and it reacts appropriately in real-time. But the experiences that most people have had so far with voice interfaces have been more like trying to get Siri to turn the lights on and it's not working or like this guy that is like trying to get ChatGPT voice mode to do things, but it keeps being like awkward and confused, right? The models that we have so far, the experiences that people most people have seen are both slow and dumb, which is like not a great combination. And so a lot of people are down on voices as an input. But speaking is the ultimate way that as humans we communicate. When we're speaking, we have more words per minute than when we're typing. But we also convey more with each word. There's a huge difference in between if you say to me okay versus if you say okay. And that's why when we have something that is really important that we need to communicate or get through to somebody, we will jump on a call. And we will or we'll speak in person. So that we can get through that high bandwidth uh communication. Um and so what we've done at Forest Walk is we have built an agent that is in our calls that can help us out in real time. And so the other day I mentioned to my co-founder in call on a call that I'd seen a bug with the Slack integration or at least I thought I had. And she responded that she had seen the same bug. And so I just said, "Okay, well, let's file that as a Linear issue." And the voice agent within a second responded that it had done so. And that feels perfectly natural when you get it really dialed in. That you have are speaking either incidentally or you know, with a purpose intentionally speaking to the AI and it responds in a way that is not interruptive of you. It doesn't need to be voice. It is just taking action on your intent. And so this is we're going to see this experience more and more in the coming months and years. But there's a huge barrier, a huge challenge to making this actually work and feel good. And that is the tyranny of latency. It's really difficult to get a response through that whole chain fast enough. We we've known since the '60s that to have a computer react to us fast enough that it feels instant, it needs to react in about 100 milliseconds, a tenth of a second. And so of course we are always aspiring to get our products to react that fast. And sometimes we can achieve that, but with networking and everything it can be challenging depending on what work needs to get done. And so sometimes we might flex up to a thousand milliseconds. Right? You get a full second. It's about the limit before people start to lose their train of thought. They ask Siri to do something, takes more than a second, we're off to something else mentally, right? And so we're always in that play in between those two kind of human limits of trying to respond within a second, ideally within 100 milliseconds. But if we want to have a seamless voice conversation with it an AI or another human, the limit is much more aggressive. We need to have a 200 millisecond latency or less if we want to have a fully conversive people are verbalizing something and they are interrupting or interjecting or agreeing and forming that sort of connection for a full voice in voice out uh conversation. Um and so imagine trying to do that where you have a network request, uh maybe you take turns speech to text, uh and then you do some sort of model inference on that text, and then send it back on the network. It's like a ridiculous uh amount of of work to do in 200 milliseconds. Um there are some clever approaches of trying to work around that. Uh Thinking Machines and Neolab has uh a really thoughtful architecture where they were demoing recently, um just a few weeks ago, a they've time sliced into 200 millisecond chunks like that 200 millisecond goal, and and has a model that's doing continuous inference in and out at 200 millisecond slices. Um so there are ways around it for voice in voice out, but there's also just we don't need to wait for novel architectures. We can just switch to having voice in visuals out and then we benefit from the more forgiving visual response envelope that people have. That if you have something that appears on screen within a second of what you've you've said, it feels like it has met that it's keeping with within your attention span. It's reacting to you in a way that feels seamless. Um and so we've been building for that over the last few months and it's a lot of fun and we've learned a lot doing that. And so I want to finish up just by sharing three of the things that we found um to be really important for getting within that latency envelope and making the experience feel really delightful um for for people who are using it. So the first one is that in order to respond in a way that feels seamless, you have to use a really fast model. Cuz that's not just a model that is so small um that it can respond fast enough in a few hundred milliseconds. Of course it it does need to be. Um but it also has to be on an inference platform that prioritizes latency. So when GPT-5 mini came out, we were excited that um that okay, well, yeah, we get this more intelligent model and presumably it'll be fast. But then in practice we were seeing latencies of 5,000 milliseconds, 7,000 P95, sometimes 10,000 millisecond latency for this small model. It was cheaper than the bigger model, but we weren't consistently or even ever seeing it respond fast enough. Um Haiku is much better um in terms of of that that P95 latency. So, you you really need a a kind of a Haiku class model as the one that's responding in this real-time and and or one of the smaller open-source ones. And then if there is a larger chunk of work that needs to get done, then that model then hands off or sends off an asynchronous message to a larger model that can think, and then the continuous real-time model or soft real-time model that's responding quickly then can re- interleave in those responses if it's doing something heavier. Um so, that's number one you need to have a fast model. Obviously, with a a short enough context that you're providing it that it can respond in a few hundred milliseconds. Um the second thing that's really critical if we're going to have this feeling of it being really responsive is to have short intervals of how quickly we're sending out for inference. So, traditionally for like a voice application, you might listen to the user for a few seconds and then they stop, you listen for a second of silence, and then you have uh some sort of inference goes on, and then you're you're blown your budget by a pretty wide margin just for waiting for silence. So, if we're going to have our something up on screen within a second, um then when we want to have our inference pretty eagerly responding as the person is talking, even if we're not entirely sure they've stopped talking, being willing to send inference every 1 or 2 seconds as they speak cuz you know, we sometimes have a sentence where, you know, we ask, "Oh, hey, we're going to want to change this and actually also list to this other thing." It feels much more seamless if you're doing those things as people talk. Um so, you have to have a a model and an an infrastructure where you're able to get these fast turns going. And then finally, in order to actually make all of that work, you need to have a stable caching regimen. So, we have this huge improvement in the way that we work with LLMs over the last year where we have on the different platforms different ideas of this prefix caching where if the beginning of the context you send to the model is the same each time, then you can get up to 90% cheaper, faster inference um depending on the conditions. And so, you need to lean heavily into this architecture really I think for for most applications we're moving towards this, whether it's like a long-running agent or it's a frequently running agent, the same principle applies that we want to the first 90% of the context window if we can to be the same from request to request, and then just use that final 10%, and then also, of course, minimize the number of output tokens so that you can get these really fast and relatively affordable inference turns to create that that delightful experience. Um So, these are some of the the techniques we've found really useful. Um I would love to hear from anybody who um has been exploring and experimenting, whether it's with real-time or any way that people are pushing the boundaries of creating delightful experiences uh with these models. Uh I'd love to chat, share what we've been learning. Uh and I hope some of this inspires some of you to go out and uh build something great. Thanks.