--- schema: 1 type: video title: 'Your Voice Agent is Just a Walkie Talkie — Neil Zeghidour, Gradium' description: >- On a phone call with someone close to you, as much as 20 percent of the time you are both talking at once. Every real time voice model shipping today is half duplex: it is either listening or speaking, never both. That gap is the whole argument of this talk. Neil Zeghidour is co founder and CEO of Gradium, a Paris company that trains audio foundation models, and he traces the arc from a 2011 phone assistant that mapped transcripts to app actions, through the first open ended voice chat that could converse but take no action, to cascaded voice agents that call tools competently while inheriting the latency and flattened emotion of routing through text. Speech to speech collapsed that pipeline and fixed latency, but he demonstrates the remaining problem live by back channeling at a model, saying mm hmm and yeah the way people do, and watching it stop dead every time. The technical core explains why audio was hard to put in a language model at all. Eight words take about three seconds to say, which at 24 kHz is 72,000 timesteps, and since attention cost grows with the square of sequence length, a sequence thousands of times longer is astronomically more expensive. Neural codecs solve it by compressing audio into token like representations a model can learn on. Full duplex then needs a further step: modeling two token streams at once so both sides can be active, silent, or overlapping. Zeghidour is candid that every gain in naturalness so far has cost intelligence, because a fixed weight budget spent on hearing and speaking is taken from reasoning, and he lays out the two ways forward. Speaker info: - https://x.com/neilzegh - https://gradium.ai Timestamps: 0:00 - Gradium and a research lab built around voice 1:55 - A 2011 assistant, and the pipeline behind it 3:39 - Open ended conversation, zero agency 5:18 - A drive through agent that takes real actions 7:48 - Where speech to speech still falls short 8:37 - Back channeling breaks turn taking 10:20 - Why raw audio will not fit in a language model 11:57 - Two streams instead of one 13:37 - The tension between naturalness and intelligence 15:18 - Path one: scale the speech to speech model 17:00 - Path two: split interface from intelligence --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/a8EcVumh71E-your-voice-agent-is-just-walkie-talkie Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/your-voice-agent-is-just-a-walkie-talkie-neil-zeghi-684c9f50094d markdown: >- https://liquidgold.cc/channel/aidotengineer/video/your-voice-agent-is-just-a-walkie-talkie-neil-zeghi-684c9f50094d.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T03:41:22.534Z' language: null children: [] videoId: a8EcVumh71E durationSeconds: 1154 publishedAt: '2026-09-15T14:00:39.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 12 endSeconds: 24 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=12' heldReason: rights - ordinal: 1 text: null startSeconds: 24 endSeconds: 126 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=24' heldReason: rights - ordinal: 2 text: null startSeconds: 138 endSeconds: 223 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=138' heldReason: rights - ordinal: 3 text: null startSeconds: 234 endSeconds: 324 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=234' heldReason: rights - ordinal: 4 text: null startSeconds: 334 endSeconds: 468 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=334' heldReason: rights - ordinal: 5 text: null startSeconds: 334 endSeconds: 468 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=334' heldReason: rights - ordinal: 6 text: null startSeconds: 480 endSeconds: 524 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=480' heldReason: rights - ordinal: 7 text: null startSeconds: 480 endSeconds: 524 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=480' heldReason: rights - ordinal: 8 text: null startSeconds: 534 endSeconds: 624 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=534' heldReason: rights - ordinal: 9 text: null startSeconds: 624 endSeconds: 635 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=624' heldReason: rights - ordinal: 10 text: null startSeconds: 635 endSeconds: 724 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=635' heldReason: rights - ordinal: 11 text: null startSeconds: 724 endSeconds: 734 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=724' heldReason: rights - ordinal: 12 text: null startSeconds: 734 endSeconds: 826 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=734' heldReason: rights - ordinal: 13 text: null startSeconds: 837 endSeconds: 926 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=837' heldReason: rights - ordinal: 14 text: null startSeconds: 837 endSeconds: 926 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=837' heldReason: rights - ordinal: 15 text: null startSeconds: 936 endSeconds: 1021 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=936' heldReason: rights - ordinal: 16 text: null startSeconds: 1032 endSeconds: 1154 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=1032' heldReason: rights - ordinal: 17 text: null startSeconds: 1032 endSeconds: 1154 watch: 'https://liquidgold.cc/watch/a8EcVumh71E?t=1032' heldReason: rights --- # Your Voice Agent is Just a Walkie Talkie — Neil Zeghidour, Gradium On a phone call with someone close to you, as much as 20 percent of the time you are both talking at once. Every real time voice model shipping today is half duplex: it is either listening or speaking, never both. That gap is the whole argument of this talk. Neil Zeghidour is co founder and CEO of Gradium, a Paris company that trains audio foundation models, and he traces the arc from a 2011 phone assistant that mapped transcripts to app actions, through the first open ended voice chat that could converse but take no action, to cascaded voice agents that call tools competently while inheriting the latency and flattened emotion of routing through text. Speech to speech collapsed that pipeline and fixed latency, but he demonstrates the remaining problem live by back channeling at a model, saying mm hmm and yeah the way people do, and watching it stop dead every time. The technical core explains why audio was hard to put in a language model at all. Eight words take about three seconds to say, which at 24 kHz is 72,000 timesteps, and since attention cost grows with the square of sequence length, a sequence thousands of times longer is astronomically more expensive. Neural codecs solve it by compressing audio into token like representations a model can learn on. Full duplex then needs a further step: modeling two token streams at once so both sides can be active, silent, or overlapping. Zeghidour is candid that every gain in naturalness so far has cost intelligence, because a fixed weight budget spent on hearing and speaking is taken from reasoning, and he lays out the two ways forward. Speaker info: - https://x.com/neilzegh - https://gradium.ai Timestamps: 0:00 - Gradium and a research lab built around voice 1:55 - A 2011 assistant, and the pipeline behind it 3:39 - Open ended conversation, zero agency 5:18 - A drive through agent that takes real actions 7:48 - Where speech to speech still falls short 8:37 - Back channeling breaks turn taking 10:20 - Why raw audio will not fit in a language model 11:57 - Two streams instead of one 13:37 - The tension between naturalness and intelligence 15:18 - Path one: scale the speech to speech model 17:00 - Path two: split interface from intelligence --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/a8EcVumh71E-your-voice-agent-is-just-walkie-talkie Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/your-voice-agent-is-just-a-walkie-talkie-neil-zeghi-684c9f50094d · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.