Conference talk by Matt Lawler, forward deployed engineer lead at AssemblyAI, on building Joey, an internal AI support agent built on the Claude Agent SDK with local markdown docs, embeddings, agentic file search, and a single CLAUDE.md file. The speaker reports that Joey replaced an off-the-shelf support bot that resolved 10% of conversations, and that Joey resolves about 80% end to end for roughly $700 a month, deployed on Railway. The talk also covers why the forward deployed engineer model does not scale at 1,000 API signups a day, turning escalations into roadmap items, and adding real-time voice with AssemblyAI's Voice Agent API, including a live demo. These figures are the speaker's reported results and are not independently verified. # Channel Presentation Observations: AssemblyAI Conference Talk ## Source Basis - **Video ID:** `01a15ca6-f4e7-780b-a22f-2d3f6f0b6c81` - **Title:** "We Built an AI Support Agent That Resolves 80% of Tickets — AssemblyAI" - **Transcript:** Supplied and used as the primary evidence. - **Curated metadata:** None supplied. Identity context comes only from the title and the transcript. --- ## Explicit Self-Description (Creator Claims) The speaker describes himself and the company directly. These are **claims made by the creator**, not independently verified facts. - **Speaker role:** > "my name is Matt Lawler. I'm a forward deployed engineer at Assembly AI. Uh mostly focused on bringing in inbound customers" - **Company category:** > "we provide voice AI infrastructure. Um, so we train our own foundation models for speech to text primarily." - **Stated scope of offering:** > "you can use us for the entire stack, not just speech to text. Um, so you can use our voice agent API and we'll string together the speech to text, the LLM, and the text to speech as well." - **Stated purpose of the talk:** > "I wanted to make sure I I gave you guys an overview of kind of some of the go to market engineering stuff that we've been doing at Assembly AI that maybe you can apply to your own FTE team" - **Stated scale claim:** > "we see around a thousand API signups every single day." --- ## Observable Presentation (Demonstrated in the Material) These observations describe how the material presents itself, not the creator's self-description. ### Format and Delivery - **Conference talk format.** The transcript opens with microphone and audio checks, and includes audience interaction ("how many other like forward deployed engineers are out in the audience right now?") and closing applause. - **Conversational, first-person narrative.** The talk is built around the speaker's own experience, for example: "when I first joined assembly on our support team." - **Live demonstration.** The speaker runs a voice conversation with the product, and says he is doing so locally to avoid "add[ing] to the queue for our actual support team." - **Frequent verbal fillers and restarts**, visible throughout the transcript (e.g., "Um," "I I," "uh"). These reflect spoken delivery rather than scripted polish. ### Structure of Argument The talk follows a recognizable problem, attempt, result sequence: 1. **Problem:** Inbound volume ("a thousand API signups every single day") exceeds human capacity. 2. **Failed attempt:** An off-the-shelf support bot resolved "about 10%" of conversations. 3. **Build:** An in-house agent, named Joey, built on the Claude Agent SDK, with docs in Markdown, Voyage embeddings, and deployment on Railway. 4. **Result claim:** "we went from 10% to 80% endto-end resolution rate in just the first week." 5. **Extension:** A voice interface built on the company's own voice agent API. ### Evidence Style for Quantitative Claims Headline figures are stated by the speaker without shown data in the transcript: - 10% → 80% resolution rate - Around $700 per month in token and infrastructure costs - About 20% of tickets escalated to humans The transcript offers verbal framing for the 80% figure ("no human in the loop," "not the case here" regarding subset-only resolution) but no external measurement method. ### Promotional Elements The material contains product promotion and calls to action, including: > "if you want to know a little bit of like how to build your own Joey... " > "go build go build something... especially if it's a voice agent, build it on Assembly AI." > "our booth is just right over here." The speaker also invites feedback and says a phone number for Joey may follow "later this week." --- ## Intended Audience (Inferred From Content) This section records who the material is addressed to, based on the content and direct address. It does not infer private motives. - **Forward deployed engineers (FTEs)** and teams that already have them, or want to build them. The speaker directly addresses this group: "for anyone who's an FTE in here, if you're looking to become an FTE..." - **Conference attendees** building AI support or voice agents. - **Developers and product teams building voice agents**, who are the stated customer base: "our customers build voice agents." --- ## Claimed Scope The creator frames the talk's scope as: - **Applicable beyond one company:** "maybe you can apply to your own FTE team or maybe you can apply to your company" - **A general principle:** "try to automate yourself out of a job. I think that should be your role every single day." - **A product-specific lesson:** "dog fooding your own product... any new feature that you launch... you need to be teaching your customers" The talk makes a case for these practices. It does not present a cross-company study or benchmark. --- ## Limits of This Observation - **Single video only.** No other videos were supplied, so nothing here supports channel-wide conclusions about content, consistency, or audience across the channel. - **Identity context is minimal.** The only channel-level signal is the title suffix "— AssemblyAI." - **Performance claims are unverified** within the material. They are recorded as the speaker's statements. - **Transcript quality.** The transcript contains transcription artifacts (e.g., "assign a BAA," "endto-end"). Quotations above preserve the transcript text as written. # Delivery Observation: "We Built an AI Support Agent That Resolves 80% of Tickets" — AssemblyAI ## Overall Delivery Style The speaker (Matt Lawler) delivers a conversational, talk-track-heavy conference presentation. The delivery reads as informal and audience-facing rather than scripted, with frequent verbal warm-ups, rhetorical check-ins, and a live demo segment mid-talk. --- ## Tone - **Casual and colloquial.** Opens with unscripted warm-up chatter: > "All righty. Can everyone hear me? Okay. All All levels coming through good. All right. Sweet. Well, uh, how's everyone doing?" - **Enthusiastic and promotional at key moments**, especially around results and product claims: > "demand is absolutely insane um, for voice AI right now" > "And the results were pretty remarkable from us building this." - **Personable toward the product itself**, treating the bot as a teammate rather than a tool: > "Isn't he so nice? [laughter] I enjoy talking to Joey." --- ## Vocabulary and Word Choice - **"Joey" as a persistent named character.** The bot is referred to by name throughout — dozens of mentions — rather than as "the bot" or "our agent." This is the single most distinctive lexical choice in the talk. - **Repeated use of "um" and doubled/restarted phrases** ("to to listen," "the at the booth," "we we shipped," "80 80%"), consistent with unscripted live delivery. - **Startup/engineering jargon used without definition:** "forward deployed engineer," "dog food(ing)," "end-to-end resolution rate," "barge/inerruption handling," "inbound," "ship/ed," "iterate." - **Informal intensifiers and hedges:** "pretty incredible," "pretty naive implementation," "quite/pretty," "kind of," "like" as filler throughout. - **Direct second-person address:** "you guys," "anyone here who's an FTE," "you can go on the site and you test him out afterwards." --- ## Recurring Phrases and Framing Devices - **"Build your own Joey"** — the central repeated call to action, appearing at least three times: > "I wanted to encourage everyone who's here to try and build your own Joey" > "it's not just that you should build your own Joey" > "go build whether it's a your own Joey" - **"A thousand API signups every single day"** — repeated as a scaling justification at least three separate times ([127], [149], [202]). - **"Automate yourself out of a job"** — a memorable repeated thesis for FTEs: > "try to automate yourself out of a job. I think that should be your role every single day." - **"Dog food(ing)"** — recurring framing for building with one's own product: > "we wanted to dog food our own product" > "you need to be dog fooding your own product all the time" --- ## Teaching and Explanation Habits - **Concrete numbers as evidence.** Almost every claim is anchored to a figure: 10% → 80% resolution, ~1,000 signups/day, 900 tickets/day remaining, $700/month cost, 30-second deploys, ~30,000-line Claude MD. - **Problem-before-solution structure.** Spends significant time on the failed off-the-shelf bot (10% resolution, no access to prompt/tools/RAG) before introducing the custom build. - **Enumeration of features in list form**, e.g., Joey's capabilities: > "he can manage his own infrastructure. He has a file system. He can write and debug code for you. He can call tools. He can give himself new abilities." - **Anticipatory objection handling.** Preempts skepticism about metrics: > "some people like to game these metrics. They'll say like, oh, we resolved 80% but only on like a subset of types of tickets. Not the case here." - **Live demo with transparency about staging.** Deliberately simulates locally rather than hitting production: > "I don't want to add to the queue for our actual support team." - **Teaching via analogy to the audience's own role**, repeatedly addressing the "FTE in the audience" directly with actionable advice. --- ## Communication Patterns - **Rhetorical audience check-ins** open the talk and reappear mid-talk: > "how many other like forward deployed engineers are out in the audience right now?" - **Meta-layer framing** — pointing out self-referential aspects of the demo: > "if you're talking to Joy about how to build a voice agent API, you're already using it." - **Mock dialogue with the product**, playing both a customer and support recipient in the demo, then commentating on it afterward. - **Closing with direct invitations** — booth location, feedback request, and a repeated "go build" imperative as the final note. --- ## Stability Assessment **Likely stable patterns** (supported by multiple instances): - Naming and personifying the agent ("Joey"), the "build your own Joey" refrain, repeated 1,000-signups statistic, number-heavy evidence style, dog-fooding thesis, and informal filler-heavy delivery. **Isolated moments** (insufficient repetition to generalize): - The laughter/light banter after the demo ("Isn't he so nice?") appears only once. - The empathy claim after building with the API ("I have a lot better empathy...") is stated once; whether it reflects a habitual teaching move cannot be confirmed from this transcript alone. - The opening audience sound-check patter is a single event and may be venue-specific. # Observation Record: "We Built an AI Support Agent That Resolves 80% of Tickets" **Video ID:** 01a15ca6-f4e7-780b-a22f-2d3f6f0b6c81 **Speaker:** Matt Lawler, self-identified as a forward deployed engineer at AssemblyAI (transcript: "my name is Matt Lawler. I'm a forward deployed engineer at Assembly AI") **Format:** Conference talk with a live, locally simulated voice demo **Source basis:** Transcript only. No curated metadata beyond the title and speaker self-identification. --- ## 1. Context and Problem Statement ### 1.1 Company Description (Speaker's Account) - AssemblyAI provides **voice AI infrastructure**, primarily speech-to-text foundation models, and is "starting to branch into other parts of the voice agent space." - The speaker states it offers a voice agent API that combines speech-to-text, an LLM, and text-to-speech. - He says the company sees **"around a thousand API signups every single day."** ### 1.2 Stated Problem - The speaker says he was **"the only uh onboarding engineer" until "at least until like two days ago."** - He states that handling a thousand daily signups through one-to-one engagement is **"just not feasible."** - He argues that the forward deployed engineer (FDE) model **"doesn't really scale super well."** ### 1.3 Speaker's Recommendation to FDEs - He encourages FDEs to **"automate yourself out of a job,"** which he calls "your role every single day." - He states: **"you can't be the bottleneck to having a good customer experience."** --- ## 2. Failed Prior Approach - The team first bought an **off-the-shelf support bot** pointed at the docs. - **Reported result:** it resolved **"about 10%"** of conversations. - The speaker characterizes this as "helped," but says a 10% end-to-end resolution rate "wasn't good enough." - **Stated limitations of the vendor product:** - No access to the system prompt - No access to tools - No access to the RAG infrastructure - When changes were needed, the vendor said **"it's on the road map,"** which the speaker says "didn't quite work for us." --- ## 3. The Replacement Agent: "Joey" ### 3.1 Capabilities (as described) - Acts as the **first line of defense** for all inbound contact: chat widget, contact sales, and support alias. - **Remembers past conversations** and tracks metrics through Pylon. - Built on the **Claude Agent SDK**. - Can manage its own infrastructure, has a file system, writes and debugs code, calls tools, and "can give himself new abilities." - The speaker says these abilities let Joey "operate more like an FTE and less like just a standard chatbot." ### 3.2 Architecture (four parts, per speaker) | Component | Speaker's Description | |---|---| | **Local documentation** | Docs, changelog, pricing, and website pages converted to Markdown and checked out locally. Updates sync automatically. Source URLs are available for citation. | | **Retrieval** | Uses **Voyage** embeddings to surface relevant documents up front. Also permits **agentic search** of the local file system for multi-resource answers. | | **CLAUDE.md instructions** | Described as "a 30,000 line Claude MD" containing guardrails and operating advice. Updated when a bad interaction or wrong answer is observed. | | **Deployment** | Containerized via Dockerfile and deployed on **Railway**. The speaker says this avoids managing EC2 instances. | ### 3.3 Iteration Claims - The speaker states a PR can be written and deployed so that **"within 30 seconds there's a new live version of Joey."** - He describes a case where the team **"monitored a live conversation, caught a bug, deployed a fix,"** and the fix applied for the rest of the session without the customer noticing. - He says Joey "is able to basically get updated whenever we want" and that the team has "full control over everything that that he does." --- ## 4. Reported Results > "we went from 10% to 80% endto-end resolution rate in just the first week of deploying this this build with a pretty naive implementation." - **Reported cost:** about **$700 per month** for tokens, inference, and infrastructure combined. - **Scope claim:** the speaker says the 80% figure is not limited to a subset of ticket types. He states that customers **"cannot reach a human directly unless you go through Joey and you tell him to escalate."** - **Escalation rate:** Joey escalates about **20%** of tickets to humans, "for the right reasons," such as: - Rate changes - Data opt-outs - Agreements requiring signature - Matters requiring FTE or legal involvement - The speaker presents escalations as a **roadmap** of what still needs automation. - He gives examples of automation in progress: - A BAA link added as a reference resource - Pricing negotiation, where Joey quotes a rate the user can negotiate against ### Caveats Noted - The speaker describes the implementation as **"pretty naive."** - The metrics are **self-reported** in a conference setting. No independent verification, definitions of "resolution," or measurement methodology are provided in the transcript. - The speaker notes the rollout is recent ("just shipped this week" for voice). --- ## 5. Voice Extension - The team added the **AssemblyAI voice agent API** to Joey "this week." - The speaker describes the voice pipeline as **"all speech in, speech out, one websocket connection,"** combining STT, LLM, and TTS, and handling pauses, interruptions, and barge-in. - He states the tools are the same as the text version. - **Planned, not yet confirmed:** a phone number "hopefully later this week." ### Stated Rationale > "the best way to understand what your customers are building is to actually build their same product yourself." - The speaker reports that building the voice version exposed him to **latency**, **turn-taking**, and **interruption handling** challenges, which he says gave him "a lot better empathy" for customers. --- ## 6. Demonstration - The demo was **simulated locally** by the speaker, who said he did this to avoid adding to the live support queue. - **Transcript of demo exchange (as spoken):** > **Joey:** "Hi, this is Joey at Assembly AI. How can I help you today?" > > **User:** "Hey Joey, I'm looking to build an ambient medical scribe with Assembly AI, but before I can test, I need to assign a BAA. Is that something that you can do?" > > **Joey:** "One sec. Let me check that for you. Yes, you can execute the BAA yourself right away. Just complete our standard HIPPA business associate agreement, but note that you'll need a paid account with a card on file first. And signing the BAA automatically opts you out of model training. The relevant docs are linked in the chat." - **Speaker's comparison:** two years earlier, this type of question would have required a human, and a non-immediate canned response would have been the alternative. - **Note:** the demo is a single scripted exchange. The transcript does not show any failure cases. --- ## 7. Speaker's Generalized Takeaways - **Dogfood your own product.** The speaker says FTEs should use the product themselves: *"It's not enough to just be teaching your customers how to get started with it."* - **Build what customers build.** He says the team should build "the exact same thing" customers are building, rather than waiting for requests. - **Use the stack.** He recommends building voice agents on AssemblyAI specifically. - **Self-automation.** FTEs should view automating their own work as their role. --- ## 8. Tensions and Unresolved Items | Item | Detail | |---|---| | **"Resolves 80%" vs. "escalates 20%"** | Consistent in the transcript, but "resolution" is not operationally defined. | | **"Handles 100% of inbound tickets"** | Stated alongside the 80% resolution figure. The speaker distinguishes these as handling versus resolving. | | **Scalability framing** | The speaker says FTE hiring "doesn't scale." He also says the goal is to keep humans on "led deals" and "more important stuff," without quantifying this. | | **Phone line** | Described as a planned next step, with no confirmed status. | | **Naive implementation** | The speaker acknowledges the implementation is naive but does not discuss its limitations, failure modes, or long-term maintenance burden. | | **Cost** | Reported as roughly $700/month, with no breakdown between tokens, inference, and infrastructure. | | **Maintenance** | A 30,000-line CLAUDE.md is described as updated after bad interactions. The transcript does not discuss how errors are detected at scale, beyond Slack review and live monitoring. | --- ## 9. Verification Notes - All claims above are attributed to **Matt Lawler** as the speaker in this transcript. - Quantitative claims (1,000 signups/day, 10% to 80% resolution, $700/month, 20% escalation, 30-second deploys, 30,000-line CLAUDE.md) are **speaker-reported** and were not independently verified in the source. - The speaker's employment with AssemblyAI is self-identified and is reflected here as a stated affiliation. # Attribution Observation: "We Built an AI Support Agent That Resolves 80% of Tickets — AssemblyAI" ## 1. Source Inventory - **Video ID:** 01a15ca6-f4e7-780b-a22f-2d3f6f0b6c81 - **Title (as supplied):** "We Built an AI Support Agent That Resolves 80% of Tickets — AssemblyAI" - **Available channels:** Title and transcript only. No curated metadata on speakers, cited sources, or comments was supplied. - **Comment availability:** Not determinable from the supplied material. ## 2. Speakers and Roles ### 2.1 Primary Speaker: Matt Lawler - **Self-identified role:** Forward deployed engineer at Assembly AI. - **Stated focus:** Inbound customer onboarding and "go to market engineering" work. - **Speaking mode:** Conference talk delivered live to an audience, with some direct audience address. - **Basis:** Self-introduction in the transcript. No independent verification is available. ### 2.2 "Joey" (AI Support Agent, Speaks Within the Demo) - **Described role:** An AI support agent built by Assembly AI, presented as "another member of our team." Described as the first line of defense for inbound contacts via chat widget, contact sales, and the support alias. - **Speech in transcript:** Joey's replies appear in a simulated, locally run voice demo. Lines are marked with `>>` but not labeled by speaker. - **Note:** Joey is a software agent. Statements attributed to him are generated outputs in the demo, not statements by a human speaker. ### 2.3 Unidentified Demo Caller - **Role:** Hypothetical or simulated customer asking about a BAA for an ambient medical scribe use case. - **Identity:** The transcript does not label this speaker. The presenter states he is simulating the interaction locally, so the caller's identity (presenter or other) is **not explicitly stated** and should not be assumed. ### 2.4 Audience - Audience reactions are recorded only as **[music]**, **[laughter]**, and **[applause]**. Individual audience members are not identified. ## 3. Promotional Segments (Flagged) The talk is **promotional in substance**. It is presented by an Assembly AI employee and centers on the company's products. The following segments are explicitly flagged: - **Company product overview:** Description of Assembly AI as providing "voice AI infrastructure," including speech-to-text models and a voice agent API. - **Product self-recommendation:** Direct calls to "build on Assembly AI" and "build your own Joey." - **Voice agent API demonstration:** Live simulated demo of the company's own voice agent API, including claims such as "we bring our own voices, so you don't have to find another provider." - **Forward-looking product announcement:** Stated plan to "hook them up to a phone number" with an "Assembly AI phone number." - **Call to action:** Invitation to visit the booth and provide feedback. ## 4. Referenced Sources and Third Parties ### 4.1 Named Organizations and Tools | Name | Context in Transcript | |---|---| | Assembly AI | Presenter's employer and the subject of the talk | | Fireflies | Cited as an example of a note-taker that uses Assembly AI transcripts | | Pylon | Support tooling used to track metrics and manage conversations | | "Cloud agent SDK" | Foundation for Joey's agent capabilities (transcribed as "cloud agent SDK"; the transcript also says "Claude" in other places, so the exact product name is ambiguous) | | Voyage | Embeddings partner for retrieval | | Railway | Deployment platform | | Slack | Used to surface bad conversations for review | | EC2 | Mentioned as an alternative infrastructure option that was not used | | Docker | Packaging for the deployment | ### 4.2 Unnamed Third-Party References - **Off-the-shelf support bot vendor:** Used earlier and described as resolving about 10% of conversations. The vendor is not named. Presenter states the vendor replied "it's on the road map" regarding changes. - **Competitors:** Referenced generally ("as our competitors know") without naming them. ## 5. Provenance of Claims All quantitative and comparative claims below are **speaker-reported and unverified** in the supplied material. They are recorded as attributed statements only. ### 5.1 Scale and Context Claims - Approximately 1,000 API signups per day (presenter's statement). - Presenter was "the only onboarding engineer" until "about two days ago." - Customers building voice agents on Assembly AI models, including phone support and drive-through ordering (presenter's statement). ### 5.2 Performance and Cost Claims - End-to-end resolution rate rose from about **10%** (prior vendor bot) to about **80%** in the first week of the new build (presenter's statement). - Joey handles **100%** of inbound tickets and escalates about **20%** to humans (presenter's statement). - Operating cost of roughly **$700 per month** for tokens, inference, and infrastructure (presenter's statement). - Deploy time of about **30 seconds** on Railway (presenter's statement). ### 5.3 Demo Claims - The demo was **simulated locally** rather than run against the live support queue, per the presenter's statement. - The demo's Joey responses are generated outputs and are not independently checked in the transcript. ## 6. Limits of the Record - **Unlabeled speaker turns:** Demo dialogue is marked with `>>` but not attributed to individual speakers. - **Transcription ambiguity:** Some proper nouns appear inconsistently (for example, "Joy" for Joey, "cloud" for Claude, and "HIPPA" for HIPAA). - **Unverified metrics:** All performance, cost, and volume figures are presenter-reported and lack supporting documentation within the supplied material. - **Hypothetical framing:** Some statements are explicitly hypothetical (for example, "if our entire doc site was down right now"). These are not factual claims about events. - **No external corroboration:** No metadata, cited documents, or comments were supplied to corroborate any claim. Cool. \[music\] All righty. Can everyone hear me? Okay. All All levels coming through good. All right. Sweet. Well, uh, how's everyone doing? How's everyone doing? Better. Okay, there we go. Cool. Well, I hope uh the conference has been good for everybody. Appreciate you all being here to to listen to a chat from uh from me. Um for anyone who hasn't met me at the at the booth yet, uh my name is Matt Lawler. I'm a forward deployed engineer at Assembly AI. Uh mostly focused on bringing in inbound customers, um helping win their business, u making sure that they have a good time building with our API. Um so, how how many other like forward deployed engineers are out in the audience right now? Any of us? Okay, a couple. Cool. Well, um, still coming through. Okay. All right. Um, yeah. So, I wanted to make sure I I gave you guys an overview of kind of some of the go to market engineering stuff that we've been doing at Assembly AI that maybe you can apply to your own FTE team or maybe you can apply to your company um, as something that you can do as an easy win for your customers. Um, so for a little bit of overview of like assembly AI in case you haven't heard of us, um, we provide voice AI infrastructure. Um, so we train our own foundation models for speech to text primarily. Um, but we're starting to branch into other parts of the voice agent space. Um, so if anyone's ever had like a Fireflies note-taker join your meeting or something like that, you've also you've already seen a transcript from Assembly AI. Um, so we train our own foundational models not only for like async use cases like meeting recordings and things like that, but also for real time uh real-time like voice agents. Um, so if you've ever used like a a phone customer support agent when you called in somewhere and it wasn't a human or like drive-through ordering, they're more than likely building with Assembly AI's models. Um, and for anyone who's building a voice agent now, you can use us for the entire stack, not just speech to text. Um, so you can use our voice agent API and we'll string together the speech to text, the LLM, and the text to speech as well. Um, as our competitors know, as we know, as anyone in the voice AI space knows, demand is absolutely insane um, for voice AI right now. Um, so assembly at our current moment, we see around a thousand API signups every single day. Um, which is somewhat of the theme of why we were focusing on automation and wanting to build kind of a new support bot for our team because at least until like two days ago, I was the only uh onboarding engineer at Assembly AI. And so you can imagine, as much as I'd want to talk to a thousand new signups every single day, that's just not feasible. Um, so we had to build something to to solve that problem. And then for anyone here who doesn't know like what a forward deployed engineer is, it's a very catch-all title. It varies a lot at every different company, but effectively we want to get deployed with our end customers. We want to understand their use case. We want to make sure that we are building exactly what they need. Um, so we frequently commit code directly to their repositories. We usually meet with them several times a month, if not a week. Um, and we really try to understand their use case and get to know them very personally. Um so like at least on the inbound side for anyone who comes into assembly who wants to build with us we are the people um that are their main point of contact for anything they need on the technical commercial side anything that they might need they get from us their FTE um and we're responsible for winning their business um and building that trust with them. So it's a great services model. It's great uh if we can actually get that kind of relationship with every customer, but again with like a thousand API signups every day, I I can't be one to many serving that many customers and getting on that many calls. Um and so ultimately this doesn't really scale super well. Uh we want to make sure that we're freeing ourselves up to be leading uh humanled deals as much as possible, but we can't do that with every deal. Not every customer win can be like handbuilt by us. Um and we can't just continue to hire more and more FTEEs if we're going to try and serve more and more customers. Um, so ultimately like for anyone who's an FTE in here, if you're looking to become an FTE, if you already have an FTE team at your company, I would encourage you to take that knowledge that you have as an FTE and try to automate yourself out of a job. I think that should be your role every single day. If you want to serve your customers better, if you want to work with them more, um, and if you want to deliver a better customer experience at scale, you can't be the bottleneck to having a good customer experience. And we learned that firsthand. So again, we were drowning in like inbound um, volume. Um, you can see like we basically had a thousand API signups up there earlier. Um, and we had the natural instinct that a lot of teams do where we were like, we just need to deflect more conversations. I just want to talk to fewer people. I want to make sure I'm freeing myself up to work on more important stuff. Um, so we basically bought this off-the-shelf support bot and we're like, let's point it at the docs. It'll answer all the docs questions for us. It's going to be great. We'll never have to answer a simple question from customers again. And it only resolved about 10% of the conversations. Um, so it helped. Um, you can imagine if we had like a thousand conversations a day, a hundred of those were now entered by a bot. But then our team still had to field 900 tickets a day. So it wasn't exactly very scalable. Um, so 10 10% uh end to end resolution rate wasn't good enough. And ultimately we couldn't iterate fast enough to be able to fix this. We didn't have access to the system prompt. We didn't have access to the tools. We didn't have access to the rag infrastructure. So anytime we needed to change something, we'd go to this vendor and they'd say, "Well, it's on the road map." and that didn't quite work for us. So, we wanted to build our own. So, we built uh another member of our team rather than hiring one and we named him Joey. Um so, anyone now who reaches out to us, whether it's on our chat widget on our site, whether you reach out to contact sales, whether you reach out to our support alias, you're going to meet Joey instead of a human. Um he's the first line of defense now for every single conversation that happens in bounded assembly. And he's built to be the most infinitely scalable FTE on our team. Um, so he remembers past conversations with you. Um, we track all of his metrics via Pylon because he's still a member of our support team and so he has access to Pylon to manage conversations and remember who you are. Um, and he's built on top of the cloud agent SDK. And so the cool things that he's able to do is he can manage his own infrastructure. He has a file system. He can write and debug code for you. He can call tools. He can give himself new abilities. Um, and so he's actually able to operate more like an FTE and less like just a standard chatbot that you'd be talking with, you know, on on most major sites nowadays. So, um, if you want to know a little bit of like how to build your own Joey, um, the architecture kind of has like four major parts. Um, so as everyone might know from working with Claude, Markdown is your friend. You should be giving Markdown files to Claude so that he understands everything about your business. And so we wanted him to have access to all of our documentation and be trained up on it all the time. So, we checked all of our docs out as local markdown and gave it to Joey. Um, and so anytime that we push an update to our documentation system, he's able to get all of those files and he has them locally in his file system. So, actually what's really cool is if our entire doc site was down right now, you could go to Joey and still get live answers on all the newest stuff that we've shipped without our team having to actually get involved. um for retrieval so we can give you a good answer. We partner with Voyage so we use their embeddings um so that he gets the most relevant documents up front so he doesn't have to spend a lot of time searching and he can get you a faster answer. And then since he has all these docs as local files if he gets confused or he needs to string together multiple resources to create a response he can agentically search the file system that he lives on so that he can go and find those resources for you. Um so he can kind of go into this like deep thinking mode where he's able to search across all these resources and then we shipped it on railway. We didn't want to deal with infrastructure. We didn't want to spin up an EC2. We didn't want to have to change instance types. So, we deployed it all on Railway. Their service is pretty incredible. Basically, we can just go ahead and see if he has a bad conversation with someone in Slack. We can write a PR real quick and we can deploy it and within 30 seconds there's a new live version of Joey. So, we're able to iterate really, really quickly. Uh we've actually had times where we've actually monitored a live conversation, caught a bug, deployed a fix, and then it's been proof for the rest of the session, and that customer doesn't even have to know that we we shipped that fix behind the scenes. Um so Joey is able to to basically get updated whenever we want. Um so we actually have full control over everything that that he does. Um and so the results were pretty remarkable from us building this. And it's another reason why and wanted to encourage everyone who's here to try and build your own Joey. So, we went from 10% to 80% endto-end resolution rate in just the first week of deploying this this build with a pretty naive implementation. And we did that all for around $700 a month in both token and inference uh or infrastructure costs. So, we resolved 80 80% of our our tickets end to end with no human in the loop. And some people like to game these metrics. They'll say like, oh, we resolved 80% but only on like a subset of types of tickets. Not the case here. you cannot reach a human directly unless you go through Joey and you tell him to escalate. So he actually does handle 100% of these inbound tickets for us now. Um and he only escalates 20% of them to a human for the right reasons. Maybe it's a rate change. Maybe it's a data opt out. Maybe you need to sign an agreement with us or you have needs that only an FTE or someone from legal needs to be involved in. These are things that actually still need a human but eventually we can automate. Um and so the best part is whatever Joey can't do yet gives us a very clear list of what we need to do to continue to scale our team. Um, if he's not able to send them a BAA, well, okay, maybe we should give him a link that he can reference as a resource. Um, if he's not able to talk to customers about pricing, well, you can negotiate with Joey. Um, if you go on the site and you test him out afterwards, you can tell him exactly how many hours you have. Um, and he'll actually quote you a rate and then you can negotiate with him. Um, so we tried to automate a lot of these other things that he was still escalating and there's still there's still plenty more to come. But the real thing that, you know, we wanted to test out by building Joey was we wanted to dog food our own product, too. We didn't want to just like build something that deflects more stuff from our customers. We didn't want to build something that just um is kind of this general support bot that anyone can build. We work in voice and our customers build voice agents. So, I wanted to build a voice agent and so I wanted to give a voice to Joey. So, now rather than just chatting with him via text, you can actually chat with him using your voice as well. Um, so we have a voice agent API as I mentioned earlier. Um, and we integrated this into Joey this week. So now if you go onto our site, you can switch to voice mode and you can talk to Joey using just your voice. Um, because I ultimately think for any FTE here, no matter how much you've been deployed with your customers, no matter how embedded you are with them, the best way to understand what your customers are building is to actually build their same product yourself. And that's exactly what we wanted to do with this. So we put our voice agent API into Joey. Um, it's all speech in, speech out, one websocket connection where we string together our speechto text model, an LLM and text to speech, all optimized for voice. It's real-time and low latency. It handles all the pauses, interruption handling, barge, all of that kind of stuff for you. U, we bring our own voices, so you don't have to find another provider. And it's all the same tools that he already had if you were communicating with him via text. It's just now you can actually talk to him like you were a human. Um, and hopefully later this week we can hook them up to a phone number. So now we'll have an Assembly AI phone number so you can get real-time support from Joey whenever you need it. So let's go ahead and talk to Joey a little bit and we can walk through some of the architecture live. Um so for anyone who wants to know a little bit more about how this was actually built um we basically have all of the the docs checked out locally in Markdown. Um so every single time someone puts an update out to our docs, these files also get locally checked out into this repo. So they stay perfectly in sync with any new features that we're shipping, our change log, our pricing, all of these major uh pages from our website get converted to markdown and get given to Joey. Um so he sees all of these in markdown format and he's able to see what the URL is so he can site his sources to the customer in case they want to double check his work or if they just want more information. Um at the same time, um all of it is provided as like Claude MD instructions. I think it's like a 30,000 line Claude MD of all these guard rails and and advice on how he should be operating with customers. Um, and this is what we majorly update whenever we see that he had a bad experience with a customer or gave a wrong answer, we ship an update to this cloud MD and now for every conversation that he has going forward, he's able to be better. Um, and it's all contained as a Docker file. Um, and so it's really easy to go and just containerize and deploy him on Railway. Um, and so again, every time we ship an update to this repo, a new version's already live on Railway and no one had to manually monitor the deployment. Um, so let's go ahead and uh I'll fire up Joey here. we can ask him a pretty typical question so you can see what our voice agent API is like. So I'm going to simulate this locally because I I know probably many of you are going to test Joey after this and I don't want to add to the queue for our actual support team. Um so I'll I'll go ahead and just do a local simulation of it. But typically, yeah, you would go into here, you'd meet Joey, you'd want to have a typical question. You might talk to him via text, but let's go ahead and talk to him with voice. Hi, this is Joey at Assembly AI. How can I help you today? \>\> Hey Joey, I'm looking to build an ambient medical scribe with Assembly AI, but before I can test, I need to assign a BAA. Is that something that you can do? \>\> One sec. Let me check that for you. Yes, you can execute the BAA yourself right away. Just complete our standard HIPPA business associate agreement, but note that you'll need a paid account with a card on file first. And signing the BAA automatically opts you out of model training. The relevant docs are linked in the chat. \>\> Cool. Thanks, Joey. Well, I got to I got to get back to my demo. I'd love to chat more, but uh thanks for all the help. You are very welcome. Good luck with your demo and feel free to reach out if you need anything else later. Have a great day. \>\> Isn't he so nice? \[laughter\] I enjoy talking to Joey. I think our customers will enjoy talking to Joey. But this is a great example of like two years ago when I first joined assembly on our support team. This is a conversation that would have to be handled by a human. This is something that if you roached out in in chat and a customer uh or a support engineer wasn't able to uh answer you immediately. They'd have to wait. They'd have to get a response from us. It was a canned response. Um it was just not a great customer experience. And now you have an FTE that you can talk to 24/7 for these basic questions. Um and you can do it through voice. So you can actually test our product at the same time. Um so there's there's kind of a meta layer there of where if you're talking to Joy about how to build a voice agent API, you're already using it. Um, and you're already seeing exactly what your product experience could look like. So, if there's anything that you should get out of this talk, it's not just that you should build your own Joey. Um, but I think that it's like if you're if you're an FTE at your company, you need to be dog fooding your own product all the time. It's not enough to just be teaching your customers how to get started with it. any new feature that you launch, any new product that you offer, you need to be teaching your customers, um, with all the knowledge that you've built from actually using and building and trying to ship the same product. Um, so I had to run through all the same walls. I had to figure out how to deal with latency. I had to figure out how to deal with turn taking or bargain or interruption handling. Um, and I think I have a lot better empathy of how to like work with customers that you want to use our voice agent API. Um, because now I've actually used it. I've used the same API. It's the exact one that my customers build on and I can give them better advice because I've actually shipped something with it. So go build go build something. Go build whether it's a your own Joey, go build whether it's a voice agent, especially if it's a voice agent, build it on Assembly AI. Um, but if you're if you're actually working in an FTE capacity, try to build the exact product that your customers are building and do it on your own. Don't just wait for them to ask you for help. Don't just wait to remove all their blockers. You should be going out and building the exact same thing. And at the same time, you might actually be able to make your customer experience better simultaneously. So, um, if anyone has like questions on how to build Joey, if you want to check them out at the booth afterwards, our booth is just right over here. Um, if anyone has feedback on Joey since this just shipped this week, if you have a weird conversation with him, I'd love to know about it so we can keep making fixes and I can show you how we deploy that live. U, but overall, really excited to uh, be here at the conference and talk to more people about voice agents. So, thanks for uh, coming and listen to this talk. \>\> \[applause\]