Talk by Bala Ramdoss (Amazon) arguing that LLM agent output is not a finished user experience, and that AI features need a dedicated rendering layer between model output and the product UI. Drawing on building agentic customer experiences on mobile at Amazon at large scale, it covers: generative UI as an engineering layer; a rendering contract — a typed, versioned agreement between model output and UI components with graceful fallbacks for unknown content types; streaming model output progressively into structured, typed UI components such as product cards, comparison modules, and follow-up prompts; and Backend-for-Frontend (BFF) patterns that absorb model unpredictability on the server while preserving conversational context across turns. # Channel Presentation Observations — "Agent Output Is Not UX" ## Video Identity - **Speaker:** Bala Ramdoss (also introduces himself as "Balaram Das" in the transcript), presenting as an individual with a **personal disclaimer**: *"I'm giving this talk on my own. The opinions in this talk are mine and not my employer's."* - **Affiliation claimed:** *"The past 6 years I've spent building Amazon Lens"* — described as *"our suite of camera features powered by AI."* ## Intended Audience (Observable) - **Builders of agentic AI products**, specifically engineers/product developers shipping LLM-powered features to end users: - *"Now, imagine if you were to build something like this for your customers."* - *"The agentic product you're building already has all the tools to support your customer's needs."* - **Mobile/app developers** in particular — recurring mobile-specific concerns: *"hundreds of millions of installs and you don't control when any of them update,"* Android vs. iOS rules, fragmented app versions. ## Claimed Scope and Expertise - **Explicit self-description:** *"I have been building customer-facing apps for over a decade."* Talk *"comes from my experience building customer-facing products that are in millions and millions of mobile devices."* - **Topic scope:** The layer between LLM output and human-interactive UI — **generative UI, streaming, and backend-for-frontend (BFF) patterns**: - *"That layer is what I'm going to talk about."* ## Presentation Style (Demonstrated) - **Conference-style talk** with opening hook (restaurant reservation demo), structured outline (*"three patterns"*), illustrative product examples (Lens live, Gemini), and closing takeaways plus a QR code for follow-up. - **Cites external work rather than only own products:** ATUI spec ("from Google"), Copilot Kit's three-tier generative UI spectrum, MC PA apps, Gemini agent-status UX. - **Opinionated framing:** *"your agent output is not the CX. You build on it"*; *"let the BFF absorb the model output so the client can stay dumb and safe."* ## Creator Claims vs. Demonstrated Patterns - **Claimed:** decade of customer-facing app experience; 6 years on Amazon Lens; opinions personal, not employer's. - **Demonstrated:** deep practitioner vocabulary (version-aware context, typed UI intent, time-to-first-chunk, server-driven UI) consistent with the claimed production background; teaches reusable patterns rather than promoting a single product, though Amazon Lens features are used as examples. # Delivery Observations: "Agent Output Is Not UX" — Bala Ramdoss ## Overall Tone & Register - **Conversational, first-person practitioner tone** throughout: the speaker frames everything as lived experience rather than theory. - > "I wouldn't have had a name for it. I'd built a couple of AI features by then and every single time I solved this part from scratch." - **Plain, audience-directed language** with direct appeals ("you", "your"): - > "Now, imagine if you were to build something like this for your customers. What would you want it to do?" - **Measured enthusiasm** appears at specific moments, e.g., "What gets me is that this thing now has a name." ## Structure & Teaching Habits - **Numbered pattern framework**: the talk is explicitly organized into three patterns, announced and then recapped: - > "We'll break into three patterns. Pattern one, the rendering contract. Pattern two, streaming... Pattern three, uh the BFF that sits in the middle." - > "Um we'll take them one at a time starting with the contract." - **Explicit transitions** between sections: - > "So, that's the contract. Now, let's move on to the streaming part." - > "We have covered streaming. Let's move on to the final and most important part." - **Takeaway summary at the end**, enumerated: - > "That brings to our takeaways. One... Two... Three..." - **Cold open with a concrete anecdote** (restaurant reservation) before credentials or framing — an inductive, example-first teaching style. - **Bookending**: the talk closes by returning to the opening scenario: - > "Bringing back to where we started, your agent output is not the CX. You build on it." ## Recurring Vocabulary & Phrases - **"Layer" as the central metaphor**, repeated: - > "the layer between the model and something that a human can interact with" - > "these are delivery problems and they live in between the model output and what's on the screen" - **"Spectrum" used twice** for scaling concepts: - > "And it's it's a spectrum." - > "And it's a spectrum of server control." - **Contrasting "model" vs. "client/BFF" responsibility** is a recurring frame: - > "It never invents a component. It chooses from a fixed menu that you provide to it." - > "let the BFF absorb the model output so the client can stay dumb and safe." ## Rhetorical Devices - **Negation-as-emphasis**: repeatedly states what something is *not*: - > "Now, it's not wrong." - > "none of these problems are due to the model itself" - > "No, none of these are about the model. The model was fine." - > "It never invents anything." - **Rhetorical questions to the audience**: - > "Is that experience going to be snappy or slow? Do we get all the information at first or one thing at a time?" - **Second-person imperatives / soft advice**: - > "But please use it sparingly." - > "I encourage you to try." - **Concrete numbers to ground claims**: - > "hundreds of millions of installs" - > "It may take 3 to 4 seconds to get there, but the wait is bearable." - > "over a decade" / "the past 6 years" ## Filler & Disfluency Patterns - Frequent **"um"/"uh" fillers and restarts**, indicating a spoken, lightly-scripted delivery: - > "Um quick dis- disclaimer before I go further." - > "So uh for this delivery problem, if if you had asked me about uh about this layer 2 years ago..." - > "the BF BFF doesn't just ship layout." - Occasional **self-interrupted corrections**: - > "the model is trying to draw the UI for" (after "it stays true to the client") ## Credibility & Attribution Habits - **Explicit disclaimer of employer affiliation** — a professional norm: - > "I'm giving this talk on my own. The opinions in this talk are mine and not my employer's." - **Cites external sources by name** when borrowing examples: - > "there's an open spec for it. ATUI from Google." - > "Co-pilot Kit, a company that does this primarily, lays out in three rounds." - > "Um here's an example I copied from Gemini." - **Honest framing of uncertainty/limits**: - > "This may not work for all, but here's a product example." ## Confidence Notes / Uncertainty - Patterns such as the numbered structure, negation emphasis, and "layer" framing are well-evidenced by multiple exact repetitions above. - **Personality traits and motivations are not assessed**; the observations above are limited to observable delivery choices in the transcript. - Tone judgments ("measured enthusiasm") are based on limited wording cues ("What gets me is...") and should be treated as lower-confidence than structural observations. # Observation Document: Agent Output Is Not UX — Bala Ramdoss ## Talk Identity - **Speaker**: Balaram Das (transcript title says "Bala Ramdoss, Amazon Lens") - **Stated credentials**: Over a decade building customer-facing apps; past 6 years building **Amazon Lens**, Amazon's AI-powered camera suite (visual product search via images, screenshots, barcodes). - **Disclaimer (explicit)**: Speaking on his own; opinions are his, not his employer's. ## Core Thesis - The example of a restaurant reservation shows the problem: model output was factually correct but left the user doing the work. A rendered UI (date, time, taps) completes the task. - **Key position**: Models/agents are here to stay; the differentiating layer is **between the model output and what's on the screen**. "Your agent output is not the CX. You build on it." - Delivery problems — not model problems — decide product success. The model "does its job well." ## Named Concepts and Claims ### Generative UI - Two years ago the speaker had no name for this layer; he solved it bespoke each time with no shared vocabulary. - Now it has a name — **Generative UI** — and an open spec, **ATUI/A2UI from Google**: the agent describes UI as data (component list); the client renders with native widgets. - **Spectrum per Copilot Kit** (cited): 1. **Control** — model picks pre-built components; invents nothing. 2. **Declarative** — model composes from a catalog; where A2UI sits. 3. **Open-ended** — model generates novel UI on the fly (e.g., MCPA apps). - The higher up the spectrum, the more trust the client must place in the model. Most production mobile apps live in the bottom two tiers — "that's where you stay safe." ### Mobile constraints - Web fixes ship in minutes; mobile has **hundreds of millions of installs** with no control over update timing. - Unrecognized content types crash, and keep crashing for days/weeks on un-updated clients. - **Rule**: "You cannot meaningfully patch the client." ### Pipeline overview Version-aware context → model outputs typed UI intent → **BFF** → client render with safe fallback. ## Three Patterns ### Pattern 1: The Rendering Contract - Make the model aware of client capabilities; maintain a version-capability map (e.g., flight card surfaced only from v2.0 onward). - Model-facing half is the hard part: model streams **blocks** (conversation block + UI block), not text. - Contract encodes layout rules: 1–3 flights → swipeable carousel; 4+ → vertical list. Model picks intent but **never invents a component** — it chooses from a fixed menu. - Acknowledged open problem: this gets "significantly harder" as features scale to more surfaces. ### Pattern 2: Streaming - Traditional request/response fails with LLM latency plus safety checks. - Streaming renders chunks: skeleton → partial → complete; total 3–4 seconds becomes bearable. - **Metric shift**: stop chasing total latency; chase **time to first chunk**. - Loading spinners don't work for AI features; alternatives: - **Lens live**: let users focus/tap objects while waiting. - "Thinking" CX — but "use it sparingly"; users are past the forgiving phase and expect visibility. - Gemini cited: showing agent activity builds trust even during 10-second tasks. ### Pattern 3: The BFF (Backend for Frontend) - Subset of **server-driven UI**; a spectrum of server control — "the more the server controls, the more you build." - Owns platform rules (Android vs iOS), so the client makes fewer decisions. - Ships more than layout: format, context, **hydration**, and **action payloads** (tap handling, deep links, impression metrics), plus conversational context across turns. - Retrofit benefit: reuse existing production components (flight row, product card); same brand, familiar, native-feeling — "a good way to build for humans." ## Takeaways (as stated) 1. Models are already capable; feed a typed, versioned contract. 2. Stream into typed components, not text; show the user what the agent is doing. 3. Let the BFF absorb model output so the client "stays dumb and safe." - Repeated qualification: none of this is about the model — the rendering layer "is what ships the product." ## Preserved Qualifications - "This may not work for all" (re: engagement-during-wait design). - "Please use it sparingly" (re: thinking UI). - Contract scaling difficulty is left as an open challenge for the audience ("I'll let you think about how to engineer that context"). # Speaker Attribution Record ## Video **Title:** Agent Output Is Not UX: Rendering Layer Your LLM Pipeline Is Missing — Bala Ramdoss, Amazon Lens **Video ID:** `01a15ca8-7464-7f15-9025-a8444ade9bfa` ## Speakers & Roles - **Bala Ramdoss (speaker also self-introduces as "Balaram Das" in the transcript)** — sole speaker; role: **presenter/creator** of the talk. - Self-described background: building customer-facing apps for over a decade; past 6 years building **Amazon Lens** (camera features suite powered by AI, per speaker). - **Disclaimer (explicit in transcript):** speaking on his own behalf; opinions are his own and not his employer's. ## Referenced Third-Party Sources / Claims - **ATUI / A2UI (attributed to Google)** — described by speaker as an open spec for generative UI; speaker attributes the name "Generative UI" and spec existence to this. - **Co-pilot Kit (company)** — speaker attributes a three-tier framework (control / declarative / fully open-ended) for generative UI. - **Gemini** — speaker states an agent-progress UI example was "copied from Gemini." - **MC PA apps** — cited as example of fully open-ended model-generated UI (transcript term as-is). - **Amazon Lens / Lens live** — speaker's own product examples; includes implicit promotion of trying the feature in the Amazon app. ## Promotional Segments - **Implicit product mention:** invitation to try Amazon Lens features via the Amazon app (not flagged as third-party promotion; tied to speaker's employer, alongside his personal-opinion disclaimer). - **Closing call-to-action:** QR code to "learn more or connect with me" — self-promotional, end of talk. ## Provenance Notes - No curated metadata was supplied beyond title and transcript; speaker identity comes solely from the transcript self-introduction. - No comment availability information present. - No cited sources beyond those listed above; no additional speakers, guests, or narrators. I asked my AI assistant to help me reserve a table at a popular restaurant. Here's what it gave me. Now, it's not wrong. The phone number is there. The hours are right. It even knows the walk-in oyster bar at the front. The model did the real work, but look at the outcome. I have to do my research and work towards actually booking the table. Like, we've been here for a while. Now, imagine if you were to build something like this for your customers. What would you want it to do? Same answer, rendered like this instead. A date, a time, a couple of taps, and then you're done. The agentic product you're building already has all the tools to support your customer's needs. The only thing that you need to focus on is the layer between the model and something that a human can interact with. That layer is what I'm going to talk about. The models and agents are here to stay. And we should learn how to make it friendly to the humans. Hello. Welcome to my talk. I'm Balaram Das. I have been building customer-facing apps for over a decade. And the past 6 years I've spent building Amazon Lens. Amazon Lens is our suite of camera features powered by AI. It enables you to shop using images, screenshots, and barcodes to discover visually similar products. If you have Amazon app installed, I encourage you to try. This talk comes from my experience building customer-facing products that are in millions and millions of mobile devices. Um quick dis- disclaimer before I go further. I'm giving this talk on my own. The opinions in this talk are mine and not my employer's. When you think of building UX for an agentic AI, obviously ChatGPT comes into mind. How do you draw these cards? How do you choose carousel versus list? How do you craft this so the human can understand and interact better? When you attempt to build something like this, you run into a wall. There are problems that need to be solved from the very early stages of your system. Is that experience going to be snappy or slow? Do we get all the information at first or one thing at a time? And how do you scale this for mobile apps where versions and device capabilities are fragmented? Like if you notice, none of these problems are due to the model itself. The model does its job well. Um these are delivery problems and they live in between the model output and what's on the screen. That is the layer that decides whether your product succeeds or not. So uh for this delivery problem, if if you had asked me about uh about this layer 2 years ago, I wouldn't have had a name for it. I'd built a couple of AI features by then and every single time I solved this part from scratch. A bespoke pattern shaped around whatever system I was in. Uh there was no shared vocabulary for any of it. What gets me is that this thing now has a name. Generative UI. And there's an open spec for it. ATUI from Google. Instead of the agent handing you a raw text or HTML, it describes the UI as data. A list of components and the client renders them with its own native widgets. A problem I used to solve alone is becoming something teams get to start from. And that's exactly what I'm excited to dig into. Um normal API returns data and the client decides how to draw it. Like your model is good at tool use, coding, and even other complex tasks. It can do UI, too. And that is what the generator UI is about. And it's it's a spectrum. Uh Co-pilot Kit, a company that does this primarily, lays out in three rounds. At the bottom, we have control. The model picks a pre-built component like a product card. It never invents anything. In the middle, um declarative. The model composes UI from a catalog. Uh date field, a time field, or a submit button. Um that's where A2UI sits. And at the top, it's fully open-ended. The model generates a novel UI on the fly like MC PA apps. The higher you go, the more your client has to trust whatever the model hands it. Um most production mobile apps live in the bottom two rounds because, you know, that's where you stay safe. And that's what we'll focus on. Um adding to the complexity of UX building, um mobile apps play an important part. Um on the web, if uh render breaks, you know, you ship a fix and it's live in minutes. Uh mobile apps cannot do that. Like you're looking at hundreds of millions of installs and you don't control uh when any of them update. So, when a client meets a content type it's never seen, it doesn't gracefully degrade, it crashes. And it keeps crashing for days or weeks um in hands of people who haven't updated. So, one rule holds everything that follows for mobile clients. You cannot meaningfully patch the client. Um how does this system look when you zoom out? Um here's the whole simplified pipeline. Um version aware context face that feeds the model. Uh yes, that is context engineering. The model outputs typed UI intent. Uh that flows through a BFF backend for frontend. And to a client render that draws it and uh falls back safely when it cannot. We'll break into three patterns. Pattern one, the rendering contract. Pattern two, streaming. Um that's the flow in between them. Pattern three, uh the BFF that sits in the middle. Uh we'll take them one at a time starting with the contract. Um the contract is all about making the model be aware of what the client capabilities are. You ensure it it stays true to the client that the model is trying to draw the UI for. Um what it ensures is that that the onus is not on the client or the rendering layer to infer what to show by looking at the token output. Um you also maintain a repository of version map with the capabilities. For example, you introduced a new flight uh card UI in version 2.0. Uh make sure to surface it to the model only from version 2.0 onwards. Um when you build the context. The takeaway here is that the model is going to choose the CX and you provide the right information in its con- context. Um let's take a look at this example. And this this is usually the hard part of building the system. This is the model facing half of the contract. And you don't want the model to send back text. Like, you want it to stream blocks of UI components. Um a conversation block for what it says in text and a UI block for what what it wants the client to render. The contract even encodes layout rules. In this example, one to three flights a swipable carousel. Four or more, a vertical list. The model picks the intent. And notice what the model never does. It never invents a component. It chooses from a fixed menu that you provide to it. Obviously, it won't be in the millions. It would be a handful. It turns out that it gets significantly harder as you scale your features to more surfaces. I'll let you think about how to engineer that context to make sure the model succeeds in picking one. So, that's the contract. Now, let's move on to the streaming part. This was eye-opening to me when I first encountered it. Traditionally, apps make an API call and wait for something to happen. When LLMs are involved, this pattern becomes ineffective as often the latency is higher due to the model themselves. And on top of it, there are tons of additional checks to ensure safety and whatnot. And we also introduce a new layer that it's going to be a little bit complex to do the dynamic UI. Um streaming helps here uh by not making the client wait for the whole response. It renders things in chunks at a time. Take a look at the illustrative example when you have to surface information one chunk at a time. First, you show a skeleton, then partially fill it, then complete it. It may take 3 to 4 seconds to get there, but the wait is bearable. That kind of changes what you measure for an app. Like you stop chasing the total latency, which we have done for over a decade. You start chasing time to first chunk. You know, the first useful thing that your user sees. For this reason, the traditional loading spinner won't work for AI features. Sometimes, you have to get creative and design your feature around it. This may not work for all, but here's a product example. And when you know it's going to take time to render something, keep the user engaged. The Lens live allows the user to focus different things, um tap on an object that they're interested in while they wait for the results. Uh if you don't have the full screen in your control, you can show the thinking CX. But please use it sparingly. The overall AI users have moved out of the forgiving phase, and now they expect to know what is happening. Now that you have a way to stream back different states, you know how to show what your agent is doing. Um here's an example I copied from Gemini. When when it starts to work on a task, it gives a glimpse into what the agent is doing. Even though it takes 10 seconds, I'm okay if I know what's happening and be able to trust the agent's final output. We have covered streaming. Let's move on to the final and most important part. Your new BFF. Pattern one was about how UI gets picked. Pattern two was about how the chunks delivered. Pattern three is all about how the chunks become meaningful UI elements. This is a concept of server-driven UI. And it's a spectrum of server control. And the more the server controls, the more you build. The BFF is a subset of it and decides how to render. It owns the platform specific rules like Android versus iOS, that kind of stuff. Uh it also helps the client to make less decisions and draw what's handed to it. That's the whole idea. The BF BFF doesn't just ship layout. It does a little more. In addition to the format and context, it does the hydration and adds actions. Every rendered element carries an action payload to handle what what a tap does, what the deep link hit opens. It could even name the impression metric to log. It carries this conversational context across turns. So, the next response know what came before. And the good thing about this is that you can actually do this to your existing apps. You can reuse the existing CX units you already have. The flight row, the product card, the components your app already shipped in production. You're not building a new agentic look into your app. And that's a good way to build for humans. Same brand, same density, same familiar feel. It looks and feels native. That brings to our takeaways. One, the models are highly capable already. You feed a properly typed, versioned contract so it can pick and choose the right CX. Two, stream into typed components, not text. Let the user know what your agent is doing. Three, let the BFF absorb the model output so the client can stay dumb and safe. No, none of these are about the model. The model was fine. This layer is what ships the product. Bringing back to where we started, your agent output is not the CX. You build on it. Thanks for watching. If you'd like to learn more or connect with me, here's a QR code.