Part 3 of 4 of Anthropic's Claude Managed Agents workshop, recorded at AI Engineer World's Fair 2026. Gabe Cemaj demonstrates multi-session fan-out with a custom tool, deployments that run an agent on a schedule, and Dreaming, a research preview that reviews 30 days of past sessions to improve a memory store. A Q&A covers observability, 10-hour sessions and compaction, workspaces for tenancy and access, skills vs multi-agent, the relation to Claude Code, AWS deployment, scaling, and on-prem options. Speakers: Priyanka Phatak and Gabe Cemaj, Members of Technical Staff at Anthropic. # Video Identity Observations: "Scheduled Agents, Dreaming and Q&A: Claude Managed Agents (Part 3 of 4) — Anthropic" ## Creator Self-Description Evidence - The video is explicitly framed as **part of a multi-part series**: the title identifies it as *"Claude Managed Agents (Part 3 of 4)"*. - The creators present themselves as **Anthropic staff demonstrating their own product**: *"this main agent is going to um evaluate if it has information"* and repeated references to *"our platform"*, *"we intend to you for you to interact with it is via the API and the SDK"*. - They position the product relative to other Anthropic offerings: - > "cloud code is a terminal tool and we obviously want to be an API product that and infrastructure uh that is going to scale that's going to be reliable" - > "We took the cloud code harness as an inspiration... cloud code is trying to give you a a product out of the box. So we have slightly different uh you know value uh props" - They describe a **product philosophy**: "everything is just API calls", "we don't want to be opinionated", and "as Claude gets smarter... those guard rails... melt away". ## Intended Audience (Observable) The Q&A and demo strongly indicate a **developer / technical builder audience**: - Audience questions reference **telemetry, OTEL export, RBAC, multi-tenancy, Kubernetes HPA, Bedrock, on-prem enterprise deployment, and evals** — all practitioner-level engineering concerns. - The presenters recommend workflows like *"run evals on your agent with like different types of tasks"* and describe API/SDK-first usage: *"you would probably build your product your UI that you're distributing to your customers and then calling into us"*. - Some acknowledgment of a **broader audience**: *"the console quick start... should make that pretty easy for even non-technical users to come in and build their first agent"* — but the material itself (live API demos, deployment configs) demonstrates a technical orientation. - Enterprise buyers are also addressed: discussion of **workspaces as tenancy primitives**, RBAC roadmaps, and on-prem options ("definitely been a customer ask from... enterprise customers"). ## Claimed Scope (Observable Presentation) - The video demonstrates a **live product walkthrough plus extended Q&A format**: scheduled deployments ("research desk weekly memo... first of the month at 1 p.m."), a "dreaming" self-improvement feature (explicitly labeled a **"research preview"**), and multi-session orchestration. - Scope claims made by the creators: - Production scale: *"this is being deployed today at scale in production. For example, notion external agents are built on top of this"* (also Atlassian/Jira). - Long-running agents: *"We are running compaction for you... We've seen 10-hour sessions in production"*. - Current limitations candidly acknowledged: **no RBAC yet** ("that's something our back is on our road map"), **Claude models only** ("it's cloud models only for now"), **no graph-based UI orchestration** ("Not yet... maybe soon"). - An in-person **workshop setting** is evident: live audience questions, a shared repo via QR code ("are you looking for the repo?"), and Wi-Fi apologies during the demo. ## Demonstrated vs. Claimed Distinction - **Demonstrated**: live product UI (console, deployments, memory stores, dreaming preview) and real audience interaction. - **Claimed but not demonstrated in this video**: production usage at Notion/Atlassian, 10-hour sessions, eval-driven uplift from dreaming, roadmap items (RBAC, on-prem, graph orchestration). # Delivery Observations — Claude Managed Agents Workshop (Part 3 of 4) ## Overall Format - **Live demo + open Q&A structure.** The video alternates between narrated product walkthroughs ("So let's move on to the finale...") and extended audience Q&A, with multiple speakers (appears to be at least two Anthropic presenters fielding questions, e.g., "we can all take questions," "the verifier agent that Gabe talked about"). - **Walkthrough narration style:** the presenter describes actions as they happen in the console, often in a speculative, live tone: *"ideally what would happen and I hope this works live for y'all."* ## Recurring Vocabulary & Phrases - **"etc., etc." / "etc. etc."** — used twice as a hand-wave to suggest extensibility: *"using memory stores to learn from one another um etc etc"* and *"sub agent prompts, etc., etc."* - **"Let it cook"** — memorable slang for model autonomy: *"let claude just let it cook uh let it do what it thinks it's best."* - **API-first framing, repeated many times:** *"everything is API based," "everything is just API calls," "everything that's right... everything through the API."* This is the single most repeated message in the delivery. - **"Evals" as the universal answer** — recurs across many answers: *"The boring answer is just run evals," "everything kind of ties back to eval," "I think it'd be good to have your own set of evals."* - **"For now"** — a consistent hedge for current limitations: *"for for now we... it's cloud models only for now," "for now you would do sub agents," "right now workspace is like a clean way."* - **"Melt away / melt"** — repeated metaphor for shrinking scaffolding: *"the boundaries... start to melt away and disappear... it will start to shift and melt and become thinner and thinner."* - **Filler-heavy speech:** frequent *"um," "uh," "like," "you know,"* and doubled words (*"for for now," "we we," "you you"*) throughout — consistent with unscripted live delivery. ## Teaching Habits - **Concrete scenario framing before concepts:** introduces deployments via a relatable use case — *"what if you want to have a list of companies you care about and you want every day at 9:00 am or first of the month to do the analysis."* - **Building-blocks mental model:** *"all of these are just building blocks uh that you can compose together to build your application into something bigger."* - **Clarifying questions before answering:** repeatedly asks questioners to disambiguate — *"What do you mean by telemetry?"* and *"Are you talking about running the agent for a year or you talking about like one specific session?"* - **"That's right" / "That's right."** as a confirmation tic when validating audience restatements. - **Anticipating the audience:** *"before I get to your questions, which I'm sure there are many."* ## Handling of Failure & Limits - **Candid about live-demo hiccups:** *"And I don't know why they're failing, but I'll ask Cloud about it later"* and *"sorry about the Wi-Fi."* - **Honest about product gaps:** *"we know that there's still a gap in what we offer," "Not yet. Not yet. Not yet. But but maybe soon," "we don't already have something that we can share with you."* - **Humor under pressure:** when pressed to trust Dreaming without evals — *"if I tell you just to trust me is that no [laughter]"* — a self-deprecating deflection. ## Answer Pattern in Q&A - Typical structure: **acknowledge → give current-state answer → hedge or point to roadmap → redirect to a general principle** (usually the API or evals). Example: *"you could build that connector fairly easily. Uh we just everybody has different requirements and we don't want to be opinionated."* - **Frequent appeal to empirical outcomes over prescription:** *"if we just let Claude figure that out, it actually ends up outperforming"* and *"your your mileage may vary on your task."* ## Uncertainty - The transcript's speaker-labeling is ambiguous in the Q&A section; distinguishing the two presenters' individual styles is **not reliably possible** from this transcript alone. Patterns above describe the combined delivery, not per-speaker habits. - Single-use phrases (e.g., "let it cook") are treated as isolated moments rather than stable patterns, since repetition across sessions can't be verified from one video. # Observed Positions & Claims — "Scheduled Agents, Dreaming and Q&A: Claude Managed Agents (Part 3 of 4)" **Speaker identification note:** No curated speaker names are supplied. The presenter(s) are Anthropic staff demonstrating the product (referenced as "Gabe" by one audience member, though which speaker that names is not established in the transcript); audience members are identified only by question order. All attributions below are based on transcript turn-taking. --- ## 1. Presenter Demo Content ### Multi-session orchestration (financial analysis demo) - The presenter demonstrated a **main agent** that evaluates whether it has information, then calls a custom **"dispatch analysis"** tool to spin up individual sessions for financial artifacts (two tickers, Google and Microsoft). - Each dispatched session contains **multi-agent collaborative work**, has an **outcome definition** for self-supervision, and uses **memory stores to learn from one another**. ### Deployments (scheduled recurring work) - **Deployments** enable running an agent on a schedule (e.g., daily at 9:00 am, first of the month). - **Claim:** Everything is API-based; the console exists, but "their main way that we intend to [use] for you to interact with it is via the API and the SDK," so builders can create custom UIs and workflows. Components are described as "building blocks" to compose. - A deployment takes: an **agent definition**, a **schedule**, an **initial set of messages**, and optional **resources** (memory stores, GitHub repos, files) used on each run. - Deployments can be triggered manually from the console or via a custom UI (list, search, kick off). - **Qualification preserved:** During the live demo, deployment runs were failing; the presenter said, "I don't know why they're failing, but I'll ask Claude about it later." ### Dreaming (research preview) - **Dreaming** lets a user select a memory store, a model, optionally custom instructions (or a default prompt), and run the **last 30 days of sessions** through streaming analysis. - Mechanism as described: it spins up a session (using managed agents) with **many sub-agents** analyzing transcripts of prior sessions that used that memory store, asking: did the memory store encode required information? Did sessions hit problems? Can instructions be updated? - Observed result in demo: dreaming found **gaps in a skill** and updated the memory store to extend the skill's knowledge with lessons learned. - **Stated future intent:** extend dreaming to improve skills directly, agent prompts, and sub-agent prompts. - **Qualification preserved:** dreaming is explicitly a **research preview** — "these are early early. It's like a big experimental thing." --- ## 2. Q&A — Claims and Positions ### Telemetry / observability - **Audience:** no telemetry instrumentation was seen in the agent demo; how did Anthropic pick what to observe? - **Presenter:** full observability of all events is available (console live view; "debug" view shows every event, e.g., model call start/finish for latency metrics). Position: "Right now you get all the events and you just filter them on your end," with an invitation to request missing events. - **Bring-your-own observability (e.g., OTEL export):** possible because "everything is just API calls" — a connector "could be built fairly easily." Anthropic's stated stance: "we don't want to be opinionated." ### Long-running agents and persistence - **Audience:** can agents run indefinitely; does the persistent store grow unboundedly and need offline recompaction? - **Presenter:** "We are running compaction for you so that you can run these for days. We've seen 10-hour sessions in production." Confirmed an agent with a daily task could keep running for ~6 months without "blowing its persistence." ### Orchestration topology - **Audience:** is there a graph-based orchestration UI (agent A delegates to B and C)? - **Presenter:** today you'd use sub-agents and tell Claude what you want. **Position:** humans have a desire to constrain Claude's topology; "As Claude gets smarter we have continuously [removed] those guard rails... if we just let Claude figure that out, it actually ends up outperforming." - **Qualification:** a topological/graph workflow UI does **not yet** exist — "Not yet. Not yet. Not yet. But but maybe soon." ### Organizational separation / RBAC - **Audience:** how should a complex organization managing multiple agents get separation and RBAC? - **Presenter:** **Workspaces** are the current primitive (segment customers/users, per-workspace access levels). **Qualification preserved:** "we know that there's still a gap in what we offer"; RBAC "is on our road map." Everything within a workspace is accessible to everyone in that workspace. ### Deployment and memory semantics - **Audience (misconception raised):** thought a deployment was a point-in-time snapshot of all agents. - **Presenter (correction):** a deployment is a way to run sessions for a specific agent on a schedule (e.g., research the NVIDIA desk quarterly). **Memory** is a state persisted across sessions, but it's an **independent entity you attach** to whichever sessions you choose. ### "Harness" definition and developer confidence - **Presenter's definition:** in CMA, "harness" mainly means the **tool loop execution** — managing context and session state. Builders often mean **coordinating multiple agents/sessions**; Anthropic provides "the abstraction of a single session as a unit of work" as a primitive for building your own harness. - **On confidence an agent works as expected:** recommendation is to **run evals** on varied tasks; **outcomes** let you define an exact rubric and max iterations, and the agent iterates until that outcome is met (referenced a "verifier agent" that "Gabe talked about"). ### Skills vs. agents ("thick harness, thin harness") - **Audience concern:** fear of perpetually re-investing in new harnessing/tooling around models. - **Presenter:** the thick/thin harness and skills question is "constantly evolving." **Position:** as Claude gets smarter, "the boundaries and help that we want to give Claude start to melt away"; skills are "a good abstraction today" but will presumably become "thinner and thinner." The "boring answer is just run evals." **Qualification:** multi-agent seems more useful "in my opinion, but your mileage may vary on your task." ### Dreaming + evals - **Audience:** how does dreaming affect final outcomes; can dreaming run automatically with evals to pick among divergent directions? - **Presenter:** you choose the dream duration; Anthropic "typically will run evals on our end" and has seen customers get "significant improvement on their evals" by running dreaming then using the new memory store. "Everything kind of ties back to eval." - **Additional presenter qualifications:** dreaming is a **research preview**; uplift is general ("We've seen uplift uh on it generally"); audience members should have **their own evals**. An audience member raised an unresolved concern: relying on a single dreaming pass may miss insights (e.g., one dissenting session out of nine) without eval integration — this was not fully resolved beyond the eval recommendation. ### Relationship to Claude Code - **Audience:** how much of this is built on top of Claude Code? Request that Anthropic "keep things open" (they harvest JSON session files, experiment with memory pools). - **Presenter:** took the **Claude Code harness as inspiration** ("very well attuned for coding tasks"); not "just Claude Code in a box." Positioning: Claude Code gives a **product out of the box**; managed agents is an **API/infrastructure product** meant to scale reliably. "Ideally it performs as well if not better in all sorts of tasks" — stated as an aspiration, not a measured claim. ### Bedrock / platform availability - The console shown is **not available on Bedrock**, but an **AWS version of the console** exists via Claude Platform for AWS. ### Model restrictions - **Claim:** **Claude models only** for now; no non-Anthropic model support. (One audience question about "Sonnet 5" / a recently released model was answered: "It is supported in cloud manage agents. Go give it a try.") ### Multi-tenancy and scaling - **Position:** "the best model for multi-tenancy is not bring your own key because you lose observability"; use **workspaces as the tenancy primitive**. Options acknowledged: one workspace with your own customer→agent mapping, or a workspace per customer (which "can get more complicated because then you have to recreate your agent"). - **Scaling:** no user-facing scaling parameters (no HPA-like settings, pods, timeouts) — "you're just making API calls"; sessions are unlimited apart from **rate limits**. - **On-prem/enterprise (sensitive data, secure proxies, in-house compute):** "definitely been a customer ask"; Anthropic is "seriously considering it at the moment" but has **nothing to share** — explicitly unresolved. ### Deploying customer-facing UIs at scale - **Presenter:** this runs in production today at scale — **Notion external agents** and **Atlassian Jira integrations** are built on it. Recommended pattern: build your own product/UI, call the API for agent management; "everything is through the API." MCP channels (e.g., "fake chat") were mentioned as a UI communication avenue. ### Co-work / Claude Code users up-leveling to managed agents - **Position:** the console quick-start ("telling Claude the agent you're trying to build") should let non-engineers build a first agent; a **managed-agent skill in Claude Code** walks users through building one in natural language. - **Differentiation:** co-work is for **personal agents**; managed agents is for **team/internal-organization agents or shipped products**. A skill exists to call the managed-agents API, enabling translation from co-work setups ("five will do it for you" — reference to a model version, as transcribed). --- ## 3. Unresolved / Contradictory Items Preserved - Live deployment failures during the demo were unexplained. - No graph-based orchestration UI exists yet ("maybe soon"). - RBAC is acknowledged as a gap; only workspace-level separation exists today. - On-prem deployment is under consideration only, with no committed offering. - Dreaming's confidence question (how to validate divergent dreaming runs against evals) was answered with the general eval recommendation, not a concrete automated dreaming-eval loop; the audience concern about missing insights was not directly resolved. - One audience member reported the workshop offering link did not work; presenters thanked them without resolution on-screen. # Speaker & Provenance Record **Video:** *Scheduled Agents, Dreaming and Q&A: Claude Managed Agents (Part 3 of 4) — Anthropic* **ID:** 01a15ca6-7727-72f1-82cb-6cc7ad0536c7 ## Speakers & Roles - **Lead presenter (Anthropic)** — delivers the live demo of Claude Managed Agents (multi-session dispatch, deployments, "dreaming") and answers most audience questions. Referenced as **"Gabe"** by another speaker: *"[...] the verifier agent that Gabe talked about."* Name appears only in that audience remark; transcript is unedited/live. - **Second Anthropic responder(s)** — at least one additional Anthropic team member answers questions (workspaces/multi-tenancy, deployments vs. memory, models supported, enterprise on-prem). Transcript's `>>` markers do not reliably distinguish individual Anthropic responders; treat as "Anthropic staff (unspecified)" where attribution is ambiguous. - **Audience members (unnamed)** — multiple questioners; no names or affiliations given except self-descriptions ("those of us who have been working with cloud code for a year or more," enterprise developer, partner manager, etc.). - **Moderator(s)** — off-mic voices directing Q&A flow ("We have a question over here"). ## Content Segments 1. **Demo (presenter):** multi-session dispatch, scheduled **deployments** (scheduled agent runs with initial messages and attached resources), **dreaming** preview (memory-store analysis over prior session transcripts). 2. **Q&A:** telemetry/OTEL export, long-running sessions (compaction, "10-hour sessions in production"), sub-agent orchestration (no graph UI yet), workspaces/RBAC gap, deployments vs. snapshots, harness definitions, skills vs. agents, dreaming + evals, Claude Code lineage, Bedrock availability, model restrictions (Claude models only), multi-tenancy (BYOK discouraged), scaling, UI deployment, Gemini ("sonet five"/Gemini 5?) mention — Anthropic response: *"It is supported in cloud manage agents"* — co-work vs. managed agents, enterprise/on-prem (under consideration), repo QR code request. > Transcript appears to be an automated transcription of a live talk: "cloud"/"Claude" and "CMA"/"managed agents" are used interchangeably; some names (e.g., "sonet five," "hepo," "arvback" = RBAC) are garbled. ## Referenced Sources & Third-Party Claims - **Products/platforms cited:** Claude Managed Agents (CMA), Claude Code, Claude Console, Claude Co-work, Bedrock (console not available), Claude platform for AWS, Notion external agents and Atlassian Jira integrations ("built on top of this"), MCP channels, OTEL, Kubernetes HPA (analogy only). - **Claimed production facts (from Anthropic speakers, unverified):** 10-hour sessions in production; Notion/Atlassian building on the platform; customers seeing "significant improvement on their evals" via Dreaming; third-party model "supported" in managed agents. - **"Dreaming" status:** explicitly stated as **research preview** / experimental. ## Promotion & Availability - **Promotional context:** entire video is an Anthropic product workshop/demo (Part 3 of 4); marketing-adjacent but substantive technical Q&A. No third-party sponsorship detected. - **Availability limits:** audience member reports **a shared "offering link" does not work** ("the offering link does not work for me"); final request for a **repo QR code / repo link** — repo URL not readable from transcript. - **No comments data** supplied in source material. ## Uncertainty Flags - Presenter named only via audience remark ("Gabe"); other Anthropic responders unnamed. - Third-party model name garbled ("sunet five") — model identity uncertain. - Transcription artifacts throughout; treat exact product phrasing cautiously. Okay. So, yeah, let's let's see what this looks like. Uh, analyze uh Google and Microsoft. Um, so ideally what would happen and I hope this works live for y'all um is that uh this main agent is going to um evaluate if it has information and then it's going to give us a custom tool where we're going to then spin up individual sessions for each of those financial artifacts. So you'll see this dispatch analysis thing. Um it's going to call it for two tickers and then it's going to kick runs for each of those. Um so this is how we can orchestrate our multiseession. um harness and each of those sessions has multi- aent uh that are doing collaborative work and each of those uh also have outcome definition so that there's self-supervision happening using memory stores to learn from one another um etc etc. Um so let's move on to the finale which is um what if you want to do recurring work? What if you want to do uh you want to have a list of companies you care about and you want every day at 9:00 am or first of the month to do the analysis over all of these companies uh on a schedule. So that's what deployments are. Deployments is everything is API based. So we you can obviously interact with our platform via the console. Um but their main way that we intend to you for you to interact with it is via the API and the SDK. So you can build uh custom UIs and custom workflows. Um all of these are just building blocks uh that you can compose together to build your application into something bigger. Um so you can see here what cloud did is it created a uh deployment um and that is right here. A deployment is takes in a agent definition. Uh in this case it's this research uh desk weekly memo using this specific agent. um it takes in a schedule uh of when it's going to run and it gives it an initial set of messages that it's going to run with. You can also pass in uh resources like uh uh memory stores or GitHub repos or files that it's going to use on each of those. Uh and so what this means is that every I think this is first of the month at 1 p.m. or something like that. It's going to run um this uh specific session. I could run one now. So I could trigger it via the console. it's going to do that. Um, but I could also build a UI for it and that's what we've done here. Um, so I can, you know, I have all the crowd. I can list and and search uh for my deployments and then I can also kick one off um at any given point. And I don't know why they're failing, but I'll ask Cloud about it later. Um, and this has come up a few times, which is what about dreaming? What about self-improvement? Um, so before I get to your questions, which I'm sure there are many, uh, I'm going to show you a preview that we've, uh, have out for for dreaming. So what dreaming is, it lets you select, um, so, okay, before I show you a previous one, it lets you select a memory store. In this case, it we'll we'll use the memory store that we've been using. You can select a model. Um, you can give it custom instructions if you want to, but we'll use a default prompt. Um, and then we're going to let it uh run uh the the last 30 days of sessions through this streaming analysis. So when I click start streaming, what it's going to do is it's gonna spin up a session using manage agents actually um spin up many sub agents to analyze the transcripts of each of the other sessions that we told it to evaluate at all of them which use the memory store. Um and it's going to analyze, hey, did the memory store encode all the information that was required? did a session uh run into any issue that uh that caused problems? Can we learn anything from it? Did this happen multiple times? Can we update the instructions in any way? You can see here what it actually did is that it found that the skill that we created actually had a few gaps um on the way that it was written and it updated the memory store to tell it even though there is this skill, you should extend this knowledge with the fact that we've learned from it. Uh and you can imagine that we will continue to extend this to not only improve memory stores but improve skills directly uh agent prompts, sub agent prompts, etc., etc. Um that's that's it. That's the demo. I sorry about the Wi-Fi. I hope you follow uh later on when when you get home. Um and let's uh keep going with some questions. \>\> Yeah, we can all take questions. \>\> We have a question over here. \>\> Yep. \>\> Go ahead. \>\> Uh thank you for the demo. Uh we didn't see any uh telemetry uh instrumentation in the agent. Is that a default native feature? \>\> What do you mean by telemetry? Because you have the full observability of all of the events uh that happened. Right? If you go into the console, you can see it live, but you can also obviously get it from the API where this transcript view is like a nice view, but if you go to debug, it's every event that occurred in that session. So you can see exactly, for example, when did we start calling the model? uh when did we finish calling the model? So you can get latency metrics in that. Is there something else that you mean by telemetry? \>\> Yeah. Uh the thing is that uh usually in software development cycle you kind of define what you want to observe. Uh how did we kind of pick what did we want to observe. \>\> Uh right now you get all the events and you just filter them on your end. If there are things that you think are missing, uh hit us up and we'll talk about adding them. \>\> Okay. And can I bring my own observability stack in here? Uh you mean like export to OTEL? \>\> Yeah. \>\> Uh you you could because it's an API you could build that connector fairly easily. Uh we just everybody has different requirements and we don't want to be opinionated but everything is just API calls. \>\> Mhm. Thank you. \>\> We have a question. \>\> Uh I I had a question. So how long can these agents run? I I I know you mentioned this dreaming thing but can you run them indefinitely or do you see that persistent store keep increasing in size and at some point you need to recompact it offline or something? \>\> We we are running compaction for you so that you can run these for days. We've seen 10-hour sessions in production uh from some of our users. \>\> So you could run these agents like for a year or indefinitely in theory. \>\> Yeah. Are you talking about running the agent for a year or you talking about like one specific session that keeps going for \>\> No, basically the agent if I had to if I had a task that I need to do every day, can I just keep it running for 6 months? Will it not blow its uh persistence? \>\> That's right. \>\> Yeah. \>\> And is there any orchestration piece like where I could have three agents, agent A does something and delegates to B and C. That's some capability that you have or \>\> for for now you would spin you would do sub agents and you would tell claude what you want. We find that uh there's a desire as humans to constrain what claude does and the way it does it. As cloud gets smarter we have continuously remove those guard rails and let claude just let it cook uh let it do what it thinks it's best. So uh right now you think maybe three sub aents is the best topology for it. But if we just let Claude figure that out, it actually ends up outperforming. \>\> But any topological or like graph based workflow thing in the UI here like where I could have \>\> Not yet. Not yet. Not yet. But but maybe soon. \>\> Uh \>\> you can probably just like speak up. Yeah. Sorry. Sorry. We'll come back to you. \>\> Sorry about that. \>\> Hey over here on your right. \>\> On my right. Okay. Yeah. So question. Um, looks great. A lot of integration points. What I'm curious about, uh, for, you know, an organ, a a complex organization that potentially manages multiple agents, um, you know, I I I think they want some sort of separation, arbback, whatever else. Um, how is that what is what is I guess the recommended approach for uh, a complex organization that has those different agents in here to effectively manage? We we have workspaces that's our primitive today where you can put different customers in different workspaces and then uh segment them that way and give access to specific users into those workspaces. Uh but we know that there's still a gap in what we offer and um that's that's something \>\> our back is on our road map. So we we do want to provide more like granular controls but right now workspace is like a clean way of like separating your agents where you have like a group of people access to workspace A and they can access all agents within that. You can have workspace B with like different set of users that can have different levels of access. \>\> So hi excellent presentation really excited. Uh two questions. One is deployment. My understanding was that a deployment is like at a snapshot a moment in time of all the agents. \>\> No \>\> deployment is essentially a way for you to run sessions for a specific agent on a schedule. So you can say like oh research desk agent go research NVIDIA once every quarter and it'll just create a session every quarter to go research and produce you that report. And then memory which is a state is persisted across all sessions. \>\> That's right. \>\> You attach your memory store to the session. So you decide which which session you want to attach which memory store too. So the memory store is like a independent entity that you just attach whenever you need to \>\> right and the final question is harnesses right one what is harnesses for you because everyone have their own definition \>\> and second is the confident right. So when I change something here, how do I know that it's not \>\> it's working as expected? Right? My own confident as a developer. \>\> Um for you're right, everybody has a different definition of a harness. Uh we when we talk about a harness in CMA, we mainly mean the tool loop execution. So managing the context and managing the session state and making sure that the agent can do the work that it wants to do. When most builders talk about harnesses, they they really mean, you know, they really mean um the ability to coordinate multiple agents and sessions together as a unit of work to achieve some sort of bigger task in a topology of sorts. Um we're giving you the abstraction of a single session as a unit of work for you to go out and build your harness uh with it as the session being one of the primitives. \>\> Um \>\> and then your second question about how do how do I know how my agent is performing? Our best recommendation would be to run evals on your agent with like different types of tasks that you want the agent to perform and see how the agent does on those tasks. Um outcomes is a really good way where we steer the agent for you to get you the specific outcome where you can define an exact rubric for like hey this is the exact outcome I want and you can define max iterations and the agent will keep going into it until it gets you that outcome. \>\> Yeah, the the the verifier agent that that Gabe talked about. \>\> Hi So \>\> after. \>\> Cool. Hey, thanks for the demo. I enjoyed it. Um my my question at the core is a bit about how to choose between skills and agents and I wanted to ask how would you all say what's like the fundamental difference? Is there an advantage to the harnessing that surrounds an agent that makes it produce better results than a scale? Um, and one of the reasons I'm asking is because I'm afraid of always investing in new kind of like I guess harnessing and tooling around models and it's it can be kind of tiring always trying to do a new thing. Um, so yeah, I just want to hear what you think is like is there a big difference and is it going to change any time? \>\> I yes, no. Um, the qu thick hardness, thin harness, thick skills um is is a question that's constantly evolving. I think what we find is that as Claude gets smarter, the boundaries and and and help that we want to give Claude start to melt away and disappear. And so what you need today might not be what you need tomorrow. The boring answer is just run evals and and evaluate these things. That sounds exhausting obviously like yeah \>\> every time a new model comes out I need to evaluate all these things. Um what what we find is skills is a good abstraction today. uh it gives the model enough \>\> information to do what it needs to do. But I presume that it will start to shift and melt and become thinner and thinner where the model will just be able to do things on its own without any special requests and instructions uh down the line. So uh multi-agent seems to be more useful uh in my opinion, but your your mileage may vary on your task. \>\> Hi. \>\> Hey. Hi. So my question ties up to last two questions and especially on top of evals. One thing I'm excited about is dreaming but how will dreaming actually affect the final outcome \>\> something that's where eval comes into the picture but you can have multiple iterations of dreaming sel that can go in different directions. So is there a plan or do you already support some automated kind of dreaming with run with evals and see that hey how to different direction go into which one should we pick based upon what's going on. \>\> Yeah. So so dreaming what it does is it looks over you you can choose the duration for which you want the agent to dream over. So you can say I want to dream over this agent session over the last 30 days and then look at all the transcripts and analyze what task you're trying to achieve and then come up with like a new memory store that's more geared towards like what you were trying to do and we typically will run evals on our end to make sure that dreaming is providing you with better outcomes and we've seen customers were able to get be get a significant improvement on their evals by running Dreaming and then using the new memory server but again everything kind of ties back to eval. So like that that's how you would know whether something is performing \>\> as you expect it. \>\> Yeah, I think it'd be good to have your own set of evals. Um \>\> the other thing is dreaming is in research preview. So these are early early. It's like a big experimental thing. We've seen uplift uh on it generally. This is we we hear you right like uh how do I gain the confidence that this is \>\> yeah it's like eval connected to dreaming and then we can find out multiple pathways that that has been taken and decide oh this makes more sense because even though nine sessions may have been saying that hey this is wrong \>\> one says that hey this is good so if you just rely on that session dreaming thing I feel like we may not be grabbing all insights without evalu \>\> if I tell you just to trust me is that no \[laughter\] Thank you. \>\> Yeah. \>\> Hi. Uh thank you very much. Um this is very impressive. Uh those of us who have been working with cloud code for a year or more um have been doing things that are similar to this but this was nice to see this in a really mature form. Um, one question I have is how much of this was just built on top of cloud code? That is if somebody was, you know, ambitious, could they could they build something similar to this? And um, what I wanted to ask you was uh, encourage you to keep things open. uh some of us are already harvesting what's in the JSON session files uh and starting to experiment with things like uh hepo and that kind of thing for for managing the um the memories and um and creating a memory pool. So uh congratulations. Thank you. Please keep it open and um can we is there are you doing anything here that's not just built on site on top of cloud code? Thank you. We we took the cloud code harness as an inspiration. We found that it was very well attuned for coding tasks. Um and different it's different levels of flexibility for other tasks. We're building an API where we want you to bring your context, your information, your capabilities and build on top of it where cloud code is trying to give you a a product out of the box. So we have slightly different uh you know value uh props um and so it's not just clocked out in a box uh though you know ideally it performs as well if not better and all sorts of tasks. Um so not not exactly but it's definitely inspired and motivated by by some of the uplifts that we have seen from cloud code \>\> and and cloud code is a terminal tool and we obviously want to be an API product that and infrastructure uh that is going to scale that's going to be reliable. So there there there were like challenges that I talked to you about um in the beginning of the presentation that we really worked in to make sure we're providing a really solid um product for you all to use. Yeah. Um, so for a bedrock, is there a console like clawed console like you have here? That's quite straightforward question, but yeah, I just want to manage and see everything. \>\> This is not available on bedrock, but it is available via cloud platform for AWS. So there is a AWS version of console that you could go use this with. \>\> Yep. \>\> Yeah. Uh first uh uh is there any uh how you say is there any restriction on the model to be used like uh non-anthropic models \>\> uh \>\> just like cloud code I can run on nonanthropic models \>\> for for now we we you \>\> yeah it's cloud models only for now \>\> okay so so the second question is what about end user use uh do they have to have a console accounts in order to use it or just a free uh I can create a customized UI for this and they can use it \>\> so cloud Cloud Console helps you with creating your API key. \>\> So that's how you get access to our APIs. So you cloud you're not paying separately for cloud console. It's free to sign up. You create an API key and then you're paying based on like token usage. So for for for the case like uh SAS like uh maybe we have multiple tenants right. So in order to provide this uh agent what kind of a uh how you say model you have already uh depict on so that we can follow for example what kind of a each company are supposed tenant are supposed to have a uh a different space or or even there is a a sub module for this uh each tenant. So are are we going to how do we actually going to organize that? In terms of this \>\> we we we have seen that the best model for multi-tenency is not bring your own key because you lose observability because it runs on your end users API key rather than than yours. So we encourage you to model things with workspaces as a tenency primitive. Um and we're improving that so that it's easier and easier to build uh on top of this for multi-tenant applications like \>\> yeah so I think currently you could either have put all your customers data into like a single workspace and then you maintain a mapping on your end on like which agent maps to which of your customers. Alternatively like Gabe said you could create a workspace per customer of yours but that can get more complicated because then you have to like recreate your agent. So, so is any you see the examples that uh people did the additional security layer or RBAC layer over the this workspace because to me it may be a little bit you know \>\> yeah that that would probably be like a good good way to do it right now because we don't have any arvback \>\> uh controls uh on our level other than like workspace because everything within your workspace is accessible to everyone in the workspace but we are working on making our back better and then making the experience better if you're working with cloud manage agents but building on behalf of your customers and how do we make that easy something \>\> so so is there any way you can show how to scale this uh if there's a multiple uh how you say tenant working on that \>\> yes \>\> any scaling parameters like uh what is the how you say the the minimal uh pod or whatever these parameters that's was yeah \>\> uh I mean you're you're defining on a you can create as many sessions as you want yeah like there's no limitation I mean we have some rate limits obviously but um but there's no scaling parameter for you to say now I need to run 10,000 pods \>\> so so currently no no such thing as like uh like HPA in the Kubernetes like you can do these kind of setting scaling uh parameters right like timeout or whatever \>\> you you don't need that \>\> you don't need to because we're managing it all for you you're just making API calls that's all \>\> okay okay sure uh thanks for your uh answer the last uh by the way I the offering link does not work for me I'm not sure if it's the only one. \>\> Yeah. Thank you for the sharing. \>\> Thank you. Thank you. \>\> Yeah. Go ahead. \>\> Hi. First, thank you very much. I've been having a ball with everything that you've been releasing and um I'm really interested in how you're using UI. Um you're spinning up a server, you're distributing this, but how might we deploy that either internally, but I'm interested in to sort of a customer base. If I want to make for example an astrologer \>\> um that person has to enter sensitive data. So it's actually really nice for me to be able to like have that in a \>\> have a layer. Yeah. \>\> So there are some procedural things you get like a star chart example. I'm not building this but it's a good example like I can have a sub agent do an interpretation that could last for like 90 seconds. It could create some assets etc. When the outcome is met it could shut down. So this gives me a way to actually just make a really clean implementation where all I need is off \>\> and then everything can spin up from there. \>\> Yeah, exactly. \>\> So there are a couple of things like MCP channels that kind of point to this with like fake chat for example, a way for you to kind of just use claude and have a channel to communicate with it through UI. But I'm seeing a lot where you're just spinning up a server inside a project that's easy to distribute internally. But how can you deploy that at scale? \>\> Um I mean we've this is uh being deployed today at scale in production. For example, notion external agents are built on top of this. So this is something that exists today that you can go and use and uh basically leverage cloud managed agents through notion in Atlassian for Jira integrations for example. Uh there's definitely ways to do it. I think if you have specific questions we could talk uh \>\> yeah we we can talk after this but typically you would probably build your product your UI that you're distributing to your customers and then calling into us for like your actual agent \>\> the phrase \>\> management everything is through the API \>\> everything that's right \>\> so you will manage all the API calls and build your product and UI around that yeah \>\> hi \>\> hi hi first thanks for a great workshop and demo and uh my question is Not really to like the agent itself. It's just like I saw the Android just released the sunet five. Yes. \>\> So I just was wondering if any anything you guys could share with us. Yeah. \>\> It's a great model. \>\> Yeah. \[laughter\] \>\> Try it. \>\> It is supported in cloud manage agents. Go give it a try. \>\> You can use it today and see. \>\> Hey there. Um hi if we've got uh business partners that are using like cloud co-work today some of them more techsavvy cloud code to build some like spot solutions for specific business cases. \>\> What do you see as sort of the level of sophistication up level to start using man like managed agents instead because this seems like more powerful \>\> but the you know cloud platform could be intimidating to folks that aren't engineers let's say. Yeah, I think the console quick start is probably like the part of my demo where you're literally just like telling Claude the agent that you're trying to build should make that pretty easy for even non-technical users to come in and build their first agent. But obviously this is an API product and is naturally going to be a little bit more intimidating. But if they're comfortable with cloud code, we do have a manage agent skill with cloud code that basically just like in natural language will walk you through building your first agent. And that would probably be my advice uh to try it out. I think the biggest difference is co-work building building personal agent. With this one, you're trying to build probably an agent that's like available to like your team, your internal organization, or you're building a product that you're shipping to your end customers. But if you're building personal agents, co-work is still a really great tool to continue doing that. \>\> Are there any skills to kind of like take something that exists in co-work? there's a skill to interact with our API. So that cloud is probably good enough. So if you say here's what I did in co and here's the skill to call the API make the translation I feel pretty confident. Yeah five we'll we'll do it for you. \>\> Um my question is about enterprise. uh there are c lot of enterprises who'd like to run these things inhouse because they have sensitive data sets that they won't want going out also they may have already set up their own secure proxies to anthropics models um GIS and also the other question was about scaling so within the enterprise now you can actually allocate compute to scale it our own way right uh so an on-prem kind of solution have you all thought about it \>\> uh definitely been a customer ask from a bunch of for enterprise customers and and we're seriously thinking considering it at the moment, but we we don't already have something that we can we can share with you, but we're we're we're seriously considering it. \>\> Excuse me. Could you show the last slide that show the QR code of the workshop? \>\> Uh are you looking for the repo or the credits? \>\> The repo. Okay. Yeah.