Part 4 of 4 of Anthropic's Claude Managed Agents workshop, recorded at the AI Engineer World's Fair 2026 in San Francisco. This closing session is an audience Q&A with Priyanka Phatak and Gabe Cemaj covering operational and architectural questions: keeping agent definitions in YAML and syncing them in CI/CD with the ant CLI, handling messages while an agent waits on a custom tool, Anthropic's eval practices, sharing resources across an org/workspace, reviewing agent-written code, tenant isolation, serving multiple customers from one agent, GPUs via self-hosted sandboxes, modeling a multiplayer Slack agent, long-running sessions, and a coordinator pattern that keeps the main agent free while a subagent handles custom tools. # Video Identity Observations — Claude Managed Agents Q&A (Part 4 of 4) ## Intended Audience This video addresses a **developer and enterprise-builder audience** in a live, in-person Q&A format: - Attendees ask about **CI/CD integration**, eval harnesses, and Terraform-like YAML workflows: *"you can put your long-term resources in YAML in your code base and then build CICD."* - Questions come from practitioners at **enterprise companies with multi-tenant needs**: *"if I have 100 customers... can I have separate Claude-managed agent instances per company?"* - Attendees reference hands-on workshop materials: *"I'm looking through a repo... you had like a skill that did workshop"* — indicating the audience just completed a technical workshop. - Questioners use professional engineering vocabulary (PRs, sub-agents, GPU allocation, sandbox specs, data segregation), confirming a technical crowd. ## Claimed Scope The video covers **Anthropic's Claude Managed Agents (CMA) platform**, positioned as a managed offering for building and operating agents: - **Product scope**: infrastructure for agents, memory stores, MCP channels, sessions, and custom tools. - **Deployment model**: cloud-managed, not on-prem — *"We're not supporting on-prem."* - **Multi-tenancy flexibility**: *"You can define an agent per customer... or you can use a workspace per customer if you want true isolation, you could do an org per customer."* - **Platform limits acknowledged candidly**: no fine-grained resource-level sharing controls (*"We don't have fine-grained controls"*), uniform sandbox compute (*"today [the provided sandboxes] are all the same"*), and messaging blocked during custom tool waits (*"You'll get a 400"*). ## Self-Description Evidence Creator statements about the channel/organization: - **Insider model access**: *"We have access to the models before you all do obviously"* — positioning the speaker as an Anthropic team member. - **Internal dogfooding**: *"So, I assume you guys use the same internally too." — "Yeah, that's how if we need to build durable long-term resources, we just can check them in."* - **Managed-offering commitment**: *"we as a managed offering want to make sure we do is that we update our assumptions about the harness as early as possible."* - **Ecosystem orientation**: references to *"our Code with Cloud conference"*, cookbooks, docs, and third-party integrations (Modal, Daytona) as the extension path. ## Demonstrated Presentation Patterns - **Live, unscripted Q&A format** with multiple Anthropic speakers fielding audience questions directly. - **Honest disclosure of limitations**, including open research questions: *"now we've removed the boundary of typing and exploded the number of PRs we can make per day... how do we keep up? And we're still exploring that."* - **Advisory rather than prescriptive guidance** on evals: *"I know it's not an out-of-the-box solution, but it generally the advice that we've seen works"* — plus the caution *"garbage in, garbage out."* - This is **Part 4 of 4**, indicating a multi-part serialized series tied to a workshop/conference event. ## Notes for Synthesis - The material demonstrates a **developer-education and product-transparency channel identity**: technical depth, live audience engagement, and candid limitation-disclosure as recurring traits. - Care should be taken not to generalize this single event video into conclusions about the channel's full content strategy. # Delivery Observation — Claude Managed Agents Q&A: Evals, CI/CD and Multi-Tenancy (Part 4) ## Format & Setting - This is a **live audience Q&A**, with multiple speakers alternating — at least one primary answerer, plus one or two colleagues who jump in with supplements. Turn-taking is frequent and conversational: > "Uh yeah, we run a lot of evals…" (main answer) followed by a colleague adding: "And you can probably look at like your historical sessions…" - Speakers acknowledge questioners warmly and briefly: *"Hey, thanks for sharing."* / *"Thanks again."* / *"Thank you. Thank you very much."* ## Tone - **Casual, conversational, hedge-friendly.** Answers open with fillers and hedges: *"Uh so…"*, *"I think…"*, *"sort of"*, *"like"*. - **Direct when facts are crisp.** Short declarative answers appear when the question has a clear answer: > "You'll get a 400." > "We're not supporting on-prem." > "They're bound to your workspace." - **Comfortable admitting open problems** rather than overselling: > "…now we've removed the boundary of uh typing and exploded the number of PRs we can make per day, hour, second. Um how do we keep up? And we're still exploring that." > "I know it's like not out of the box solution, but it generally the advice that we've seen works." ## Vocabulary & Recurring Phrases - **"Basically"** is the most frequent connective: *"a CLI that basically wraps our API"*, *"We basically pause execution"*, *"basically synchronize them"*. - **"So" as an answer-launcher**: nearly every substantive reply begins *"So, …"* or *"Uh so, …"*. - Architecture vocabulary repeats across answers: **workspace(s)**, **session(s)**, **agent**, **mapping** ("maintain that mapping of like which customer maps to which agent"), **tooling** ("there's basically tooling to enable you to do each of each of those things"). - Recurring optionality framing: *"It's however you choose to define it"*, *"There's many ways to do it"*, *"So it depends on how you want to set up"* — presenters repeatedly present **2–3 options** rather than one prescription. - Concrete filler-flavored examples: *"like let's say customer ID"*, *"like rebuild blank"*, *"or something like that"*. ## Teaching Habits - **Acknowledge → structure → example.** Typical arc: validate the question ("that's a great question"), name a tool or pattern, then walk through mechanics: *"we have something called Ancho CLI… you can put your long-term resources in YAML… and then build CICD to update…"*. - **Enumerated options for design questions.** On tenant isolation: *"You can define an agent per customer… or you can use a workspace per customer… you could do an org per customer."* - **Colleagues supply the concrete example** after a main answer. After the multi-tenancy answer: *"you can like make a copy of across every workspace… or you use a single agent and then have the mapping…"* - **Hands-on, product-anchored references** to slides, docs, cookbooks, and the live UI: *"you can see that here"*, *"Yeah, you can find that in our docs"*, *"there was an ask to see the QR code slide one more time."* - **Invitation to continue offline** bookends the session: *"If you're if you're shy…"* and *"we'll be just outside if you want to come talk to us."* ## Communication Patterns - **Tag-checking comprehension**: *"Does that make sense?"* - **Collaborative overlap** — speakers finish or reinforce each other's answers rather than disagreeing; frequent *"Yeah"* and *"Exactly"* confirmations. - **Answer-then-extreme-case**: claims are backed with scale or vividness: *"spin up 10,000 sessions to do this all in parallel"*, *"it will just mercilessly iterate until it matches the the rubric."* - **Metaphor with acknowledgment of limits**: *"garbage in, garbage out"* for eval data quality; *"CMA is part of the answer, I think."* ## Stable Patterns vs. Isolated Moments - **Stable (repeated across the video):** "So/ basically" openers, option-enumeration answers, honest "we're still exploring" framing, and the collaborative two-speaker supplementation rhythm all recur throughout. - **Isolated (thin evidence):** the humor register appears only once or twice (*"if you're if you're shy"*, *"mercilessly iterate"*), so it can't be confidently called a consistent trait. Similarly, the *"Does that make sense?"* check occurs once; whether it's habitual is unverifiable from this transcript alone. # Claude Managed Agents Q&A: Evals, CI/CD, Multi-Tenancy (Part 4 of 4) — Observed Positions **Speaker note:** The transcript contains no curated speaker metadata and turn-taking is unattributed. Speakers are identified below only as **Audience** (question askers) and **Anthropic presenter(s)** (respondents), per explicit conversational evidence. Multiple presenters appear to alternate; individual attribution among them is not possible from the transcript alone. --- ## 1. CI/CD for Agent Resources **Audience question:** How do you handle CI/CD for agents? **Anthropic position:** - They offer **Ancho CLI**, a CLI wrapping their API that enables "Terraform-like" workflows: agent/resource definitions are declared in **YAML** and checked into a codebase. - CI/CD pipelines can then update the resources via Ancho CLI on every change. - Qualification offered: this is not hypothetical — presenters stated they **use the same approach internally** for "durable long-term resources." --- ## 2. Messaging During Custom Tool Waits **Audience question:** Can you send messages while the model waits for a custom tool result? **Anthropic position:** - **No** — execution pauses during a custom tool wait; message sending is **blocked for ordering purposes**. - If a message is sent anyway, the caller receives a **400 error**. - Distinction preserved: while the model does *other* work, messages **can be queued**. --- ## 3. Eval Harness Quality and Best Practices **Anthropic position on internal evals:** - They run "a lot of evals," especially as new models and capabilities arrive; as a managed offering they have **early access to models** and update harness assumptions proactively. - Internal tooling runs datasets against sessions; because everything is cloud-based, they can **spin up ~10,000 sessions in parallel**, then evaluate, tune prompts, and adjust assumptions per model release. - Claimed scope: evals cover "all sorts of things for generic purposes, not just coding." **Advice to customers (with explicit hedge):** - Best advice: **get high-quality evaluation data** — "garbage in, garbage out." - Tentative admission: some of their own initial eval ideas were "ill-defined," yielding **100% scores that were meaningless** in performance terms. - Acknowledged limitation: this is **not an out-of-the-box solution** ("bring your own solution" framing). - Suggested technique (second presenter): score **historical sessions** to build a dataset. --- ## 4. Resource Sharing Across an Organization **Audience question:** Can agents, memory stores, and MCP channels be shared across departments? Are they account-bound? **Anthropic position:** - Resources are bound to the **workspace**, not the individual account; any API key in the workspace can access them. - Access control model: **org → admin → workspaces**; the admin decides who belongs to which workspace, and workspaces should be designed around desired access-control boundaries. - **Explicit limitation:** there are **no fine-grained sharing controls** (e.g., share the memory but not the MCP channel) at the platform level. - Workaround offered: builders/integrators can implement **partial sharing themselves at the application layer** "pretty granularly." --- ## 5. Code Review Bottleneck with Many Agents **Audience question:** Agents generate lots of code; how do you review and merge it? Is "deploy every 30 days" tested? **Anthropic positions:** - The original workshop topic was exactly this: **building agents to review code**. A Claude Code (CMA) integration can trigger **multiple sessions with sub-agents on each PR** to evaluate changes and find bugs. - Internal claim: these agents "do a lot of the uplift," reducing per-PR time and bugs while keeping quality high. - **Explicit open problem:** removing the typing bottleneck has exploded PR volume — "how do we keep up? And we're still exploring that." One presenter added: "CMA is part of the answer, I think" (tentative). - Supplementary suggestions: - Iterative review loops (review → comment → review), but **you still need to run the thing**; deployments/custom deploy agents help test per-environment. - Sandboxed self-validation: Claude can run the app and **act as a QA agent**. - **Limitation stated:** no tunneling to interactive UI sessions today; Claude drives the browser and can return **a video** of validation. --- ## 6. Multi-Tenancy and Tenant Isolation **Audience question:** Can an enterprise get separate Claude-managed agent instances per customer? **Anthropic positions:** - **On-prem is not supported** (stated directly). - Isolation granularity is "however you choose to define it": agent per customer, **workspace per customer**, or **org per customer** for "true isolation" — with the builder responsible for the customer→agent mapping. - Sharing one agent definition across customers: options include (a) YAML definition in code with **CI tooling to synchronize to multiple workspaces**, (b) a single agent across all customers with a **session→customer mapping** via a **metadata field (e.g., customer ID)** on sessions, or (c) template copies per workspace (system prompt, model name, tools). - **Data segregation:** if segregation matters (raised in context of legal requirements), resources **must go in individual workspaces**; resources are allocated per workspace. --- ## 7. Custom Compute for Sandboxes **Audience question:** Can agents get more memory or GPUs? **Anthropic position:** - Provided sandboxes are **all the same today** — no custom resource allocation. - Recommended path: **self-hosted sandboxes** via providers like **Modal** (which has GPU offerings); first-party integrations and cookbooks exist for Modal, E2B, Daytona, and others. - Environment specs and integration cookbooks are stated to be in the docs. --- ## 8. Building a Claude-Tag-Like Multiplayer Chat Agent **Anthropic positions:** - Feasible today: map **thread → session**; inject each user message with attribution ("this is coming from Gabe..."). - **Shared memory across sessions** so the agent learns from all participants; give it explicit reply tools (e.g., Slack post tool). - Design caveat: don't mirror all agent output — let the agent **decide explicitly when to post**. - For Claude-Tag specifically: **one agent per Slack channel**, each with separate memory scoped to that channel's messages. --- ## 9. Long-Running Sessions and Checkpointing **Audience question:** How to run multi-hour (6+ hour) agents, and how to check status? **Anthropic positions:** - The **outcome system** is "the best way" for long sessions: define exit criteria; Claude "will just mercilessly iterate until it matches the rubric." - Claim: nothing special is needed for sessions beyond six hours — "it should just keep going." - Observability: via the **session event stream / events API**; alternatively a custom tool like `update_status_for_human` that Claude calls when it perceives progress. --- ## 10. Handling Messages During Sub-Agent Tool Waits (Coordinator Pattern) **Audience follow-up:** If the main agent blocks on a custom tool (human-in-the-loop), how do you stay responsive? **Anthropic positions:** - Sub-agents **can** have custom tools; pattern: a **coordinator agent** delegates to a sub-agent (agent B) whose definition holds the custom tool; the caller populates tool results on that thread. - Threading detail: the sub-agent runs on a **technically different thread** within the **same core session**; you interact only with the main thread, and routing happens by ID. - Idle semantics: the **main thread can be idle while sub-agents work or wait on tool results**; the session is not idle if a sub-agent is working — illustrated from a demo on screen. - Summary advice (hedged): "if you're trying to do something where you want the main agent to be always chatty... the coordinator pattern works pretty well for that." --- ## Tensions and Open Items Preserved - **Fine-grained sharing is not supported** at the platform level despite a direct request; the answer pivots to builder-side implementation. - **Code review at agent scale is acknowledged as an open question** ("we're still exploring that"), with CMA offered only as "part of the answer." - **Evals are explicitly not turnkey**; the presenters admitted their own early eval designs produced misleading 100% scores. - **No on-prem, no sandbox tunneling, uniform sandboxes** — three stated current limitations. - Data segregation guidance (per-workspace) was given without the presenters confirming whether the underlying requirement is legal in nature; they directed the questioner to the per-workspace solution regardless. # Video Attribution & Provenance Record **Video:** Claude Managed Agents Q&A: Evals, CI/CD and Multi-Tenancy (Part 4 of 4) — Anthropic **Format:** Live Q&A session (Part 4 of a 4-part series) ## Speakers & Roles - **Anthropic representatives (hosts/presenters):** Multiple unnamed speakers answering audience questions. Roles are identifiable from context only — they describe internal Anthropic practices ("we have access to the models before you all do"), internal tooling, and product capabilities. Individual names are **not provided** in the transcript. Roles: product/engineering staff answering questions, including one who demos an "outcome system" and session UI. - **Audience members (questioners):** Multiple unnamed attendees asking questions. No names given; identifiable only by question topic. > Transcript uses only `>>` markers; speaker identity mapping between turns is inferred, not explicit. ## Topic Segments (who said what, at a glance) - **CI/CD with Anchor CLI** — Anthropic rep: YAML-based Terraform-like resource management; confirmed used internally. - **Message queuing / 400 error** — Rep: messages blocked while awaiting a custom tool result; sending yields a 400. - **Evals** — Reps: internal eval harness with parallel cloud sessions (~10,000); advice: high-quality data, iterate on datasets, use historical sessions. "Garbage in, garbage out." - **Multi-tenancy / sharing** — Reps: resources bound to **workspace**; org admin controls workspaces; no fine-grained partial resource sharing (builders control granularity); per-customer isolation via workspace-per-customer or org-per-customer; **no on-prem support**. - **Code review at scale** — Reps: Claude-built code-review agents on PRs; sandbox self-validation/QA; no tunneling available today. - **Compute customization** — Reps: use **self-hosted sandboxes** (e.g., Modal for GPUs); docs/cookbooks cover Modal, EB2 (as transcribed), Daytona. - **Slack/multiplayer chat agent** — Reps: per-thread session mapping, injected user identity, shared memory, one agent per Slack channel pattern. - **Long-running sessions** — Reps: outcome/exit-criteria system; sessions run past 6 hours; observability via session event stream / events API; custom "update status" tool suggestion. - **Sub-agents & idle coordinator** — Reps: custom tools on sub-agents; sub-agent threads within same core session; main thread can idle while sub-agents work. ## Referenced Sources & Claims - **Ancho CLI** (Anthropic CLI wrapping the API, YAML-based resource management) — speaker claim. - **Code with Cloud conference** (London, ~1 month prior) — workshop talks stated to be on **YouTube**; repos have README files. - **QR code slide / credits slide** — shown again on request; content not described. - **Third-party providers named:** Modal, EB2 (per transcript), Daytona — sandbox/cookbook integrations. - **Internal practices claimed by speakers:** internal eval tooling/datasets; agents doing internal code review "uplift"; 30-day deployment cadence (audience-stated, speakers did not confirm the figure). - **Product limitation claims:** no fine-grained resource sharing; no tunneling; no on-prem; 400 error on messages during custom tool wait — all speaker statements. ## Promotion & Availability - **Promotional elements:** References to docs, cookbooks, YouTube conference talks, and workshop repos function as resource pointers; the session itself is Anthropic product content (Claude Managed Agents). No explicit sales pitch or paid offering flagged in transcript. - **Comment availability:** No information about comments or where the video is hosted is present in the source. ## Uncertainty Flags - Speaker names and exact role titles (e.g., engineer vs. product manager vs. devrel) are **not established** — only "Anthropic representatives" vs. "audience." - "ICD the agent convex" and "Cloud Tag"/"Claude tag"/"CMA" appear to be transcription artifacts (likely CI/CD, Claude Code, and Claude Managed Agents); noted as transcript noise, not verified. - 30-day deployment cadence is an audience member's characterization, unconfirmed by speakers. \>\> All right, we have a question here. \>\> Hey. How do you see ICD the agent convex? Um \>\> Ah, that's a great question. Uh so, we have something called Ancho CLI, which is a CLI that basically wraps our API to make calls um and allows you to do sort of Terraform-like things where you pass in YAML. So, you can put your uh long-term uh resources in YAML in your code base and then build um CICD to basically update the Ancho with Ancho CLI using those YAMLs uh every time you make a change to your to your resource. Does that make sense? \>\> Got it. Uh so, I assume you guys use the same internally too. \>\> Yeah, that's how if if we need to build durable long-term resources, we just we can check them in. \>\> All right, thanks. \>\> Thanks. \>\> Um so, I think you mentioned you can send messages whenever you want. Um can you still send messages if the model's waiting for a a custom tool result? \>\> Uh not when it's waiting for a custom tool result. We basically pause execution uh at that time. Um but if the model is doing other work, uh then we do allow you to queue up messages. Um but when it's doing a custom tool result, we're expecting you to send that um so, because of ordering purposes, we we block the calls. \>\> Well, what happens if you do send the message? \>\> You'll get a 400. \>\> Okay, that makes sense. \>\> Uh there was an ask to see the QR code slide one more time. \>\> The uh \>\> The uh the credits one, yeah. \>\> Any any other questions? If you're if you're shy, we \>\> Yeah, we'll hang out after for any any questions you don't feel \>\> Yeah, I'm looking through a repo. There's a lot of interesting workshop talks. Is there anything that's missing from that like slides that would help us go through other \>\> Yeah, yeah. \>\> sessions like that you can post or is it pretty much self-explanatory through the the readme files you just like have you know cuz you had like a skill that did workshop. \>\> Yeah, yeah. The other workshops came from our Code with Cloud conference. Most of which you can find online for the version we did in London about a month ago. \>\> Yeah, all all the all those talks are on YouTube if you're interested in the other repos that are in there, but this one should give you like a pretty good overview and it's pretty similar to what we've done at other conferences. \>\> Yeah. \>\> Yeah, so just going back to evals, can you talk a little bit about you know how you guys are maintaining quality, best practices around the the eval harness? \>\> Uh yeah, we run a lot of evals especially as new models come up as new capabilities come up. We have access to the models before you all do obviously. So one of the things that we as a managed offering want to make sure we do is that we update our assumptions about the harness as early as possible so that we can remove things that are no longer needed and we do all of that in a eval base. So we have internal tooling that basically looks like different data sets that we can then run our different sessions. It's because it's all in the cloud it's easy to say like spin up 10,000 sessions to do this all in parallel and then we evaluate those results and tune prompts and tune assumptions, remove things, add things to make sure that for every model release we're doing the best we can and we're evaluating all sorts of things for generic purposes not just coding. \>\> So so how does that extend into the agents that we would build? Is there a recommended path for for evals or is there kind of just bring your own solution? \>\> Uh I think the the best advice we can give you is to get high quality data to evaluate over. I think Claude Claude code is pretty good at building an eval for you, but garbage in, garbage out. We've seen that some of our initial ideas for how to evaluate things were sort of ill-defined and you get 100%, but what does that mean in performance? So, iterating on your data and making sure that you're setting up the problem correctly. Um I know it's like not out of the box solution, but it generally the advice that we've seen works. \>\> And you can probably look at like your historical sessions and like what results you got and then sort of try to score it and see how the model's doing. Um that would be a good way of like building your data set looking at your historical sessions. \>\> I have a question over here. \>\> Hey, thanks for sharing. Uh I have a question about how to share those agent and all the resources across all the the entire organization. The reason I'm asking is because we use Claude as code quite a lot. We're building shared skills. We're building shared connectors. But for the everything you shared today, for example, agent, memory store, or MCB channel, can we actually share them or can we partially share them across the organization to different departments? So, what is boundary? What is the privilege about those resources? Are those resources bonded to my account or how can I control this? So, what's the what's the way to do that? \>\> They're bound to your workspace. So, any API key within your workspace can have access to the resources that \>\> So, can my can my teammates edit or modify what I'm building or is Yes, completely controlled, right? \>\> So, I think your Claude platform admin, so you'll have an org for your organization. And then your admin will be the one setting up the workspaces and deciding who gets access to which workspace. So, I think \>\> you would basically design your workspaces depending on your access control. \>\> Correct. \>\> You'd probably want to do like wh- whatever group of people you want access to resources X, put them in one. \>\> So, you can share whatever you want, even partially of the, you know, you build an agent that can do something, but I only want to share part of that. For example, I share the memory to someone that share the MCP channel to the others. Is that possible? Or I need to share the entire tech stack of what I'm building? \>\> We don't have fine-grained controls of like this to share this or this not, but you as a as a builder and integration would be able to control that at pretty granularly so that your end users can have access to partial sharing. \>\> Cool. Sweet. Thanks very much. \>\> Thank you. \>\> Yeah. \>\> Yeah, so basically we can generate a lot of code with a lot of agents, and then we're getting to the bottleneck of uh reviewing that code, making sure you can actually merge it. Um how do you guys think about that in terms of are you deploying like I see you deploy every 30 days, which uh might be a different definition of deployment, but is it deploying something you can actually test to make sure it works well, or how are you reviewing uh this many lines of code from all the agents? \>\> So, the the original workshop that we wanted to do was actually answering that question. How do you How do you build agents to review code? Um what we found is you can build an integration with CMA so that on PR, it kicks off one, two, three, many sessions with many sub-agents to evaluate what has changed and find bugs. Internally, we we have uh these sorts of agents do a lot of the uplift for us, and we've seen that that enables us to uh reduce the amount of time we need to spend on a per PR basis and reduce the number of bugs and keep the quality uh high. Um but this is partially an open question of now we've removed the boundary of uh typing and exploded the number of PRs we can make per day, hour, second. Um how do we keep up? And we're still exploring that, but \>\> Yeah. \>\> CMA is part of the answer, I think. \>\> Yeah, and you can iterate in a loop, you know, to like review, add comments, review until it's done, but then at some point you still need to run the thing to make sure it works well locally. So, I think that's where the deployments can come in, where you can um you know, auto deploy something that you can test with a custom deploy agent, like you said. Um it just depends, like if you have different environments for each branch, but \>\> You you can run your application in the sandbox as well and have Claude self-validate the changes that it made. Um and make sure that like it acts as a QA agent as well. \>\> Are these like long-running sessions that you can go in and actually play around on the UI? \>\> We don't have uh kind of tunneling today, \>\> So, you would have Claude go in there, play around with the browser. \>\> That's right. And then \>\> and validate it. And then maybe even give you back a video, maybe. \>\> Yep, exactly. \>\> Cool. Thanks. \>\> Any other questions? \>\> Hello. \>\> Hi. \>\> Uh for an enterprise company, if at all we want to maintain true tenant isolation that we want to support, I heard that you are not supporting yet the enterprise uh company uh use cases, probably. \>\> We're not supporting on-prem. \>\> On-prem. \>\> Yeah. \>\> But what if I want to have uh let's say I have 100 customers that as an enterprise company, can I have separate uh Claude-managed agent instances \>\> Yeah. \>\> per company of ours that we want to support? Is that a viable option that you support? Is it only one instance that we get, or can we get one instance per company that we support as an \>\> It's however you choose to define it. You can define an agent per customer, and then that way the data will be segregated, or you can use a workspace per customer if you want like true isolation you could do an org per customer. So it depends on how you want to set up. Your agents and you would probably want to maintain that mapping of like which customer maps to which agent. \>\> Okay. I think I have a follow up question. So just to given the scenario that we have two customers and we want to build the agent once but the agent going to be shared by the two customers. How to achieve that? \>\> There's many ways to do it. You could do it through automation. So you could have your agent definition live in Yamo in your code and then build tooling to update it to both of your workspaces and basically synchronize them or you can model everything in a single organization and then do the the modeling that way. Uh \>\> You could create like a single agent across all your customers and then you manage the mapping of like which customer is using which session. \>\> Yeah, then in in which workspace I can accurate this agent? \>\> Yeah, if if you do it per workspace then you need we give you tooling to synchronize a single definition in your code to multiple places so that whenever you update it in your code through CI you can update all of the definitions at once. \>\> So you can either make a copy of your agent which is essentially a template of system prompt model name tools and you can like make a copy of across every workspace. Let's say you go down like the one workspace per customer or you can just put all your customers in a single workspace and then you maintain the mapping of which either you do again have like copies of your agent or you use a single agent and then have the mapping of like which session maps to which customer. We have like a metadata field on the session where you can put in like let's say customer ID so you know which session what map to which customer. \>\> Okay, that you know like a putting all the customer data in one workspace, how do we satisfy the data segre- segregation \>\> Yeah. \>\> requirement, you know. \>\> Yeah, if you if you care about data segregation, then you need to put them in individual workspaces, and the resources are allocated per workspace. But, there's basically tooling to enable you to do each of each of those things. \>\> And when you say data segregation, like is is that like a legal requirement, or is that something just you want to \>\> Yes, so \>\> something legal, like customer doesn't want to share data with other customers, you know, they want to dedicate \>\> Yeah, you would do it per workspace. \>\> Okay. Okay. Thank you. \>\> I saw a question here. You have to speak up, or \>\> Um Is there a way to modify the compute environment, meaning uh in some cases, I may need to have higher memory for my agent to do the work, or I need a GPU for my agent to actually do the work. Is it supported in this, or \>\> To to the Today, if you want to do custom resource allocations for your sandbox, you would use self-hosted sandboxes. You would use something like Modal that has GPU offerings. And we have cookbooks and kind of first-party integration, so that you can hook up your Modal instances to run your agent loop inside of Modal instead of our provided sandboxes, which today are all the same. So, if you care about different scale, you would use a different provider for that. \>\> And do you have like a list of specifications of what the environment these agents have access to? \>\> Yeah, you you can find that in our docs, and you can find cookbooks to integrate with third-party providers as well in our docs. \>\> Yeah, we have a bunch of cookbooks on like how to work with Modal or EB2, Daytona, whatever provider that you want to use for sandboxes. Yeah. Yeah. Hi. \>\> Hi. \>\> Thanks again. So, out of curiosity, how to build a sim- version of Cloud Tag using CMA in a different chat channel, especially for the multiplayer mode. \>\> Uh this definitely something you could do today where you would on a per thread level map it to a session and then anytime a user messages that session, you inject that as a user.message letting the agent know, hey, this is coming from Gabe or if it's coming from Priyanka. And so everybody can message that agent independently. You can have it share memory across sessions so that it learns from each other. You can give it tools to reply back using like the Slack post tool or something like that so that the agent can decide when it wants to post. You don't necessarily want to just mirror whatever the agent is responding. You want it to be explicit. Uh but you could definitely model this using CMA and \>\> And if you want it to be like Cloud Tag, you'd probably do like one agent per Slack channel and then every agent has its separate memory and only has access to messages within that Slack channel. \>\> Yes. \>\> Hey, thank you for the presentation. Very useful. Quick question in terms of um looping and doing long runs um in this Cloud Console, how do you see that? Um is this something you have like some specific module that I I'm not finding or um are you you're planning to develop that um more in the future? Um thank you. \>\> Do you mean how do we provide the observability for really long sessions or \>\> No, no, no. Um so more focus on um using this platform to build those uh long-running agents like hours and hours like last 6 hours for example. Um is this something that is already embedded in what you have built in the agents or I'm more curious to know if you are developing something specifically for that or how you see that trend. \>\> The the outcome system that I demoed is the best way to do really long sessions because you define an exit criteria and Claude will just keep going until it meets that criteria. And it could be a really long task like you know, rebuild blank and it will just mercilessly iterate until it matches the the rubric. \>\> There there there's nothing special you would need to do to get the session running longer than six hours. It should it should just keep keep going. \>\> One quick follow-up. So, if I run at the agent for hours and hours and I want to understand like checkpoints status, the best way to that will be through the session tab. \>\> Uh yeah. \>\> be the session event stream. Yeah, there's the events API on under the session. \>\> Okay. \>\> You could have a custom tool called like update status for human or something like that and Claude would on intervals whenever it feels like it's made progress call that tool and give you a status update. \>\> Thank you. Thank you very much. \>\> All right, any other final questions? Yes. \>\> Um following up on that previous question about using this for Claude tag or just similar functionality. I've been uh analyzing a whole bunch of uh agent frameworks. So, if you get a message while the um agent is waiting for a custom tool result for example, human in the loop, um you said it would 400. Do you have uh any suggestions on how you would deal with that? \>\> Um \>\> Can you do custom tools on a sub agent? I forgot. \>\> You you could do custom tools on a sub \>\> Cuz then cuz you could basically have like a sub agent that's handling your custom tool and then that way your main kind of coordinator agent's always free for receiving. \>\> In the API, can you directly return a custom tool result for the sub agents tool. \>\> Well, the sub agent would emit the custom tool use and then you would populate it back to that. \>\> you you define like your main coordinator agent and then you can say agent A can talk to agent B and C and then agent B is the one let's say processing your custom tools and your agent B definition is where you'll define your custom tool and then that's the session thread that you would listen to get the \>\> Is that a different session thread or the same? \>\> No, it's it's part of like the same core session. \>\> a different thread technically, but you only interact with the main thread and we correctly delegate it cuz we know where it needs to go based on IDs. Um \>\> Okay. \>\> And then the main agent will pass it along to the sub agent. \>\> So the session is still idle even though the \>\> If if if a sub agent is doing work, it's not idle. It's \>\> But like if the main coordinator agent is idle the sub agents could still be \>\> still doing work. Yeah, you you can see that here. \>\> Okay, that's pretty impressive. \>\> This uh gap here, the main session is is idle. The main the main thread is idle while the sub agents were doing work. \>\> And they could be waiting for tool results. \>\> Exactly. Yeah. So if you're trying to do something where you want like the main agent to be always chatty, then I think the coordinator pattern works pretty well for that. \>\> That's awesome. \>\> Yeah. \>\> I think we're we're at time, but we'll we'll be just outside if you want to come talk to us. Uh thank you for having us. \>\> All right. Th- Thank you so much. \>\> \[applause\]