Conference talk by Tisha Chawla and Susheem Koul (Microsoft) on FinOps for AI agents: building a cost control plane at the layer where application code calls a model. Covers why API gateways alone are insufficient, an SDK design with annotations marking spend boundaries and a governor of authorized actions, and a control plane that groups runs into segments with time-windowed budgets and policies. Actions include halt (circuit breaker) and steer, where a cost guard predicts budget overruns from spend rate and injects instructions to keep outputs succinct. Includes a preview-mode demo, benchmark results from two open source repos (about 78% lower agent spend and completion rates rising from 67% to ~96% with the full policy suite), and a discussion of a self-learning control plane. # Channel Presentation Observations: FinOps for AI Agents ## Video Identity - **Title:** *FinOps for AI Agents: Who Spent All the Tokens?* — presented by **Tisha Chawla & Susheem Koul, Microsoft** - **Format:** Conference-style technical talk with slide walkthrough, architecture explanation, live demo, and Q&A invitation — evidence: "let's just quickly walk over to the demo," closing "if you guys have any questions... just let us know." ## Explicit Self-Description / Creator Claims - The speakers identify as **Microsoft employees** (per title metadata); the transcript itself names only first names and affiliations to the product. - They explicitly frame the talk as a **proposal, not just a pitch for an existing product**: > "we'll instead talk about the first principles of how we can design a system which is actually true enough to solve the problem from the very root." - They name and describe their own system: **"Token Ops," "a runaway token governance for AI agents."** - They claim a **benchmark result**: "the average spend goes down by almost 78% with Token Ops enabled" and completion "from 67% to roughly 96%," benchmarked on "browser use as well as meta GPT." - They point to a **public wiki** as an ongoing resource: "you can scan this QR code, that's the public wiki. We are updating it almost regularly." ## Intended Audience - **Developers and platform engineers building production AI agents**, evidenced by: - Framework-specific framing: "it doesn't matter what framework you're using, you might be using... LangChain, LangSpout." - Code-level integration talk: "you just have to apply the annotation on the methods that you have." - Production-safety framing: "if you want to include this product into your production agents, you want to have a safe environment." - **FinOps / cost-governance minded practitioners**, evidenced by the topic framing: "when you opened an AI bill... you couldn't actually trace back where that bill was actually coming from." ## Claimed Scope of Content - **Topic:** Token cost management for agentic AI workloads — "the shift from token maxing to value maxing." - **Coverage demonstrated:** - Historical framing (SaaS → cloud → agentic eras and their control surfaces). - First principles: token as unit of cost, cost created "at the LLM... model call boundary," attribution, in-place policies, budget caps as last resort. - Architecture: out-of-band control plane with **instrumentation, accounting (ledger), and enforcement** modules. - Live demo with three scenarios: **preview mode** (policies without enforcement), **halt** (circuit-breaker kill), and **steer** (cost-guard instruction injection). - Competitive positioning against existing tools: "the tools like this light at a lamp port key, cloud flare... happen at again the request level" — a creator claim about the landscape, not independently verified in the material. ## Observable Presentation Patterns (Distinct from Claims) - **Duo-presenter format** with handoff: "with this I would like to hand it over to Sachin for the demo." - **Demo-driven credibility approach**: they show a "simple two-agent workflow" (research agent + summarizer) rather than describing results only. - **Forward-looking roadmap framing**: "a self-learning module within the control plane, which can look at this ledger" — presented as vision, explicitly "not the end state that we envision." - **Privacy/tenancy reassurance** as a selling point: "the control plane lies in your own tenant. So, you do not need to worry about any data leaks." ## Notes on Source Limits - No curated metadata was supplied; observations rest on title and transcript only. - The 78%/96% figures and landscape comparisons are **creator claims** from the talk; no external verification is present in the material. # Delivery Observation: "FinOps for AI Agents: Who Spent All the Tokens?" ## Overview Two-presenter conference talk (Tisha Chawla on principles/architecture, then co-presenter on design and demo). Delivery is conversational, demo-driven, and structured around a repeated principle → architecture → demo pattern. ## Tone - **Casual, talk-track register** throughout, with frequent filler: "um," "you know," "like," "right?" > "So, I'm Tisha and I have Sushil with me as my co-presenter." > "But um and I don't think that's a problem right now because right now the industry is valuing, you know, uh token maxing" - **Playful, hook-style framing** in the opening — the title question is reused as a rhetorical device: > "we'll be talking about the most expensive question in AI today." > "who spent all the tokens?" - **Light humor** via coined terminology: > "people are proud to call themselves token millionaires and I think that's all right." ## Recurring Phrases & Rhetorical Habits - **Tag questions and confirmation-seeking "right?"** — extremely frequent across both speakers, closing nearly every explanatory clause: > "you couldn't actually trace back where that bill was actually coming from, right?" > "cost is created at the LLM like the model call boundary. So, that is what we'll have to track." > "the control plane is where the mind of the system lies, right?" - **"you know" as a connective filler** — dozens of instances, used to bridge clauses mid-sentence. - **Restatement/self-repair**: speakers often restart or rephrase mid-sentence ("the the spawning," "which is the which is where we are trying to add value"), suggesting an unscripted, improvised flow rather than a read script. - **"So" as a topic-shift marker** — used to open new sections by both speakers: > "So, for that let's like dive onto the principles." > "So, let's talk about the bridge layer very briefly." > "So, before we move on to the demo, I'll just briefly touch upon the test bench" ## Teaching Habits & Structure - **Historical analogy teaching**: frames the problem via prior eras (SaaS → cloud → agentic) before introducing principles: > "when we talk about the SaaS era, the interface was UI and the control was in the form of usage caps" - **First-principles framing** explicitly stated as a method choice: > "we'll instead talk about the first principles of how, you know, we can design a system" - **Numbered/enumerated walkthroughs** in the demo section: "the first one being instrumentation," "The first layer is the segment," "we have three different scenarios." - **Concrete worked example** for abstract mechanism — the RAG chunk example: > "you have a rag retrieval tool, which is generating like 20 chunks every retrieval... the LLM is not even using the chunks that are after five" - **Scenario-based demo progression** (preview mode → governance on/kill → steer), each narrated as "So, this is the first scenario," "For the second scenario," building incrementally. - **Quantified results close**: benchmarks stated with specific numbers ("78%", "from 67% to roughly 96%") as the value payoff. - **Contrast framing** as a recurring persuasive device: token maxing vs. value maxing, halt vs. steer, request layer vs. run layer, existing tools vs. proposed platform. ## Vocabulary & Terminology - Dense domain jargon delivered without much pause: **attribution, ledger, control plane, boundary annotation, governor node, cohort, throttling, in-place compaction**. - **Custom product vocabulary introduced and defined inline**: "Token Ops," "cost guard," "preview mode," "wrap complete." - **Emotive/dramatic cost language**: "runaway loops," "unbounded consumption," "hundreds of millions of dollars." ## Audience Interaction - Minimal direct interaction; one audience-check moment: > "Everyone in the back can hear me? All right, perfect." - Closing invitation to follow up in person: > "Tisha and I are around, so if you guys have any questions... just let us know." ## Uncertainty Notes - Transcript names are inconsistent (title says "Susheem Koul," speaker intro says "Sushil," later "Sachin") — the second presenter's name cannot be confirmed from the transcript alone. - Audience reaction is only captured at the end ("[applause] Woo!"), so overall audience engagement during the talk is unobservable. - Filler-word frequency is directly observable but exact counts are not claimed here. # FinOps for AI Agents: Who Spent All the Tokens? — Position Record **Speakers:** Tisha Chawla (Microsoft) and "Sachin" (second presenter — identified by name in transcript; title lists Susheem Koul, but transcript attribution uses "Sachin"; note this discrepancy rather than resolving it). ## Framing Thesis (Tisha Chawla) - Core problem: organizations cannot trace AI bills back to their source in agentic workflows. - Industry currently rewards **"token maxing"** (spending maximum tokens for exploration); people are "proud to call themselves token millionaires," and she says that is acceptable *for now*. - The talk advocates a shift from **token maxing to value maxing**. ## Historical Pattern Argument (Tisha Chawla) | Era | Interface/Model | Control Surface | |---|---|---| | SaaS | UI | Usage caps, seat limits, tier policies | | Cloud | Pay-as-you-go | Auto-provisioning, auto-scaling | | Agentic | Model calls | **Missing / incomplete** | - Claim: in the agentic era, cost is created at the model-call boundary, but there is no proper control plane there. Existing model gateways offer hard caps or model-downgrade routing, but not control at the code-to-model boundary. ## Cited Evidence of Unbounded Consumption (Tisha Chawla) - Uber's AI budget reportedly exhausted within 4 months. - Companies reportedly hitting hundreds of millions of dollars in months or days. - Runaway loops causing massive cost increases without control mechanisms. - *(Note: cited as "news" without specific sources.)* ## First Principles Proposed (Tisha Chawla) 1. **Token as unit of cost** — value must also be measured in tokens. 2. **Cost originates at the model-call boundary** — that is what must be tracked. 3. **Attribution is essential** — without knowing which agent run made a call, control is impossible. 4. **In-place policies before halting** — steer loops, compact context, etc., with budget-cap halting only as a last resort. ## Critique of Existing Landscape (Tisha Chawla) - Existing tools (she names "light at a lamp port key, cloud flare" — likely Litellm/Portkey/Cloudflare, as transcribed) operate at the **request level**. - Claim: "you can't control the cost at the request layer / at the model layer" — run-level control is the missing piece. ## Token Ops Architecture (both speakers) **Tisha Chawla — design principles:** - Out-of-band plane; does not interfere with code. - Three modules: **instrumentation** (OpenTelemetry, cost metrics, enrichment, attribution), **accounting** (ledger of runs), **enforcement** (steering via policies; halt as final measure). **Second speaker — three layers:** 1. **Agent runtime** (user's own). 2. **Bridge**: attribution (runs mapped to user dimensions), **boundary annotation** (tracks method input/output to ledger; bidirectional channel for control-plane actions), **governor** (applies allowed actions non-destructively), wrap-complete (annotation for objects). 3. **Control plane**: segments (cohort-level budgets), ledger, budgets (static thresholds over time windows), actions (halt vs. steer), policies. - Privacy claim: control plane lives in the customer's own tenant — no data leaks. - Governor configs constrain which actions the control plane may apply. ## Demo Claims (second speaker) - **Scenario 1 (preview mode):** policies execute but enforcement is disabled — for safely testing guardrails in production. - **Scenario 2 (halt):** pre-call cost cap exceeded; agent killed immediately ("circuit breaker"). - **Scenario 3 (steer / cost guard):** considers budget consumed and token velocity; if it predicts budget exhaustion, injects instructions (e.g., "make outputs more succinct") to steer the run. ## Benchmark Results (second speaker) - Benchmarked on open-source repos **browser-use** and **meta GPT**, across multiple iterations and stress tests. - **~78% reduction** in average spend with full policy suite. - **Completion rate uplift from 67% to ~96%** vs. simple throttling (which he claims kills runs regardless). - Policy catalog covers spend management, context management (compaction, tool output reduction), loop detection, progress detection. ## Future Vision (second speaker) - A **self-learning module** in the control plane that reads the ledger, asks "what failure mode am I still not catching?", then either generates new policies on the fly or refines existing policy parameters. ## Unresolved Tensions / Qualifications - Speaker-name discrepancy: metadata lists Susheem Koul; transcript names the second presenter "Sachin" (also "Sushil" in the intro). Not resolved. - News citations (Uber, "hundreds of millions") are unsourced in the talk. - Benchmark figures are presenter-reported without methodology details shown in the transcript. # Speaker Attribution Record — "FinOps for AI Agents: Who Spent All the Tokens?" ## Speakers & Roles - **Tisha Chawla** — **Co-presenter / creator-side speaker** (Microsoft). Opens the talk, frames the problem ("token maxing to value maxing"), presents first principles and the proposed **Token Ops** platform architecture, then hands off to the second presenter. - **Susheem Koul** — **Co-presenter / creator-side speaker** (Microsoft). Note: identified in the title metadata as "Susheem Koul"; introduced in-transcript as "Sushil," and later referenced by Tisha as "Sachin" — name variants are unverified as to which is correct. Delivers the design walkthrough, code/demo portion, and benchmark results. - **No narrator, guests, or audience speakers** appear beyond applause and music markers. ## Quotations Anchoring Attribution - Tisha (self-identification): > "I'm Tisha and I have Sushil with me as my co-presenter." - Handoff: > "with this I would like to hand it over to Sachin for the demo." - Susheem (self-identification of demo role): > "Now let's shift gears, talk about the design part of it, and maybe get into the code and eventual demo." ## Referenced Sources & Third-Party Claims - **News/media claims** (no specific outlets cited): Uber's AI budget reportedly exhausted within 4 months; companies running into "hundreds of millions of dollars" within months or days; runaway agent loops causing cost spikes. - **Third-party tools/products named** (as competitive landscape, characterized as request-level rather than run-level): **LiteLLM, Portkey, Cloudflare** (transcript renders these as "light at a lamp port key, cloud flare"). - **Frameworks mentioned** (illustrative, not evaluated): LangChain, "LangSpout" (possible transcription of LangGraph/LangSmith — uncertain). - **Benchmarks claimed by presenters** (self-reported, not independently verified): tested against open-source repos **browser-use** and **MetaGPT**; claimed ~78% average spend reduction and completion uplift from 67% to ~96% vs. simple throttling. ## Promotional / Call-to-Action Segments - **Product promotion**: The talk proposes and demos **Token Ops** ("runaway token governance for AI agents"), presented by the speakers' team at Microsoft. - **QR code CTA** at the close, directing viewers to a **public wiki** described as regularly updated: > "you can scan this QR code, that's the public wiki." - Invitation for in-person follow-up discussion with both presenters. ## Availability & Uncertainty Notes - **No curated metadata** beyond title was supplied; roles are inferred from transcript self-identifications. - **Speaker name discrepancy**: metadata title says "Susheem Koul"; transcript uses "Sushil" (Tisha) and "Sachin" (Tisha, and possibly self-reference). Explicitly uncertain which name is authoritative. - **Comment availability**: not indicated in supplied material. - Figures (78% spend reduction, 67%→96% completion) are presenter-reported claims without cited methodology detail. \>\> Okay, um good morning everyone. So, I'm Tisha and I have Sushil with me as my co-presenter. All right, so we'll be talking about the most expensive question in AI today. I think a lot of you would have come across the scenario that um you know, when you opened an AI bill like through your agency workflows, um you couldn't actually trace back where that bill was actually coming from, right? And um and I don't think that's a problem right now because right now the industry is valuing, you know, uh token maxing, that is like uh spending the most amount of tokens for exploration, for all of those purposes. And um people are proud to call themselves token millionaires and I think that's all right. But this talk is, you know, the shift from token maxing to value maxing. You know, how do we get there? And we'll talk about it from this question, um who spent all the tokens? And um if anyone spent all the tokens, there has to be value associated with this, right? And that is uh the talk about. All right, now in order to minimize the gap, you know, from token maxing to value maxing, we'll kind of see we'll observe the patterns which the like the existing uh uh like the past software evolution eras had. Like for instance, when we talk about the SaaS era, the interface was UI and the control was in the form of usage caps, right? Like with the seat limits or tier-based policies. Now, when we moved on to the cloud era, the control surface again changed. The model became pay-as-you-go, and the control moved like in the form of auto-provisioning and, you know, auto-scaling policies. Now, we are in the agentic era, right? And um now, how the cost is calculated here is in the form of model calls, right? Like uh how uh like the code calls your model. But, what we've observed is that there isn't a proper control plane in place for that. Like, we do have control plane in place for your in place as model gateways where there um are hard caps or there is small routing to downgrade the model. But, the part like the where the code, you know, calls the model, that um is what we'll be talking about uh uh today. And um we also, you know, see um like in the last year, we've seen a lot of unbounded consumption happening. Like if you uh read the news, uh there was news about the like the uh AI budget for Uber getting exhausted within 4 months, and there were companies whose like who ran into you know, um like hundreds of millions of dollars within just months or days, and like there were a lot of um like other news in place as well where like these runaway loops um led to a very like massive increase in the cost. And there wasn't proper mechanisms to control it. Uh so, when we see all of this, the first thing that comes to our mind is is there a tool or is there a product to save us? But we'll instead talk about the first principles of how, you know, we can design a system which is actually true enough to solve the problem from the very root. So, for that let's like dive onto the principles. First of all, let's talk about token being the unit of cost. Right, we are charged in terms of token. So, the Now, we have to see value also in terms of token, right? Next We all know that cost is created at the LLM like the model call boundary. So, that is what we'll have to track. And if we don't have proper attribution, like if we don't know what agent one run made that particular call, we we can't, you know, control it, right? We We just know the like the broad picture of what went wrong, but we don't we can't, you know, trace it back or narrow it down. So, that is why attribution is a very important element to have. And like once you know which particular run or which particular agent is actually, you know, attributing to the cost, you should have proper policies in place to actually stop it. Like let's take example that if you have a, you know, a loop which is, you know, running very excessively and which is not required or, you know, if your context is growing very out of range, you should have in place policies which can like solve that particular thing there and there instead of halting that. And if And as the last resort only like a halting or a should happen from a budget cap. So, these are the first principles. Now, let's see how we can, you know, define an ideal platform on top of that from these principles which we talked about. All right. So, one thing which is very important and which matters here is that when we talk about like the existing frameworks for token ops or for token management, most of them are at the um like basically monitor the model request. They like they are like model gateways which will you know basically do like model routing or hard budget capping. But, what we need right now is something which you know um like monitors you at the run instead. Like if you see, we need something which can control the loop between like the agent call between the tool and the agent. Something, you know, which can um see or control the the spawning of multiple sub-agents happening from the one main agent or like something which can control the growing of context. So, like that is the need of the hour, right? And that is what we need. So, for all of this, we like kind of are proposing a platform which first of all has a cumulative budget across like the like the attribution runs which happened. And then where enforcement actually happens in call path rather than you know separate thing. Like for example, if something goes wrong, if your if your context is just growing heavily, then like in-place compaction should happen or like in place caching or something like that should happen. And after that if like after basically exhausting the list of all in place policies only like the budget gap hole should happen at the very last. Uh so that is something which we are proposing. But if we look at the landscape today, if you see the like the tools like this light at a lamp port key, cloud flare, all of those, they happen at again the request level, right? Like if you see like halting is there, routing is there for some of those, but all of this again is at a request and you can't control the cost at the request layer. Uh at the model layer, right? So this is the missing piece which is you know the uh basically navigating it at the you know the model the agent run layer. Uh so for that we have token ops which is uh you know a runaway token governance for AI agents. And uh this is the architecture for that. So first of all, one thing I would want to highlight is the like the intentional design decision we took here was an out of bound plane. So it doesn't interfere with your code at all. Um so if you see here that out of the band plane has three modules which I'll be talking about. The first one being instrumentation. It it is a common observability layer where you know you'll like have like the basic telemetry, the open telemetry, the cost in microns and like the like enrichment layer and basically the attribution like what caused that uh like particular run. And then there is obviously accounting where uh you will basically accumulate it in a kind of a ledger. Uh like the total runs which are happening. And finally we have this enforced layer which has uh two main purposes. One is steering it um through the policies which we've defined which I think Sachin will cover later. And then we have halt in place as the you know final uh like final thing if you know your budget is getting exhausted. Uh So yeah, that is there. Now when we again look at the landscape, this kind of will solve a lot of problems um which kind of happened when we like look at the previous tools and products there because uh it is at happening at run and it is you know helping you solve the problem from the very root by steering it in place. All right. So with this I would like to hand it over to Sachin for the demo. Yeah. \>\> Oh yeah. Now I think I should be able to Everyone in the back can hear me? All right, perfect. So yeah, we have established the principles behind Token Ops till now, right? Now let's shift gears, talk about the design part of it, and maybe get into the code and eventual demo, right? So what I have behind me on the screen is the like bird's-eye view of what Token Ops looks like today. It's It's three layers. We'll go left to right and top to bottom. So on the leftmost you have your own agent runtime which you're trying to instrument and kind of manage the cost for, right? The middle layer is what we're calling the bridge. That basically shuffles data between your agent and the control plane. And the control plane is where the mind of the system lies, right? So, let's talk about the bridge layer very briefly. Uh if we go from top to bottom, you have the attribution on top. So, what we're trying to do here is every agent run that you do, it's attributed to some user dimensions. So, the idea is everything that you do, every run of the agent is accounted to some usability or some usage. This comes in handy later, we'll talk about it. Uh the second part, which is the boundary annotation that you see, this is pretty much the heart and soul of this middle layer. So, the idea behind the boundary annotation is that you take any method, it doesn't matter what framework you're using, you might be using uh let's say LangChain, LangSpout, whatever. If you have a method, you can annotate it with boundary. What this annotation is going to do is it's going to do two things. First, it's going to track the input and the output, and it's going to flight that up to the control layer, and record it there as a ledger entry. Now, this will be annotated with the further agent run ID and the other attributes, and so on. The second thing the boundary annotation does is it acts as a channel through which the control plane can push actions down to the agent. This is where the entire intelligence lies. So, we do not have a single directional highway. We want the control plane to be able to tweak the behavior of the agent on the fly to ensure that we are able to squeeze in more runs inside our budget cap, right? Now, let's say the control plane pushes down an action. Let's take a small example. Let's say you have a rag retrieval tool, which is generating like 20 chunks every retrieval for every call, and that's eating up eating up your budget. And let's say the LLM is not even using the chunks that are after five because they're just not relevant, right? They're sorted by relevance. So, let's say the control plane observes this, and it wants to limit the output to this five chunks. So, it can push down an action, but that action has to be received by boundary, and then has to be executed by something. That is where the third node, the governor node, comes in. The governor knows what actions are allowed on your agent by you as a developer and it receives those actions from the control plane and knows how to apply it in a non-destructive way. So, that's the first three. The fourth one wrap uh the wrap complete is essentially just a helper method. So, as we know most of the agent providers or the model providers, they provide objects rather than methods for their LLMs, right? So, wrap complete is just another way of applying boundary on objects rather than methods. Let's shift right to the control plane. On the control plane, the first layer is the segment. Now, this is where the attribution that we talked about earlier comes into picture. So, any dimensions that you float from the attribution layer, let's say you have a preview agent that you share with everyone in this room and your agent is floating a dimension saying that cohort is AIA 2026, right? So, you can create a segment which is a cohort of users which is based on this tag like dimension being AIA 2026, right? And you can apply your budgets at this cohort level. So, you don't necessarily have to restrict everything at an agent level or a run level. You can do you can do roll-ups, you can do fine-grain or coarse-grain control, right? So, that's the segmentation part of it. Ledger, as I mentioned, is just one agent run all the traces in one place. Then, you have budgets. Budgets are basically just the static thresholds that work across a time window against a particular segment or an agent run. And then, you have actions. So, on the actions part, we have broadly two flavors. First is the halt type actions which basically just kill your agent if it exceeds a budget. The second part where we are adding value is the steer type actions. So, here we do not kill the agent, instead we try to steer the behavior of the agent or the components of the agent to try and fit that particular run within the allotted budget, right? And then, the policies layer is where it all comes together. You basically group the budgets, the actions, and then set your policies against certain segments or agent runs and that is where it executes, right? So, moving on, uh what changes in your code? That is the boundary annotation that we just talked about. As they have mentioned earlier, this is all out of band. So, you do not have to change your code. You just have to apply the annotation on the methods that you have. This boundary annotation will take care of floating all the information up to the control plane. And the control plane lies in your own tenant. So, you do not need to worry about any data leaks or anything. Then, if I talk about the governor, so for the governor, you just have to create an instance. You just have to pass it your own configs. These configs will basically declare what sort of actions are allowed for those agents, right? So, that your control plane cannot just willingly do any random things on your app on your agents. So, before we move on to the demo, I'll just briefly touch upon the test bench that we're going to use today. So, it's a simple two-agent workflow. We have a research agent which has access to a search tool. You give it a question. It's allowed to look up on the web as many times as it wants. And once it knows that it has all the data, it passes the findings on to the second agent which is a summarizer which creates creates a research report. Okay. So, with that out of the way, let's just quickly walk over to the demo. So, for the demo, we have three different scenarios that we're going to talk about. For the first one, we're going to run the token ops in what we call preview mode. So, in preview mode, what happens is that all the policies run as is, but the enforcement doesn't happen. So, if you see, we ran a particular run over here which completed, but we did not see any sort of failures there. The policies executed, but the actions that were associated with those policies were not allowed to be executed. So, we're just going to load the dashboard screen here. Yeah. So, this is the governance output. Governance is off. The run completed. But in the dashboard, you can see the policies have executed. So, you can see the cost budget, the cost guard, and so on, right? So, this was the first scenario. For the second scenario, what we're going to do is we're going to turn on the governance now. While that is happening, I just want to touch upon why this is important. So, if you want to like include this product into your production agents, you want to have a safe environment or a safe way to firstly put it in your production environment, test the guardrails, tweak the guardrails, see what the policies are doing, and then finalize the thresholds, right? So, this is the second one where we have now enforced the governance, and you can see in the dashboard that the pre-call cost cap has exceeded. So, you had a budget allotted for this run, but the agent exceeded the budget, and it was killed immediately. So, that's the simple circuit breaker breaker sort of a methodology. So, this is the halt behavior, and now let's see the steer behavior, which is the which is where we are trying to add value to this entire cost management scenario. So, this time we're going to run the third the second prompt. The budget allotted for this one is slightly higher, but it's still not high enough for the agent to complete in time. So, what instead happens is there is something called cost guard which kicks in. This cost guard, it takes into account two things. First, how much of your allotted budget have you consumed? Second, what is the velocity at which you are consuming tokens? \>\> \[music\] \>\> Now, based on these two things, if it predicts that you're going to run out of your tokens or your allotted budget by the end of the run, it's going to inject something into your system instructions. That something could be as simple as, "Hey, you're running running out of budget, so make sure that the LLM outputs are more succinct or more summarized, right?" So, that is the way we are doing the steering. Now, this was a very simple test bench to show you like how this works on a like working code. We have also benchmarked it on a couple of open-source repos, so we have benchmarked it on browser use as well as meta GPT. We ran it across multiple iterations, across stress tests, across simple scenarios, hard scenarios, and everything, and the results we see are the average spend goes down by almost 78% with Token Ops enabled with the full policy suite that we have today. On the completion part, when we compare it with throttling, just simple throttling, your simple throttling is going to kill your agent runs, no matter what, right? So, with the reduced average spend, what you get is you get an uplift in that completion percentage from 67% to roughly 96%. So, that is the value add that Token Ops is doing here. Now, this is the policy catalog that we run this benchmark against. This is what we support today. We kind of research what are the different failure modes that are there today out in the wild and try to cover most of them here. So, you have things across spend management, you have things across context management like context compaction, tool output reduction. You have things across loop detection and progress detection and stuff like that. So, this is the entire set of policies that we support. And at the bottom you can see the actions. So, as I mentioned earlier, we have two flavors. You have the uh the halt type actions and then the steer type actions. So, for the steer we can do allow, mutate, inject, and so on. And for the halt it can be a simple kill. But, this is not the end state that we envision for this. Uh the end state is we have a lot of data, right? We have a ledger that is continuously being updated. So, what we want to try is we want to try a self-learning module within the Token Ops plane, within the control plane, which can look at this ledger and ask this question, "Hey, why or what is the failure mode that I'm still not able to catch?" And then based on that, it can do two things. One is it can enhance, it can generate new policies on the fly based on the missing or the still uh runaway costs, or it can refine the existing parameters for the existing policies that are there so that the runaway costs are managed more effectively in the future. So, with that, I think that is all we have for you guys today. Thank you so much for your time. And uh you can scan this QR code, that's the public wiki. We are updating it almost regularly. So, you can scan this and stay up-to-date. And Tisha and I are around, so if you guys have any questions or if you want to discuss more about it, just let us know. That's it. Thank you. \>\> \[applause\] \>\> Woo!