Conference talk by Erik Hanchett (AWS) on reducing AI agent costs without sacrificing quality. Drawing on his experience deploying an agent to production that worked correctly but cost too much, he shares three small code-level changes that lowered costs without changing prompts or switching models, including techniques such as prompt caching and model routing. Code examples are shown. # Channel Presentation Observations — "Your Agent Is Wasting Tokens and You Don't Know It" ## Explicit Self-Description (Creator Claims) - **Role/affiliation**: The speaker states, > "my name is Eric Hanchett. I am a senior developer advocate at AWS" — an explicit claim of professional identity and employer affiliation. - **Format claim**: He frames the video as a brief talk: > "my very quick, almost lightning talk today." - **External presences**: The speaker invites follow-up via named channels: > "find me on LinkedIn at Eric Hanchett. I also blog at programwitheric.com. And then you can find me on social media at EricCH." ## Intended Audience - **AI/LLM application developers**, specifically those **building or operating AI agents**: > "how you can save on token costs... while using and creating agents." - Audience is assumed to have coding familiarity — the talk is structured around **live code walkthroughs** ("Let me show you some code", "Let me show you some code on here") without explaining programming basics. - Audience is assumed to understand LLM concepts already (system prompts, context windows, tool calls, model tiers like "Claude Haiku" and "Claude Sonnet") — no definitions are provided. ## Claimed Scope of Content - A **focused, enumerated technical topic**: "five ways that you can reduce your token costs" for agents, explicitly enumerated in the conclusion (cache system prompt, route by difficulty, offload tool results, cap tool loops, trim history). - **Ecosystem focus**: Demonstrations use **AWS Strands Agents**: > "I'm using AWS's Strands agents. This works with all different different providers." He notes it is "a little bit of pseudo code," hedging that examples are illustrative rather than production-ready. ## Observable Presentation Patterns (Demonstrated, Not Claimed) - **Code-centric teaching style**: At least three points are introduced with "Let me show you some code," indicating screen-shared code demonstration is the channel's primary instructional mode. - **Opinionated practical guidance**: Direct recommendations with rationale, e.g., > "I highly recommend don't use the most expensive model for everything you're doing" and > "always set a max iterations of how many times it will loop." - **Trade-off transparency**: Presents downsides of techniques, e.g., the sliding-window history trimming's "downfall... is that you will lose the message history from the beginning." - **Cost/efficiency framing**: The entire talk is organized around a concrete operational concern (token spend), including pre-deployment advice ("run some observability tools... before you deploy your agent"). - **Personal experience cited as evidence**: > "I've had this happen often where it calls the tool over and over and over again" — drawing on the speaker's own development experience. - **Audience-engagement posture**: Closes with an open invitation to continue conversation across LinkedIn, blog, and social media ("Love to talk to you guys more"), suggesting an accessible, community-oriented presenter style. ## Notes for Synthesis - All identity claims above come directly from the transcript; **no curated metadata was supplied** for this video. - No claims are made about the channel's broader upload schedule, subscriber base, or series structure — nothing in this material supports channel-wide conclusions beyond this single presentation. # Delivery Observations: Erik Hanchett — "Your Agent Is Wasting Tokens and You Don't Know It" ## Overall Format - **List-structured talk**: five enumerated tips delivered in sequence, with an explicit recap at the end: *"So, in conclusion, we have five things. Cache the system prompt... Route by difficulty..."* - **Short "lightning talk"** framing, self-described: *"Thank you for listening to my very quick, almost lightning talk today."* ## Tone and Register - **Conversational, low-formality developer-to-developer tone** throughout: *"Hey everyone,"* and *"You can play around with this."* - **Direct address to the viewer** with second-person imperatives: *"you want to look into routing,"* *"always set a max iterations,"* *"Make sure you use observability tools."* ## Vocabulary and Content Habits - **Code demonstration as a teaching device**, signposted with a recurring phrase: - *"Let me show you some code."* - *"Let me show you some code on here."* - **Reference to concrete tools/models by name** as evidence: *"AWS's Strands agents," "Claude Haiku," "Claude Sonnet," "sliding window conversation manager."* He also flags pseudo-code explicitly: *"This is a little bit of pseudo code."* - **Cost/token framing dominates the vocabulary**: "token costs," "cheap model," "save on tokens," "eat through hundreds, if not thousands, of tokens." ## Recurring Phrases and Patterns - **"Every single"** as an intensifier, repeated multiple times: - *"every single call"* - *"every single time"* - *"every single tool"* - *"for every single task"* - **"So," as a sentence starter**, very frequent: *"So, the first way," "So here's a code example," "So, you can play around with this," "So that way," "So always set," "So rather than sending all of it."* - **Pattern of tip → explanation → trade-off/downfall**, e.g. after the sliding-window tip: *"The downfall of this, or the trade-off of this, I should say, is that you will lose the message history from the beginning."* ## Hedging and Self-Correction - **Occasional mid-sentence self-repair**, e.g. *"this a a manual way,"* *"which which this does,"* and the on-the-fly rewording above (*"the downfall... or the trade-off... I should say"*). - **Mitigated recommendations** alongside firm ones: *"you may want to use one of the newer frontier models"* vs. the stronger *"I highly recommend don't use the most expensive model for everything."* - **Small oral disfluencies**: *"uh this will save you a lot of tokens."* ## Teaching Habits - **Pre-empting the obvious**: *"This may sound obvious, but you want to look into routing..."* - **Anecdotal evidence from experience**: *"I've had this happen often where it calls the tool over and over and over again."* - **Concrete numbers to illustrate risk**: *"it might run 10, 20 times. It might get into an infinite loop."* - **Closing CTA with multiple channels**: LinkedIn, *"programwitheric.com,"* and social media at *"EricCH."* ## Uncertainty Notes - The transcript lacks audible audience response, so **pacing, pauses, and audience interaction cannot be assessed**. - No visual detail is available; claims about "showing code" rely on the speaker's verbal signposts, not observable screen content. # Position Record: "Your Agent Is Wasting Tokens and You Don't Know It" — Erik Hanchett (AWS) ## Speaker - **Erik Hanchett**, self-identified **Senior Developer Advocate at AWS**; blogs at programwitheric.com, social handle EricCH. ## Overall Claim Agents often waste tokens, and developers can reduce token costs through **five techniques** when building/running agents. ## The Five Positions ### 1. Cache the system prompt - Demonstrated with **AWS Strands Agents** (pseudo-code); noted it works with "all different providers." - Setting `cache prompt equals default` sends the **full system prompt on the first call**, then a much reduced cached version on subsequent calls. - Tool prompts and messages can also be cached. ### 2. Route requests by difficulty - **Claim:** Don't use the most expensive model for everything; route based on task difficulty. - Example: use **Claude Haiku** for simple tasks, **Claude Sonnet** for harder ones, via an if statement. - Suggested a very cheap model could **decide which model to use** (model-as-router). - Framed as strong recommendation: "I highly recommend." ### 3. Offload tool results - **Claim:** Large tool results should be stored locally or in the cloud and **summarized**, so the full result isn't re-added to context on every agent/tool loop iteration. - Noted Strands offers additional APIs for this; manual approach shown in code. ### 4. Cap tool loops - **Claim:** Agents sometimes call a tool repeatedly — potentially 10–20 times or into an **infinite loop** — which is "very bad for your token usage." - **Recommendation:** Always set a **max iterations** limit. - **Pre-deployment practice:** run observability tools to inspect per-tool call counts, loop counts, and run times to gauge efficiency. ### 5. Trim conversation history - **Claim:** In multi-turn agents, the whole history is resent each call and "can eat through hundreds, if not thousands, of tokens." - Strands' **sliding window conversation manager** sends only the last 10 messages (configurable). - **Stated trade-off (preserved):** you lose early message history; mitigation is to **summarize history** and insert the summary into the context window. ## Reasoning Pattern Consistent cost-optimization logic: identify where tokens are repeatedly re-sent (system prompt, tool results, conversation history) or over-consumed (loops, overpowered models), then cache, summarize, cap, or route accordingly. ## Qualifications & Caveats - Code examples are "a little bit of pseudo code." - Sliding-window trimming has an explicit **downfall/trade-off** (lost early history). - Offloading tool results is described as having both manual and API-based approaches. # Speaker Attribution & Provenance Record **Video:** Your Agent Is Wasting Tokens and You Don't Know It - Erik Hanchett, AWS **Video ID:** `01a15ca8-ddce-726a-965f-6331e749b0a5` ## Speakers | Speaker | Role | Evidence | |---|---|---| | Erik (Eric) Hanchett | **Creator / presenter** | Self-identified in opening: *"my name is Eric Hanchett. I am a senior developer advocate at AWS"* | - Single-speaker presentation; **no guests or additional narrators** appear. - Speaker describes it as a *"very quick, almost lightning talk."* ## Referenced Sources & Third-Party Claims - **AWS Strands Agents SDK** — used in all code examples ("This is a little bit of pseudo code"); claimed to work with multiple providers. - **Amazon Bedrock models** (third-party AI models): - **Claude Haiku** — cited as a cheaper model option. - **Claude Sonnet** — cited for more difficult tasks. - Mentioned but **not named**: observability tools for inspecting tool-call usage. ## Promotional Segments - **Closing promotion** — speaker directs viewers to his own channels: - LinkedIn: "Eric Hanchett" - Blog: programwitheric.com - Social media: "EricCH" - Code examples reference **AWS Strands Agents** throughout, consistent with the speaker's AWS employer affiliation. ## Unavailable / Uncertain - **Comment availability:** not stated in transcript or metadata. - **Curated metadata:** none supplied beyond title and transcript. - Specific timestamps: not present in transcript. ## Notes - All five tips (prompt caching, routing by difficulty, offloading tool results, capping tool loops, trimming history) are **first-party claims by the presenter**, not attributed to external publications. Hey everyone, my name is Eric Hanchett. I am a senior developer advocate at AWS and I'm going to talk to you about how you can save on token costs. Now, I'm going to show you five ways that you can reduce your token costs while using and creating agents. So, the first way you can do that is to cache your system prompt. Let me show you some code. Now, I'm using AWS's Strands agents. This works with all different providers. This is a little bit of pseudo code, but the idea is that you can add cache prompt equals default. And what that'll do is on the first call of your agent, it will send the full system prompt over and then on every subsequent call, it will have a much reduced system prompt being sent over. So, it'll be cached. You can also cache the tool prompts and messages as well. This may sound obvious, but you want to look into routing your different messages based on the difficulty. So, here's a code example. Let's imagine that you have a task that's very difficult. You may want to use one of the newer frontier models. However, if it's something simpler, you want to use a cheaper model. In this case, maybe we use Claude Haiku for a cheap something cheap and then use Claude Sonnet for something a little bit more difficult. And then you can use an if statement. You can even have another model that's very cheap decide which model to use. So, you can play around with this, but I highly recommend don't use the most expensive model for everything you're doing. You want to use multiple different models based on the use case. And then try to route to it inside your agent. Another good tip is to offload the tool result. Let me show you some code on here. Once again, I'm using strands agents. This is a a manual way to do it. There is some additional APIs that the strands agents offers to do this. If you have a large tool result that's coming back, you can store it locally or in the cloud and then use some kind of summarization that saves on tokens. So that way when it's being called over and over again, the tool result isn't added into the context every time every time the tool loops or every time the agent loops. So if you can find any way that where you have this tool result that you don't necessarily send it on every single call back to the large language model, that will save a lot of tokens for you. And like I said, there's a few APIs to do this, but it essentially you can do the summarization technique. You can also cap your tool loops. So when you're dealing with the agent loop and it decides to do a tool call, I've had this happen often where it calls the tool over and over and over again. And if you don't cap that tool call, then it might run 10, 20 times. It might get into an infinite loop, which would be very bad for your token usage. So always set a max iterations of how many times it will loop. A good thing you can do before you deploy your agent is to run some observability tools and take a look at the tool call use for every single tool and then see how long each one of them is running and how many times they're looping. And that way you can get an idea of how efficient the tool call is. Last but not least, we can trim the history. So if we're using a multi-turn agent and we are talking back and forth, you will find at times that the conversation history will get very large on every single call, that whole conversation history will be sent back to the large language model. And this can eat through hundreds, if not thousands, of tokens. In Strand's agents, we have something called sliding window conversation manager, which which this does is it looks back at the last 10 messages and only sends those back. And you can set this to whatever you want. And that way you're not sending these huge message histories back to the agent every single time a new message comes in. The downfall of this, or the trade-off of this, I should say, is that you will lose the message history from the beginning. The way you want to deal with that is you can use uh some sort of summarization of the history and then put that into the context window. So, rather than sending all of it, you may send a small amount once you hit this sliding window. Uh this will save you a lot of tokens. So, in conclusion, we have five things. Cache the system prompt. And if you can, maybe the tool prompt and messages. Route by difficulty. Don't use the same expensive model for everything you're doing, for every single task. Offload these big tool results. So, if you have a large tool result, you can summarize it and not have it being sent in the agent loop and overloading your context. You can cap those tool loops. Make sure you use observability tools to see how long tool calls are taking and how many loops they're they're doing and and then try to iterate over that. And then, of course, trim the history if you're using a multi-turn conversation with your agent, so that way the whole conversation isn't being sent over and over again for very long conversations. Thank you for listening to my very quick, almost lightning talk today. If you like to go deeper, find me on LinkedIn at Eric Hanchett. I also blog at programwitheric.com. And then you can find me on social media at EricCH. Love to talk to you guys more. Thanks.