Talk by Jake Broekhuizen of LangChain Labs on giving AI agents memory that changes their behavior. Using a financial services agent whose tone drifted from advisory to pushy, it argues that traces and logs become memory only when a lesson is written into durable context the agent reads on its next run. Topics include semantic, episodic, and procedural memory; working versus long-term memory; the read-write cycle (read context, run, filter evidence for signal, write back); how LangSmith and Context Hub capture, analyze, and update context; and three lessons on what to keep, caching on the hot path, and human review of key rules. Recorded at the AI Engineer World's Fair 2026 in San Francisco. # Video Identity Observations ## Source Overview - **Video title:** *"Giving AI Agents Memory That Learns — Jake Broekhuizen, LangChain"* - **Format:** A conference talk (~15–18 minutes, per the speaker: *"hopefully I can keep you engaged for the next uh 15 to 18 minutes"*) ## Speaker Self-Description (Explicit Claims) - **Role at LangChain:** >"My name's Jake and I lead our labs team here at LangChain." - **Organizational affiliation:** The talk presents LangChain product territory directly — observability and a new feature attributed to **LangSmith** and **"LangSmith's context hub"**: > "The analyze step in the middle is something new that I'm really excited about... promoting that signal to the memory store... that is LangSmith's context hub." - **Practitioner credibility claims:** Describes hands-on client work: > "the labs team at LangChain have been working with a company in the financial services sector who's building a... budgeting and non-investment advice agent." - **Personal debugging experience shared:** >"I was banging my head on a wall for a while while I was trying to sort of understand why my behavior wasn't changing." ## Intended Audience (Observable Presentation) - **Agent builders / practitioners**, not general consumers: - >"a lot of teams that I speak with who are building agents" - References assumed familiarity with industry vocabulary: *traces, working memory, progressive disclosure, sleep-time compute, sub-agents, harnesses, SDKs*. - >"You've probably seen progressive disclosure with agents" — implies a technically fluent audience. - **Conference attendees:** Opens with >"Good morning everybody. I know it's the last day of the conference" and closes with an in-person invitation: > "please come and see me. We're at booth Eugene 19." ## Claimed Scope of the Talk - Framed as a **framework/mental-model talk**, not just a product pitch — though the product is embedded: - >"One implementation of this, and I won't claim it's the only implementation... is the way that we do this with LangSmith." - Core conceptual claims: - Memory defined as **durable context**: >"A trace... only becomes memory when that lesson is converted into durable context." - A **read-write cycle** borrowed *"from cognitive science"* (semantic / episodic / procedural memory taxonomy). - Design guidance offered as team-drafted principles: memory filtering, hot-path caching concerns, and human review for procedural memory (>75% of agent behavior, per his estimate). ## Creator Claims vs. Demonstrated Patterns - **Claim:** Financial-services client case with tone-slipping agent (creator-provided anecdote, not verifiable from the material). - **Demonstrated:** The speaker consistently positions memory as turning *"a trace... into a source of signal"* and the refrain >"The next run should be better." - **Promotional signals (demonstrated):** QR code to a build walkthrough, booth invitation, and introduction of a new LangSmith feature — indicating the talk doubles as **product evangelism** for LangChain's tooling. ## No Evidence Found For - Channel upload cadence, subscriber audience, or broader channel identity — this material is a single conference talk transcript and supports no channel-wide conclusions. # Delivery Observation — "Giving AI Agents Memory That Learns" (Jake Broekhuizen, LangChain) ## Overall Register Conversational, fast-paced conference talk with a startup-style casualness. The speaker frequently restarts sentences mid-flow and uses hedge/filler phrases, but compensates with strong structural signposting and consistent callbacks to a single running example. --- ## Recurring Verbal Patterns ### "I think" as a hedging scaffold One of the most frequent openers and mid-sentence hedges: > "I think a lot of teams have run into" > "I think that it's helpful to group them into three distinct buckets" > "I think that distinction is really important" Notable: he hedges even claims he then asserts strongly, e.g., *"Procedural memory is where actually most of the visible gains come from"* — a confident claim delivered right after heavy hedging. ### "And so / So" as a connective reflex Used to launch nearly every explanatory move, sometimes twice in a sentence: > "And so, that's where memory And so, that's where memory comes into play." > "And so, that's what gives Oh, that's what lends itself to this really helpful mental model" ### In-sentence restarts and self-corrections A distinctive delivery tic — abandoning a sentence halfway to rebuild it: > "It might be skills, it might be instructions. You've probably seen progressive disclosure... That's that phase moving from long-term memory into short-term memory." > "Obviously, that that doesn't scale, or rather it's challenging to scale." > "I won't I won't claim it's the only implementation, but just as just a kind of like put some concrete structure" The restarts sometimes soften absolute claims ("doesn't scale" → "challenging to scale"), functioning as live calibration. ### Immediate self-repetition for emphasis Key ideas are stated, then repeated nearly verbatim within the same breath: > "very reliable very reliable agents" > "able to change are able to change and guide themselves" > "some of the memory or some of the context that an agent references is temporary" --- ## Teaching Habits ### Single running example with explicit callbacks The financial-services tone agent is introduced early and deliberately anchored: > "let me give you an example to hold onto for the rest of this talk today" He returns to it at least three times later — for procedural memory (*"in that financial assistance that I the example that I gave at the beginning"*), for the write-back step (*"that was be measured and not pointed or not directive"*), and when framing LangSmith's loop. ### Naming and numbering abstractions Heavy use of taxonomies and named cycles to structure the material: - Three-bucket taxonomy from **cognitive science**: *"what the agent knows"* (semantic), *"what the agent has experienced"* (episodic), *"how the agent should behave"* (procedural). - A **read-write cycle** mental model: *"Memory informs the run. The run makes the evidence. We filter that evidence to make signal, and then we use that signal to update memory."* - A simplifying **three-part shape**: *"capturing what happened, analyzing what might matter, and then updating the context."* ### Framing as tension → resolution He sets up problems before solutions: > "observing them and then turning those observations... into lessons for future runs are obviously very different things" And reframes logs vs. memory aphoristically: *"Agents that are observable tell you what happened. Agents that are adaptive are able to change."* ### Experiential anecdote as authority Personal struggle used to make a lesson memorable: > "I was I was banging my head on a wall for a while while while I was trying to sort of understand why my behavior wasn't changing" ### Practical closing mechanics QR code walkthrough and booth invitation (*"booth Eugene 19"*), plus direct audience acknowledgment: *"I know it's the last day of the conference."* --- ## Vocabulary Notes - Favored intensifiers/softeners: **"obviously," "really important," "sort of," "kind of like,"** and **"durable context"** as a technical term of art. - Concrete quoted agent output used to make tone-slip tangible: *"users in your similar situation would generally set a cap"* and *"it looks like you need to cancel some subscriptions."* - Quantification used sparingly but memorably: procedural memory *"probably going to drive about 75% of the behavior."* --- ## Confidence / Uncertainty - The "I think," restart, and repetition patterns are high-confidence — each appears many times across a single ~15-minute talk. - Whether these are stable long-term habits or artifacts of this specific delivery (last-day-of-conference, live, unrehearsed) **cannot be determined from one transcript**. - The single-example technique is clearly deliberate here (*"to hold onto for the rest of this talk"*), but a single talk is thin evidence for it being a default teaching style. # Observation Record: "Giving AI Agents Memory That Learns" **Speaker:** Jake Broekhuizen, LangChain (identified by the title and the self-introduction in the transcript: "My name's Jake and I lead our labs team here at LangChain") **Format:** Conference talk, approximately 15–18 minutes, single speaker **Evidence basis:** Transcript only. No slides, data tables, or external sources were provided. Where the speaker refers to slides, the record reflects only what the transcript states. --- ## 1. Core Thesis > "The next run should be better. Memory allows us to do this." - **Central claim:** Agents should improve from one run to the next, and memory is the mechanism for this. - **Key distinction the speaker draws:** Observability tells you *what happened*. Adaptation requires converting observations into *durable context* that changes future runs. - **Stated framing:** A trace "only becomes memory when that lesson is converted into durable context the agent can reference on future runs." --- ## 2. Motivating Example: Financial Services Tone Drift The speaker uses a case involving a company in financial services building a spending, budgeting, and non-investment advice agent. ### Reported Problem - The agent's tone sometimes shifted from *advisory and helpful* to *pointed and directive*. - Examples given: "users in your similar situation would generally set a cap," and advice to cancel subscriptions and move money into savings. - The company considered this a **tone violation**, and the speaker reports there were "similar instances." ### Reported Current Process - Review is **manual**: someone must read traces, find where tone slipped, edit the context the agent references, and check for regressions. - The speaker states this "doesn't scale, or rather it's challenging to scale." ### Proposed Direction - The surrounding system should be "a place where it can continuously learn." > **Note:** The speaker describes the company's view and the problem in general terms. No quantitative results from this deployment are presented. The claim that tone changes can now be observed is stated as the intended outcome of the loop, not as a demonstrated result. --- ## 3. Taxonomy of Memory The speaker proposes grouping memory into three types, "borrowed from cognitive science": | Type | Speaker's Description | Examples Given | |---|---|---| | **Semantic** | What the agent *knows* | Facts, preferences | | **Episodic** | What the agent has *experienced* | Learned patterns, past interactions, examples | | **Procedural** | How the agent *should behave* | Instructions, skills, rules | Six items were listed on a slide (not visible in transcript): facts, preferences, patterns, previous examples, skills, and instructions. ### Claimed Relative Importance - **"Procedural memory is where actually most of the visible gains come from."** - Later, the speaker estimates procedural memory will "probably... drive about 75% of the behavior of my agent." - *Qualification:* This is an approximate, personal estimate ("probably," "about"). No method for the estimate is given. --- ## 4. Working Memory vs. Long-Term Memory - **Working memory:** Context used in the current run, such as scratch pads, tool results, and retrieved files. - **Long-term memory:** Durable stores of instructions, skills, and reference material used across future runs. ### Speaker's Note on Implementation - Harnesses differ in how they inject long-term memory: prompt injection, tool retrieval, direct store access, files, or runtime state. - The speaker says "the implementation can vary, but the distinction is really important." --- ## 5. The Read–Write Cycle The speaker presents memory as a **continuous loop**: 1. **Read:** At run start, relevant context moves from long-term into working memory (e.g., progressive disclosure of skills). 2. **Run:** The agent produces a trace, which is evidence of tool calls, decisions, and sub-agent activity. 3. **Filter:** Decide what is signal and what remains history. 4. **Write:** Signal is written back into durable long-term context. > "Memory informs the run. The run makes the evidence. We filter that evidence to make signal, and then we use that signal to update memory." ### Simplified Form - **Capture** → **Analyze** → **Update context** The speaker calls the "filter" step "really interesting" and presents it as the key design question. He also notes that sleep-time compute and "dreaming" refer to memory updates that occur during or outside active work. --- ## 6. LangSmith and Context Hub (Speaker's Implementation) The speaker offers one implementation while explicitly disclaiming exclusivity: "I won't claim it's the only implementation." - **Capture:** LangSmith observability (tool calls, sub-agent calls, retrieved context). - **Analyze:** A background process that extracts signal from traces based on user-defined criteria. The speaker describes this as "something new that I'm really excited about." - **Store:** LangChain's Context Hub, a remote store the agent pulls from on its next run. ### Financial Assistant Loop (as described) 1. Agent context is loaded from Context Hub (instructions, skills, markdown files, policies, rules). 2. A trace records failed tool calls, user corrections, or tone deviations. 3. The analysis step identifies patterns and proposes changes to static skill markdown files. 4. Changes are applied, and later runs are expected to reflect them. > **Note:** The transcript says the analysis step "looks for patterns" and "will make that change" without describing whether changes are automatic, reviewed, or how they are validated at this point. Later the speaker argues for human review (see Section 8). --- ## 7. Design Principles Stated by the Speaker ### Principle 1: Not Everything Should Become a Memory Update - Agents produce "exhaust" or "feedback." - **Most trace data** should remain as referenceable **history**. - **Some data** may become **datasets for offline evaluation**. - **A small subset** should change the agent's memory. - The speaker says "the devil is in the details" of this decision, without giving a specific rule. ### Principle 2: Memory Updates Must Actually Be Accessible to Future Runs - The speaker describes a "gotcha": updates made early in a long-running background agent may not be reflected in later behavior because of runtime state or caching. - He reports spending time "banging my head on a wall" before recognizing this issue. - Tension acknowledged: optimize (e.g., caching) versus keeping memory updates available "in the hot path." ### Principle 3: Decide What Self-Updates vs. What Requires Human Review - He references open-source agents with self-update processes. The transcript reads "open core and... Hermes agent." The transcript wording may be imperfect; the record reflects it as written. - **Position:** Procedural memory (instructions, policies, tone guidelines) should have human-in-the-loop review before committing to the agent's "hot reload path." - **Rationale given:** Procedural changes drive most of the agent's behavior, so "protecting important behavior" warrants review. --- ## 8. Stated Tensions and Qualifications | Tension | How the Speaker Handles It | |---|---| | Self-improvement vs. human oversight | Favors human review for procedural memory; self-updates appropriate for other categories (implied, not explicitly enumerated) | | Optimization vs. memory freshness | Acknowledges caching can prevent updates from reaching future runs; advises understanding what can and cannot be cached | | Observability vs. adaptation | Observability is "a critical foundation," but insufficient alone without the update path | | Implementation maturity | "The implementations... are still evolving"; "this three-part shape is a really useful starting point" | | Claimed gains | Procedural memory described as the main source of visible gains; no empirical data presented | --- ## 9. Claims Without Presented Evidence The following are stated as positions or observations but are not supported by data in the transcript: - Procedural memory accounts for most visible gains. - Procedural memory "probably" drives about 75% of agent behavior. - The company's tone problem is resolved or improving under the described loop. - The three-part loop "maps cleanly" to the financial assistant case. --- ## 10. Closing Position - Memory is defined as **durable context that guides future runs**. - The goal is a **flywheel** in which traces become signal and signal improves memory, producing agents that "feel like they get better with time." - The speaker points to a QR-linked video demonstration (not available in this transcript) and invites follow-up discussion at a booth. --- ## Source Notes - Speaker identity is taken from the video title and the transcript's self-introduction. No additional curated metadata was supplied. - The transcript appears to contain transcription artifacts (repeated phrases, such as "And so, that's where memory comes into play" and "that's where memory And so, that's where memory comes into play," and possible misrecognitions such as "open core"). These are preserved as written and not corrected. - Booth location and event logistics are omitted as non-substantive. # Speaker Attribution and Provenance Record **Video ID:** `01a15ca6-e315-78f0-ac15-a0d7f959997a` **Title:** *Giving AI Agents Memory That Learns — Jake Broekhuizen, LangChain* --- ## 1. Speakers and Roles ### 1.1 Primary Speaker - **Jake Broekhuizen** - **Role (self-identified):** Leads the labs team at LangChain. - **Function in video:** Sole speaker. Presents a conference talk on agent memory. - **Attribution basis:** Title metadata and in-transcript self-introduction ("My name's Jake and I lead our labs team here at LangChain"). ### 1.2 Other Persons and Entities Referenced (Not Speaking) - **Unnamed financial services company** - Described as a client of the LangChain labs team that is building a spending, budgeting, and non-investment advice agent. - Referenced as the running example. The speaker reports its tone-compliance issues and the process used to correct them. - **LangChain labs team** (the speaker's team): referenced as having worked with the financial services company. - **Unnamed teams and "people building different frameworks":** referenced generally, with no specifics. ### 1.3 Audience - Conference attendees, addressed as "everybody" on the final day of the conference. No audience speech is captured. --- ## 2. Referenced Sources and Frameworks | Source / Concept | How Referenced | Attribution Note | |---|---|---| | Cognitive science memory taxonomy | Speaker says the three-bucket taxonomy is "borrowed" from cognitive science. | Speaker-attributed. No citation given. | | Semantic, episodic, procedural memory | Named as the three buckets. | Terms attributed by speaker to cognitive science and to "often referred to as" usage. | | Working vs. long-term memory | Presented as a distinction in agent design. | Speaker-presented framing. No source named. | | "Sleep-time compute" and "dreaming" | Referred to as existing terms for memory updating. | Speaker says "you might have heard of" these. No source named. | | Progressive disclosure | Mentioned as a pattern where agents pull in relevant skills. | Speaker says "you've probably seen" this. No source named. | | **LangSmith** | Described as the speaker's team's implementation for observability and analysis. | Speaker-attributed; see Section 3. | | **LangSmith context hub** | Described as the remote store where extracted signal is promoted. | Speaker-attributed; see Section 3. | | Open-source / popular agent frameworks | Speaker references "open core" (as transcribed), "Hermes agent," and "a lot of agents" that self-update. | Named in transcript as spoken. Spelling "open core" is as transcribed and unverified. | | Speaker's own QR-code video | Speaker says a QR code links to a video walkthrough of building such an agent. | Unavailable in transcript; the QR target is not captured. | --- ## 3. Promotional Segments > **Flagged as promotional.** The following segments promote LangChain products or the speaker's presence. - **LangSmith (promotion):** Presented as "one implementation" of the loop, including observability for the capture step and "LangSmith's context hub" for storage. The speaker states they "won't claim it's the only implementation." - **LangChain context hub (promotion):** Described as "what LangChain's context hub does." - **QR code call-to-action:** Directs viewers to a video of the speaker building the described agent. - **Booth call-to-action:** "We're at booth Eugene 19" (transcribed as spoken; the identifier may be a transcription artifact and is unverified). Invites follow-up discussion. **Non-promotional but company-linked:** The financial services example is described from the speaker's own team's engagement. The client's identity is not disclosed. --- ## 4. Provenance Facts ### 4.1 Claims Attributed to the Speaker's Experience - The financial services example and its tone-slip problem are reported as the speaker's team's observed work. - The claim that manual review "doesn't scale" is the speaker's characterization. - The speaker reports personal debugging experience: "I was banging my head on a wall for a while" about memory updates not persisting in long-running agents. ### 4.2 Unsourced Quantitative or Generalized Claims - **"Procedural memory... probably drive[s] about 75% of the behavior of my agent"**: an approximate, speaker-stated estimate. No data source given. - **"Most teams arrive at the general same instinct"**: generalization about industry practice. No source given. - **"Agents are producing more signal and more traces than ever before"**: stated as fact. No source given. - **"Most evidence should stay as history"** and **"a small segment... should end up becoming what changes your agent's memory"**: stated as design guidance by the speaker. No source given. ### 4.3 Third-Party Entities Named as Context - Financial services company (unnamed). - Open-source or popular agent projects named in passing (as transcribed: "open core," "Hermes agent"). --- ## 5. Comment Availability and Evidence Limits - **Comments:** No comment data was supplied in the record. Comment availability is **unknown**. - **Curated metadata:** Only the title and speaker name were supplied. No description, cited-source list, chapter markers, or timestamps were provided. - **Timestamps:** None present. Segment locations cannot be cited by time. - **Linked materials:** The QR-code video, the LangSmith and context hub pages, and any cited papers are **not accessible** from the transcript. Their contents are not represented here. - **Transcript artifacts:** The transcript contains repeated phrases and likely transcription errors (e.g., "open core," "booth Eugene 19," "Engine then looks for patterns"). Terms are recorded as transcribed and flagged as unverified where relevant. --- ## 6. Self-Check - [x] Speaker identity and role are explicit in source (title and self-introduction). - [x] All promotional content is flagged in Section 3. - [x] Referenced sources are inventoried with attribution notes. - [x] Unsourced claims and speaker estimates are marked. - [x] Comment availability and missing evidence channels are stated. - [x] No quality judgment on the content is recorded. \>\> Testing. Awesome. Good morning everybody. Uh I know it's the last day of the conference, so super excited to have you here. Hopefully hopefully I can keep you engaged for the next uh 15 to 18 minutes or so. My name's Jake and I lead our labs team here at LangChain. And today I want to talk to you about a topic that I have uh been speaking a lot about with with our team and and with a lot of teams that I speak with who are building agents, and that's how to give your agents memory. Whether you're building a coding agent, a support agent, a uh a research agent, most teams arrive at the general same instinct that it should get better from one run to the next. So, let me give you an example to hold onto for the rest of this talk today. Recently my my the the labs team at LangChain have been working with a company in the financial services sector who's building a uh spending, budgeting, and non-investment advice agent. Given the highly regulated nature of the financial services sector, obviously you would imagine that it would need to follow some strict tone guidelines. And so, the way that it speaks and and the and its vocabulary and the way that it interacts with different users is obviously the the tell on whether or not it's following and abiding by the correct tone. It will derive the way that it generates this tone from the context and from the memories and from the references that it has to its rules and policies. And you can imagine in some contexts, and actually in some contexts that we experience, its tone would slip. And so, it would move from being advisory and helpful to actually pointed and directive, saying things like users in your similar situation would generally set a cap and it would actually say, "Oh, well, actually it looks like you need to cancel some subscriptions and move some money into savings." The company that we're working with thought that this is a violation of tone and there were similar instances of this, too. And so, to correct that, the review process is manual. You have to have someone understand the traces and understand where its tone slipped, make those changes to the context that it needs to reference when it's giving its tone, and then make sure that there are no regressions, as well. Obviously, that that doesn't scale, or rather it's challenging to scale. And so, the system should be or or the system around the agent should be a place where it can continuously learn and benefit from its interactions and its experiences. And that's the version of memory that I want to talk about today. To get there, I want to start with a tension that I think a lot of teams have run into. I think all of us here today, the reason why we're at this conference is we're curious, we're building agents. I think we've gotten very good at observing them, but observing them and then turning those observations and those agent experiences into lessons for future runs are obviously very different things. And so, that's where memory And so, that's where memory comes into play. Agents are producing more signal and more traces than ever before. And this is going to be something that's obviously that will obviously increase as more agents are built. That's fantastic for us because every run produces a trace and that trace is stored somewhere. That's incredibly useful. It means that we can understand the tools that were called, the reasoning that the agent made as it was on its way to making its decision, the artifacts that it referenced. Observability is obviously a critical foundation for building very reliable very reliable agents. But I think the challenge really is after the trace is stored or after the log or after the transcript of what the agent did is stored. If the future context that that agent references never changes, then a misguided skill or instruction will mean that the agent The mistake that the agent makes today will still persist in future runs. The goal or the system that we want to work towards is where a trace stops just being a log and becomes a source of signal. Memory. Agents that are observable tell you what happened. Agents that are adaptive are able to change are able to change and guide themselves on the following runs. So, I've mentioned memory. I think it will really help to sort of define what I mean when I say memory. It's the durable context that an agent references that will guide future runs. A trace, a transcript, or a log is evidence of what happened. It only becomes memory when that lesson is converted into durable context the agent can reference on future runs. Now, you'll see six things on this slide behind me. Facts, preferences, patterns, previous examples, skills, and instructions. I think that it's helpful to group them into three distinct buckets, a taxonomy that we borrow from cognitive science, and actually that applies a lot when we're talking about agents that are language-based. First, there's what the agent knows. This is often referred to as semantic memory. This includes things like the facts and the preferences that guide the way that it responds. Oops. Then there's the Then there's what the agent has experienced. This is the episodic memory. This is the learned patterns that it's seen, past interactions, and examples. And finally, there's how the agent should behave. This is also known as procedural memory. And that's the instructions, the skills that you're all probably very familiar with, and the rules that guide how it should how it should behave. Procedural memory is where actually most of the visible gains come from. So, in that financial assistance that I the example that I gave at the beginning, tone changes that we wanted to to promote to the agent and and make it follow came in the form of updating its procedural memory. We weren't giving it a new fact or we weren't telling it not to use a bad word. We were giving it a rule to follow as it engages with its users. That's the procedural memory that guides our agent. Another distinction that that I want to make and I think this is very important when thinking about the general agent agent memory paradigm is the separation between working memory or short-term memory and long-term memory. Working memory is what the agent is referencing in its current run. So, these might be things like intermediate scratch pads. It might be tool results. It might be the files retrieved as part of its operation. Everything that it needs to do to perform its current job. Long-term memory is what exists typically in some data structure or some store somewhere else that has instructions and skills and things that it needs to reference as part of future turns and and continuous behavior. Different agent harnesses will leverage and inject long-term memory in different ways, whether you're building your own SDK or or using a variety of different popular agent frameworks. Some inject it directly into the prompt. Some will use tools to retrieve that. Some will Some will pull it directly from a store. Some load it through files or some change the run time state. The The implementation can vary, but the distinction is really important. Some of the memory or some of the context that an agent references is temporary while some of it is durable and long-term and is useful to guide future runs. And I think that distinction is really important when we think about traces and logs and artifacts that the agent produces as it runs. A trace isn't just a record of the final output. It's a record of how our working memory changed over the course of the run. Some part of that should stay as history. Some part of that should become durable long-term context. And so that's what gives Oh, that's what lends itself to this really helpful mental model that I have or that I and and the team at LangChain and generally I think is is a very useful way to sort of think about the relationship between working or short-term memory and long-term memory. It's a read-write cycle. And so you might have heard of sleep-time compute or dreaming. Those terms refer to the way that memory is updated or pulled from long-term into short-term whilst the agent is doing work or actually in some cases whilst the agent is also not doing work. That's the sleep-time compute. If we start somewhere at the beginning of an agent run, it needs to read relevant context from long-term memory into its short-term memory. So this is anything This is anything important for the current run that it has It might be skills, it might be instructions. You've probably seen progressive disclosure with agents that actually pull in the relevant skills for a certain task. That's that phase moving from long-term memory into short-term memory. Then when the agent runs, it produces a trace. This evidence is really important to us because now we can understand how it retrieves context, the the tool calls that it made or the decisions or the sub-agents or anything that's part of its working job. As I mentioned, most evidence should stay as history. If we turned everything that was from a If we turned all of that evidence into long-term memory, then it would make the agent struggle and it would make it harder to reason over future tasks. And so what I think is really interesting is the filter step. How do you decide How do you decide what is useful signal and what should become durable long-term context and how do you decide what should stay as history. Once you've made that decision and we'll get into that in a moment. Once you've made that decision, that is that signal is then written back into durable long-term context. So, in the case of our financial assistant, that was be measured and not pointed or not directive, so that it can update the way that it behaves on future turns. And that's the core loop. Memory informs the run. The run makes the evidence. We filter that evidence to make signal, and then we use that signal to update memory. If you can build that flywheel, then we trend towards a place where we can build very reliable and very powerful powerful agents that just feel like they get better with time. And so, once we have that mental model, I think, you know, an even more simple way to break this down or to to abstract this away is is capturing what happened, analyzing what might matter, and then updating the context so that future runs can use this. The implementations around how different how how different people building different frameworks are still evolving, but this three-part shape is a really useful starting point. One implementation of this, and I won't I won't claim it's the only implementation, but just as just a kind of like put some concrete structure to what I'm talking about is the way that we do this with LangSmith. The capture step, to be able to understand and see the trajectory and the things that the agent did, is where our observability comes in. You're able to see the tool calls. You're you're able to see the way that the agents called sub-agents. You're you're able to see the context that was retrieved. The analyze step in the middle is something new that I'm really excited about, but basically, it is that intelligent process that's doing the background analysis and doing the background work to extract signal from traces based on the things that matter to you, and then promoting that signal to the memory store or the point of reference that the agent will use for future runs, and that is LangSmith's context hub. So, you can imagine now if there is a place where this analysis can be done, signal can be in track extracted, a remote store where you can now go and update what's in that so that the agent can pull that down the next time it runs. That is what LangChain's context hub does. And the abstract loop it maps cleanly. You capture the experience, you extract the signal, and then you store that durable context for future runs. To kind of put that in some more detail and and also to frame it in the in the the context of the financial assistant that I was speaking about at the beginning and and sort of the way that we we built this loop. When that agent runs, its context is loaded from the context hub and so that might include its instructions, its skills, relevant markdown files that matter to it, policies, and rules. When it functions or when we invoke that agent, then the a trace is produced that shows us the the uh failed tool calls or user corrections or in this case it flagged where our tone strayed from what we wanted it to be. Engine then looks for patterns and then identifies where different static skill markdown files that are based in context hub need to be changed or promoted, and it'll make that change. And then the next time our agent runs, we're able to actually go and and understand and see that its tone changes when it interacts with engages with users. So, some things that I and my team have currently uh or kind of always think about when when designing memory and and this should also as well ideally guide the the way that you design and build some of these abstractions is that agents produce so much we call it exhaust or we call it kind of like feedback. Not everything should become a memory update. And actually the devil is in the details around what should become one as as we've sort of discussed over the course of this presentation. It's understanding and deciding what matters to you. You know, most trace data actually should stay as history as referencable history. Some some might become data sets to do offline evaluations and things like that. A small a small segment of that or or subset should end up becoming what changes your agent's memory. Now, this second one is sort of like a gotcha that I I ran into a lot and I think that you sort of tread the fine line of trying to build a system that's optimized, but then also one that has a memory store or updates available to it in the hot path. And what I found was uh for particularly for long-running background agents, right? Imagine if you have an agent that's running over long time horizons and you make some memory updates in the beginning of its operation, but because of the run time state and because of the way that it's accessing memory, you don't actually access that or that's not available to it for future runs. And so, you know, understanding what you can cache, what you can't cache to be able to make sure that memory updates are available for future runs, very important uh and sort of I was I was banging my head on a wall for a while while while I was trying to sort of understand why my behavior wasn't changing and I needed to recognize and think about that second principle. And then the third one as well. I think that uh open core and uh you know, Hermes agent and and a lot of agents that are uh becoming widely popular that sort of do this self-update process, right? I think that it's another really important consideration of like what do you make self-updateable as opposed to what do you make something that needs human review? And so, if we're talking about procedural memory with instructions and policies and tone guidelines, those are probably going to drive about 75% of the behavior of my agent. And so, that actually probably should have some level of human in the loop and human interaction. You should have somewhere to surface it. The changes that you make to the parts of the agent's memory that are procedural before actually committing those and and putting those in the in the in the agent's sort of like hot reload path. So, protecting important behavior with the is, I guess, another really important third third kind of like level to this that I find really important. So, I'm going to come back to this initial refrain. The next run should be better. Memory allows us to do this. The word memory is thrown around a lot. I think in the context of building agents and working with harnesses and working with like providing context to your agent, memory really does give us the ability to do this. And it's more than just a place to store information. It's how experience becomes context. And then that context is how our next runs get the chance to improve. If you're curious about an actual implementation of this, by the way, that QR code has a video of me walking through how I built an agent that does exactly what I've just described. Uh hopefully that'll kind of like make it tangible and and will sort of connect some of those dots. Uh and if you want to have any further discussions about this, please come and see me. We're at booth Eugene 19. Uh I'd love to