Anant Srivastava, principal technologist for data and AI platforms at Oracle, argues that prompts, memory, and model weights are three tools for three different jobs—not a ladder to climb when answers go wrong. Using a support assistant that kept inventing product names after a fine-tune, he shows how retrieval architecture accumulates by accident and offers a diagnostic: prompts for small, stable behavior; memory for knowledge that's current, large, citable, or access-controlled; weights only for reflexes that have stopped changing. He closes with a circulating architecture where the agent improves by doing its job. Topics include treating every prompt edit, indexed doc, and training sample as an architecture decision; why you fine-tune reflexes, not facts; and moving knowledge between context, memory, and weights over time. Recorded at AI Engineer World's Fair 2026 in San Francisco. # Channel Identity Observations — Video `01a15ca7-1bd8-7db7-a89e-11b6d54c81d7` ## Video Presentation Overview - **Title:** *Stop Fine-Tuning to Fix Retrieval Problems — Anant Srivastava* - **Format evidence:** A delivered talk with a live audience — the transcript includes an interruption ("**\>\> good afternoon. Hello. \>\> Oh sorry that was I was like who's that?**") and closes with "**Thank you. Thanks for coming up**," indicating an on-stage presentation, likely at a conference or meetup. - **Subject matter:** Enterprise AI architecture — specifically deciding whether knowledge should live in the **prompt**, **memory/retrieval (RAG)**, or **model weights (fine-tuning)**. ## Explicit Self-Description The transcript contains **no direct self-description of the channel itself** — no name, mission statement, or audience pitch. The only identity signal in the title is the speaker's name, **Anant Srivastava**, presented as the talk's author. - The speaker references personal experience: "**Even I have made such decisions in my in my journey**" — a first-person claim of practitioner experience, but no stated role, employer, or credentials. ## Observable Presentation Patterns ### Intended Audience (Demonstrated) The talk is explicitly framed for **enterprise AI/ML practitioners and technical decision-makers**: - "**For most enterprise AI systems, the interesting engineing problem is not the model itself**" — opens with an enterprise framing. - Assumes audience familiarity with concepts used without definition: **fine-tuning, RAG, chunking, denormalization, context windows, frontier models, drift monitoring**. - Speaks to team roles the audience would recognize: "**the product manager makes some changes to the prompt**," "**an engineer shuffs the product catalog into the prompt**," "**the machine learning team decides to train the model**." - Audience interaction assumed: "**have you heard of medical claims coding? I think we must have**," and "**let me make it a little concrete for you all**." ### Claimed Scope of Content The talk claims a **decision framework ("the lens") for knowledge placement in AI systems**, positioning itself as opinionated and corrective: - "**Today I'm going to argue that this is a decision that most enterprise teams make by accident**" — an explicit thesis statement. - "**These are three different tools for three different jobs**" — the core claim being argued. - Offers a takeaway artifact: "**if you don't remember anything from the talk, this is a simple one, right? This table helps me decide**." - Acknowledges limits of the framework: "**Of course there can be exceptions. I'm not saying this is always the rule**." ### Teaching / Educational Style (Demonstrated) - Structured as a lecture: "**I'm teaching it**… the rest of the talk is is about the lens**." - Uses **diagnostic questions** as a recurring device: "**is the knowledge too large to fit changing faster than I can retrain**," "**has the information stopped changing have the humans converged**." - Uses **worked examples**: an internal support assistant gone wrong, a SaaS customer support agent, a code assistant with org-wide repo access, content moderation/claims processing, and medical claims coding (citing "**Mount Si Mount Sinai and IMO health**" and ICD-10 codes). - Summarizing mnemonic: "**prompt is more about behavior. Memory is about what to know and facts and weights are about how to reason**." ### Positioning / Point of View The talk takes a **contrarian-to-common-practice stance** on fine-tuning: - Title claim: "**Stop Fine-Tuning to Fix Retrieval Problems**" — echoed in the body: "**instead of fixing the retrieval problem, the team actually went ahead and fine-tuned the model. That's not what you should do**." - Nuanced rather than absolute: fine-tuning is endorsed for "**reflexes, not facts**," stable centers of human judgment, and cost reduction ("**you can use a small model, fine-tune it and run the volume cheap**"). ## Metadata Context No curated metadata was supplied beyond the title; no additional identity context is available from this source. ## Summary for Channel-Level Synthesis - **Demonstrated audience:** technical enterprise AI builders (engineers, ML practitioners, product/engineering leaders). - **Demonstrated scope:** architecture decision-making around prompts, retrieval/memory, and fine-tuning, delivered as a framework-driven, example-heavy argument. - **Demonstrated tone:** opinionated practitioner talk — argues a thesis, admits exceptions, draws on first-person experience and named real-world cases. - **Claims vs. demonstration:** the creator claims teams misplace knowledge "by accident" (an argued position, not a demonstrated fact about all teams); the material demonstrates an educational, framework-oriented presentation style with enterprise focus. # Delivery Observations — Anant Srivastava, "Stop Fine-Tuning to Fix Retrieval Problems" ## Overall Communication Style - **Conversational, spoken-first register.** The delivery includes natural disfluencies and self-corrections throughout: *"the path to rec to to making it on purpose"*, *"your model, your your architecture was accumulated"*, *"fine-tune, I meant not retrain, sorry, fine-tune"*. This reads as an unscripted, live talk rather than a read script. - **Direct audience address.** Frequently uses second person and rhetorical questions: *"How do you decide what to put where?"*, *"Would you fine-tune your model to learn the code? No."*, *"How did that happen?"* - **Real-time signposting.** Regular verbal transitions organize the talk: *"Okay, let me make it a little concrete for you all"*, *"Now let's come to the more interesting part which is weights"*, *"Okay, that's the architecture."* ## Recurring Teaching Habits - **Job/diagnostic framing.** A stable structural pattern: each layer (prompt, memory, weights) is explained via its *job*, its *wrong job*, and a *diagnostic question*: - *"The job of the prompt is behavior, is tone, is the persona"* - *"The wrong job for the prompt is to store facts"* - *"The diagnostic for prompt is... small, stable and about how to behave rather than what to know"* - *"The diagnostic for fine-tuning is has the information stopped changing"* - **Rule-of-three vocabulary.** A recurring triad organizes the whole talk: *"prompt is more about behavior. Memory is about what to know and weights are about how to reason."* Also *"small, stable, and editable"* and *"current, large and citable."* - **Concrete scenario-based examples.** Abstract claims are repeatedly grounded in worked scenarios: the support assistant with a leaked product catalog, the code assistant over repos, medical claims coding (ICD-10). Patterns: *"Let me make it a little concrete for you all."* - **Rhetorical question followed by immediate answer**, especially in the fine-tuning sections: *"So would you fine-tune your model to learn those codes?... You won't. That's fact. That's knowledge."* - **Antipattern-then-correction.** Repeatedly names the mistake first, then the fix: *"instead of fixing the retrieval problem, the team actually went ahead and fine-tuned the model. That's not what you should do."* - **Hedged absolutism.** Strong claims softened with qualifiers: *"Of course there can be exceptions. I'm not saying this is always the rule."*, *"perhaps"*, *"typically"* (*"You'd fine-tune your model to learn reflexes, not facts typically"*). ## Signature Phrases - *"That is the accident"* / *"by accident"* — the central recurring motif of decisions made unintentionally. - *"Reflexes, not facts"* — the most quotable compressed formulation, repeated across examples. - *"The lens"* — the framing device: *"the rest of the talk is about the lens."* - Filler-adjacent tics: *"Okay"* as a frequent segment closer; *"um"*, *"right?"*, *"you know what?"* recur throughout. ## Notable Moments - **Handling an interruption.** A mid-sentence audience/event interruption (*"good afternoon. Hello."*) is acknowledged lightly — *"Oh sorry that was I was like who's that?"* — before resuming the same sentence where he left off. One instance; recovery pattern uncertain as a stable habit. - **Self-disclosure of fallibility.** *"Even I have made such decisions in my in my journey... without really giving it a lot of thought."* Single occurrence in this transcript. - **Memorable closing line:** *"the model is the easy part. What you build around the model, the harness around the model... is the key architecture that you got to build."* ## Confidence Notes - The **job/wrong job/diagnostic** structure and **behavior/memory/weights** triad are clearly stable patterns, each appearing multiple times. - The interruption recovery and self-disclosure are **isolated moments** — insufficient repetition to characterize as habits. - Filler frequency ("Okay", "um", "right?") is high and consistent across the transcript, suggesting a genuinely improvised delivery style. # Position Record: "Stop Fine-Tuning to Fix Retrieval Problems" **Video ID:** `01a15ca7-1bd8-7db7-a89e-11b6d54c81d7` **Speaker attribution:** The transcript has no in-speech identification. The speaker is named as **Anant Srivastava** in the video title, and all positions below are attributed to that speaker on that basis. --- ## 1. Core Thesis > The key architecture decision is how knowledge reaches inference: through the **prompt/context**, through the **memory/retrieval layer**, or through **model adaptation (fine-tuning)**. - **Claim:** Most enterprise teams make this decision **by accident**, not by design. - **Framing:** The three options are **"three different tools for three different jobs,"** not rungs on a **ladder** to climb. - **Stated reasoning:** Teams escalate when outputs are wrong (prompt → memory → fine-tuning), and every routine edit silently makes an architecture decision anyway. --- ## 2. Two Mechanisms of Accidental Architecture | Mechanism | Description (as stated) | |---|---| | **Escalation** | Wrong answer → fix the prompt → fix memory/retrieval → fine-tune. The speaker says fine-tuning "doesn't happen very often." | | **Unowned accumulation** | Every prompt edit, indexed document, and training sample "decides" where knowledge lives, even when no one intends it to. | The speaker states that architecture was **"accumulated, not designed,"** and that **"everybody owned a piece of it" and nobody owned the whole.** --- ## 3. Illustrative Anecdote: Internal Support Assistant The speaker presents a sequence of plausible-looking changes: 1. **Product manager** edits the prompt to fix tone → *prompt decision* 2. **Support team** adds refund policy documents to retrieval/memory → *memory decision* 3. **Engineer** pastes the product catalog into the prompt → *prompt decision* 4. **ML team** fine-tunes on support tickets from up to 6 months prior → *weights decision* **Stated outcome:** After the new catalog launches, the agent still produces **product names that don't exist.** **Speaker's explanation:** The product catalog "leaked into the weights" through fine-tuning on stale tickets. > The speaker frames this as an example they have **witnessed** and also **personally committed** ("I have made such decisions... without really giving it a lot of thought"). *Note: This is a single illustrative case. No frequency or outcome data is presented.* --- ## 4. The Three Lenses ### 4.1 Prompt: Behavior - **Job:** Behavior, tone, persona. Content that is **small, stable, and editable.** - **Wrong job:** Storing facts. The speaker cites unnecessary cost and the **"lost in the middle"** problem with large token counts. - **Diagnostic:** *Is this small, stable, and about how to behave rather than what to know?* - **Example given:** A SaaS customer-support agent with a professional tone, human escalation after three failures, and concrete next steps. "None of this changes per query per user." - **Scope clarification:** "Prompt" includes authored instructions generally, such as **MD files**, not only the system prompt. ### 4.2 Memory: Knowledge (Agent and External) - **Job:** Ground the agent in knowledge that is **current, large, and citable.** - *Current:* changes faster than fine-tuning cycles. - *Large:* cannot fit in the prompt. - *Citable:* production systems must point to sources. - **Scope:** Covers both **agent memory** (knowledge about a user) and **external memory** (enterprise knowledge retrieved via RAG, managed by an external application). - **Wrong job:** Shoving behavior in, or expecting memory to supply reasoning. - **Reasoning caveat:** Graph-style RAG "will help the agent do reasoning, but that's it." The speaker asserts that **reasoning capability must come from the model:** *"if your model cannot reason over two or three or five documents, it won't be able to reason over 50."* - **Access control:** The speaker treats per-user scoping as a reason to keep knowledge in memory, since it allows control over *who sees what.* - **Diagnostic:** *Is the knowledge too large to fit, changing faster than retraining, and/or scoped (user, repo, etc.)?* - **Example given:** A code assistant with access to organization-wide repositories. - Not fine-tuned on code, because code "changes very often." - Not stuffed into the prompt. - **Recommended practices:** - Code-aware chunking - Denormalized metadata (repo, permitted committers) - Filtering, to avoid "a RAG mush" of irrelevant chunks ### 4.3 Weights: Reflexes - **Job:** Encode what has **stopped changing**, specifically **reflexes**: input formats, selection among formats, and similar patterns. - **Driving axis:** **Rate of change** is the only axis the speaker uses for this lens. - **Qualified caution:** The speaker does **not** recommend freezing what is stable and retraining freely. Retraining requires "a very strong reason." The method is to **lock in what is changing, then proceed from there.** Freezing a boundary that is still moving is the risk. - **Counterexample:** Fine-tuning an internal document assistant on runbooks and procedures. Speaker's assessment: this was a **retrieval problem misdiagnosed as a knowledge gap**, and runbooks are **facts** that belong in memory. - **Positive use case (conditional):** Hard, ambiguous domains (content moderation, claims processing): 1. Start with a frontier model producing recommendations, with humans correcting inconsistently. 2. Patterns emerge: a **stable center** and a **contested edge.** 3. Fine-tune on the stable center, **monitoring for drift.** - **Medical claims coding example:** The speaker cites ICD-10-style codes (approximately **70,000**) mapped from doctors' notes. Position: do **not** fine-tune to learn the codes (they change occasionally, and fine-tuning requires substantial training data). Fine-tuning may instead teach **input-format understanding and format selection.** --- ## 5. Diagnostic Summary - **Prompt:** Is it small, stable, and about *how to behave*? - **Memory:** Is it too large, changing faster than retraining, and/or access-scoped? - **Weights:** Has the information **stopped changing**, and are humans **converged**? And is the reason **capability** or **cost**? **Speaker's final rule of thumb:** *Prompt is about behavior, memory is about what to know (facts), and weights are about how to reason.* **Stated caveat:** "There can be exceptions... I'm not saying this is always the rule." **On fine-tuning motivation:** The speaker says the reason is usually **cost, not capability**, since "frontier models are quite capable." Small models fine-tuned on a stable pattern can "run the volume cheap." --- ## 6. Architecture: Circulation Loop The speaker describes a **harness** in which knowledge moves between layers: - **Prompt/context → memory:** Signals generated in a session may become durable memory. *(Described as already common practice.)* - **Memory → prompt/context:** Relevant items are pulled into context at session start. - **Memory → weights:** Recurring retrieval patterns and formats the model "should reflexively know" are fine-tuned in. - **Weights → memory:** Once a model knows formats, examples no longer need to be retrieved each time, which changes what is worth retrieving. The speaker says teams have built the first flow but are focusing on the **memory-to-fine-tuning** flow, and describes the result as a loop in which **"the agent gets better at its job by virtue of doing its job."** **Closing position:** "The model is the easy part." The durable differentiator is the **harness** that places information correctly and circulates it. --- ## 7. Qualifications, Tensions, and Open Points - **Tension: "Reflexes, not facts, typically."** The speaker says fine-tuning is for reflexes, but also says it **should not** be used for ICD codes, which are arguably formats/patterns. The boundary is not fully specified. - **Tension: Capability vs. cost.** The speaker says frontier models are capable and fine-tuning is usually about cost, yet presents content moderation as a fine-tuning case without addressing whether cost or capability is the motive there. - **Tension: "Most teams decide by accident."** The claim is about most enterprise teams, but the only evidence offered is one illustrative anecdote and personal experience. No data is presented. - **Unresolved: Drift.** The speaker says "monitor for drift" in the stable center but does not describe how. - **Unresolved: Reasoning limits.** The speaker says models must reason well enough to use retrieved documents but gives no criteria for assessing reasoning capacity. - **Incomplete claim:** A reference to a partner organization (Mount Sinai, IMO Health) is cut off mid-sentence in the transcript. The intended claim is **not recoverable** from the transcript. - **Hedged position:** Exceptions to the three-lens rule are acknowledged but not enumerated. --- ## 8. Transcript Notes - An unattributed interjection ("good afternoon. Hello.") appears mid-talk and is treated as an audience or environmental event, not as speaker content. - Several phrases appear to be transcription artifacts (e.g., "I'm teaching it" likely refers to "treating it," "graphical kind of rag" likely refers to graph-based RAG). These are noted without substitution. - Speaker self-corrects once: "fine-tune, I meant not retrain, sorry, fine-tune." The corrected term is used here. # Speaker & Provenance Record — Video `01a15ca7-1bd8-7db7-a89e-11b6d54c81d7` ## Speakers & Roles | Speaker | Role | Basis | |---|---|---| | **Anant Srivastava** | **Presenter / conference speaker** | Named in video title ("— Anant Srivastava"); delivers the full talk, refers to "the rest of the talk" and "the lens we are going to talk about" | | Unidentified voice | Audience member / second speaker | Brief off-mic exchange: *"good afternoon. Hello."* answered by the presenter ("Oh sorry that was I was like who's that?"); no identity available | - **Narrator/Guest:** Not applicable — single continuous presentation format. - **First-person experience claims:** Presenter states he has personally made such architecture decisions *"in my journey"*; also says *"I've seen this play out"* regarding an internal support assistant example (anecdotal, not attributed to a specific organization). ## Referenced Sources & Third-Party Mentions - **Mount Sinai and IMO Health** — referenced in connection with medical claims coding (ICD-10 codes, doctor's notes mapped to codes). The transcript is truncated mid-sentence ("Mount Sinai and IMO health actually") and the number "~70,000 such codes" appears in an interpolated second voice line; attribution of that specific claim is **ambiguous** between presenter and the other voice. - **ICD-10 codes** — cited as a domain example of stable, non-changing facts. - **Concepts referenced without attribution:** "lost in the middle" problem, RAG, code-aware chunking, frontier models, fine-tuning. No formal citations, papers, or links are given. ## Promotion Check - **No promotional segments identified.** No product, company, course, or service is pitched; conclusion is a summary of the architectural argument only. ## Third-Party Claims Inventory - Rates of human overrides "flatline" over time in content-moderation/claims workflows — presenter's own claim, no source cited. - Frontier models "are quite capable" and fine-tuning is usually cost-driven — presenter's claim. - Claim that fine-tuning on support tickets caused product-catalog leakage into weights — anecdotal scenario presented as observed experience, no corroborating source. ## Availability & Provenance Limits - **Curated metadata:** Title only; no description, speaker bio, event name, or date supplied. - **Comments:** No information available about comment availability. - **Cited external documents/links:** None present in transcript. - **Untrusted content note:** All claims above are recorded as provenance facts from the transcript; no quality evaluation is made. ## Self-Check Result - All named roles trace to the title or transcript; unnamed interjection explicitly flagged. - No promotion detected — nothing to flag beyond the explicit "none" finding. - No timestamps, quotes, or sources invented; ambiguous attribution (Mount Sinai/IMO code-count line) marked as uncertain. How do you decide what to put where? For most enterprise AI systems, the interesting engineing problem is not the model itself. The model consumes tokens and produces tokens. Your knowledge lives in files, databases, and APIs. The key architecture decision is how that knowledge reaches inference um from the prompt or the context through the retrieval or memory layer or by model adaptations such as fine-tuning. Today I'm going to argue that this is a decision that most enterprise teams make by accident. And the path to rec to to making it on purpose is to recognize that this is not a ladder that you climb. These are three different tools for three different jobs. So how is this an accident? Well, there are two mechanisms. One is the obvious one. The other one not so much. So the obvious one is that teams escalate. You don't get the right answer from your model. The easy thing is go put something in the prompt like go fix the prompt. Still wrong? Maybe there is something with the memory system or the retrieval system. Let's take a look there. Even then, if you're wrong, hey, you know what? Let's fine-tune the model. Doesn't happen very often, but yes, sometimes teams reach out to fine-tune the model. There is a layer underneath it. The teams that are not escalating are still making these decisions every day. Every edit that you make to the prompt decides what behavior lives in the prompt. Every document that you index decides what knowledge is held in the memory. And every training sample that you bake in will every every training sample that you identify will bake something into the weights. Um all these are architecture decisions. None like that. Okay, let me make it a little concrete for you all. Um, so and I've seen this play out actually. Uh, an internal support assistant just released. In a few weeks, the product manager makes some changes to the prompt to fix the tone of the support assistant. After a few weeks, the support team adds refund policy documents to your retrieval on memory systems. I'll talk about retrieval and memory. I'm teaching it. I'm I'm treating it all as memory, but I'll come to that. Um, and then since your model is not your your agent is not giving you the right answers, an engineer shuffs the product catalog into the prompt. And after a few weeks, the machine learning team decides to train the model or fine-tune the model on support tickets from 6 months ago from up to 6 months ago. All decisions look fine look okay except one by the way. But even I have made such decisions in my in my journey. I have made such decisions of doing such things without really giving it a lot of thought. Um, so when the new product product catalog has launched, looks easy, right? Oh, the engineer has shoved the product catalog into the prompt. Let's put the new product catalog there. And you would think that everything is working fine, but your model or your agent is still giving you product names that don't even exist. How did that happen? Well, the product catalog leaked into the weights when you fine-tuned it on your support tickets. And that is the accident. Your model, your your architecture was accumulated. It was not designed. That is the accident. It is what 6 months of normal product work led to. Nobody owned it. Meaning everybody owned a piece of it. Okay. Um, nobody asked the diagnostic question. Nobody asked where this should where this thing should live. That diagnostic those diagnostic questions are the lens we are going to talk about and the rest of the talk is is about the lens. Okay. So the first part is prompt. What is the job of the prompt? The job of the prompt is behavior, is tone, is the persona of the agent. Things that are small, stable, and editable. The wrong job for the prompt is to store facts. You don't put product catalog into the prompt because you are unnecessarily providing context to your model and paying the cost for it. And when you have too many tokens, you run into lost in the middle problem. The diagnostic for prompt is is the small knowledge small stable and about how to behave rather than what to know. A concrete example for a good prompt use case is a customer support agent for a product SAS. Right now this agent is has a professional tone. It is it offers human escalation after three failures and it provides you concrete next steps. None of this changes per query per user. This is this this exact thing belongs to the prompt. And one other thing is prompt is something I'm not only talking about system prompt. I'm talking about authored instructions for your for your model. So they could also come from MD files. They could also come from other things, right? But this is what I mean by prompt here. The second path is memory. The job of memory is to ground your model or your agent in in knowledge that is current, large and citable. Current meaning it changes faster than you can fine-tune. It changes every now and then. Large is because it cannot be held anywhere. It cannot be held in your prompt. and citable because production AI systems need to be able to point to the source of the knowledge. This is the this is what belongs to memory. Now when I use the term memory people think of agent memory knowledge that is acquired about people about people who about a person who is interacting with the agent right what I'm also referring to here is external memory which is which you retrieve using rag and all that like the refund policies right it's not being managed by your agent it is managed by an external application that is included here so I'm talking both about the agent memory and the memory Mor of the enterprise. Okay. The wrong job of for memory is shoving behavior in or thinking about how to reason with the memory. Now, of course, you can have a graphical kind of rag which will help the agent do reasoning, but that's it. That's where you stop. For reasoning, you got to have the right model. Because if your model cannot reason over two or three or five documents, it won't be able to reason over 50 that you provide. The more context doesn't help if it's not able to reason. So the diagnostic or the question you need to ask is is the knowledge too large to fit changing faster than I can retrain. And another important aspect is whether it has some access control whether it is scope per user. You have got to have that in memory. If there is some kind of access control that this user cannot see the other person's information or the data or whatever, you've got to have that knowledge in memory because then you can control who sees what. A good example for storing something in external memory or memory is a code assistant, right? Which has access to your all your repos as an organization. So uh would you fine-tune your model to learn the code? No. It's it's a simple one, right? It changes very often. You wouldn't do that. You'd finetune your model to learn reflexes, not facts typically. And would you stuff it in the prompt? No, you don't stuff such things into the prompt. You may stuff or you may add certain things that are important in the context window, right? That's something different. But you wouldn't stuff the entire code base into the prompt. Okay? Or or even a part of it if it has no use there. Uh now the other important thing is um when you build a rack system right like for for for your code for your code you need to make sure that you are chunking properly you are using code aware chunking with a perhaps uh and then the other important thing is use filtering. You should denormalize. You should perhaps denormalize things such as who is the what is the repo this code is for. You get a code chunk. You have to create metadata about which repo it belongs to. What is the user what are the users who are allowed to commit and all that because that is what is going to help you get the exact data that you need versus creating a rag mush where you are returning a lot of chunks back from the database without and and then confusing the model. So again the diagnostic is something that's too large to fit changing the faster than you can retrain the model and it's scoped at a certain level. It could be it could be a user, it could be a repo or whatever. All that knowledge belongs to memory. It could be external memory which is your like your applications database or it could be your agent memory as well. Now let's come to the interest more interesting part which is weights. So what belongs to the weight? Something that stopped changing. That is what belongs to the weight. And the only axis that drives this slide is rate of change. You can lock in what has stopped changing and perhaps retrain. And I'm not saying just lock in what stopped changing and just go retrain your model. No, that's not what you do all the time. You would you have to have a very strong reason to retrain your model. So what you do is you lock in what's changing and then go from there. But if you get it wrong, then you could freeze a boundary that's still moving. Okay? And it's not as subtle as like, hey, can I re would I re Nobody's going to retrain their model on like or fine-tune, I meant not retrain, sorry, fine-tune their model on pricing list. Nobody does that. It is more subtle than that, right? And a current version of that is um there is an let's say there is an internal document assistant and the support team or the or your team starts decides to fine-tune the model because it's not giving the right answers. So to teach the domain, it says, you know what, let's fine-tune this model on on our documentation. Let's fine-tune the model on or or maybe not all your all their documentation but runbooks and procedures. And what happens with that? There is an issue. If you do that when a your model will be giving you information that's old. This was actually a retrieval problem. If the model was not giving the right answer, what the model needed was the right chunks, the right pieces of documentation. So basically instead of fixing the retrieval problem, the team actually went ahead and fine-tuned the model. That's not what you should do. Runbooks, procedures, these are facts. These belong to either your me either your agent memory not there but of course your external memory. That is where these things should reside. Not if the if the model is not \>\> good afternoon. Hello. \>\> Oh sorry that was I was like who's that? So if your if your model is not giving the right answers, fix the retrieval problem. Don't just go ahead and fine-tune your model. Okay. Uh and if you flip it, same axis. If you reverse it, u a good use case where I could fine-tune is let's say it's a hard ambiguous problem such as content moderation or claims processing. Right? When you begin, you can use a frontier model that makes recommendations and humans correct it although inconsistently. Over a period of time, patterns will emerge. Patterns will emerge and you will have sort of a center which is sort of fixed and stable. Okay? And then you will have the contested edge. the the rates of human overrides will flatline over after a certain point and you'll you will most likely emerge in a pattern where you have a contested edge where you still need humans but a center which is pretty stable. Okay, now that center is something you can fine-tune your model on. Okay, but when you do that you have to monitor for drift. Is there a drift in my center? And if that happens then you're going to run into problem. And um actually a stable version of that is I don't know have you heard of medical claims coding? I think we must have like ICD 10 codes doctor's notes are mapped to like codes for different purposes like insurance purposes and all that. So Mount Si Mount Sinai and IMO health actually \>\> there are 70,000 of such codes by the way. So would you fine-tune your model to learn those codes? They don't change so often. You won't. That's fact. That's knowledge. That's you should not be looking at fine-tuning your model. One of the obvious reasons is is that they change sometimes. And the other is you will need a large test training data to even fine-tune your model on it. Okay. What you fine-tune our model is on the reflex whether whether it is understanding the format of the input and whether it is able to and when it you have to make selection across the formats like which one is the right one that's where you fine-tune your model to do that. Okay. So the diagnostic for fine-tuning is has it has the information stopped changing have the humans converged and more important you should either h have one of these two reasons is the is are you fine-tuning because your model is not capable or are you fine-tuning to save cost and a lot of times it's not capability frontier models are quite capable it usually comes down to cost where you can have a small if you have nice pattern in your work, you can actually perhaps use a small model, fine-tune it and run the volume cheap, right? So, if you don't remember anything from the talk, this is a simple one, right? This table helps me decide. You can b you can bounce your current AI system against this and and decide this is the lens basically that helps you decide what lives where. Of course there can be exceptions. I'm not saying this is always the rule. There can be exceptions to this but prompt is more about behavior. Memory is about what to know and facts and weights are about how to reason. Okay. And this is the architecture. Your prompt or your context window, your memory and your finetuning needs to circulate. You need to build a right harness around it so that these things can circulate. Your prompt or your context window has information. It generates signals. Some of them becomes durable memory. Okay. Now when you are restarting a new session things get pulled out of from the memory into the prompt or context window and that's when you're going from the memory to the prompt. The more so this is lot of teams have built this architecture but what people are looking to are focusing on building is the right side the memory to fine-tuning flow. Okay. Now what happens is when you build an agentic system over a period of time you will have retrieval patterns. You will have formats that you think the model should reflexively know. So those things move from your memory to fine-tuning once you have fine-tuned. So that's memory to weights. Once you have fine-tuned what is worth retrieving changes if you have changed if you have fine- tuned your model on the formats of like notes and the ICD codes you don't have to retrieve certain examples from your memory or external memory each time and provide it to the model because now the model knows the format reflexively you don't have to provide examples to it that hey this is the format or that is the format so that's weights to memory and so basically This becomes a loop over a period of time and the agent gets better at its job by the virtue of doing its job. Okay, that's the architecture. So what I would like to conclude with is the model is is the easy part. What you build around the model, the harness around the model that helps you store the right information at the right place and circulate among them is the key architecture that you got to build. Thank you. Thanks for coming up.