Raymond Feng (Applied Compute) explains post-training custom enterprise models with reinforcement learning that keep learning on the job. He describes an orchestrator that fans interactions out to inference engines, collects graded rollouts, and feeds a GRPO-style training loop applied to real multi-turn, long-horizon work rather than toy question-answer pairs. Key challenges covered include reward hacking, faithfully replicating production environments for training, why replaying real customer traffic breaks on non-replayability and off-policy data, and where automated data pipelines and self-evaluation may lead — toward a model that treats every interaction as new training signal. (Conference talk; topics inferred from title, description, and timestamps.) # Observation Document — Video: "Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute" ## Explicit Self-Description Evidence - **Speaker identifies as presenting "frontier work on post-training"**: > "Today, I'm going to be sharing some of our frontier work on post training and how we envision a future where agents can learn new skills on the job." - **Affiliation signal in title**: The video title credits **"Raymond Feng, Applied Compute,"** — the only stated organizational attribution in the material. No further self-description of the organization appears in the transcript. - **Talk format**: A conference/event presentation with a host ("thank you, Jack"), an audience Q&A offer, an on-screen email contact, and closing applause — indicating a **public speaking venue rather than a solo studio video**. ## Intended Audience (Observable) - **Practitioners/researchers in ML post-training**: The talk assumes familiarity with terms used without definition — **GRPO, rollouts, off-policy data, self-distillation, reward hacking, weight updates, task traces**. - **Enterprise-oriented framing**: Repeated references to enterprise deployment needs suggest an audience including enterprise AI customers or those serving them: - > "an increasing demand for agents that can just be deployed in a plug-and-play way into how enterprises use the agents" - > "we can meet customers where they're at" - > "existing enterprise harnesses" - **Research-community awareness**: The speaker cites related work by name and recency — > "a related work by Nvidia… a paper from around a month ago where they introduce… Polar" — signaling an audience tracking current literature. ## Claimed Scope of Content The speaker explicitly frames the talk as **three escalating levels of post-training**, each "build[ing] on top of the last": 1. **Simple single-turn Q&A training** inside a fully controlled training stack (orchestrator, grader, training engine, inference engines). 2. **Synthetic environments** for long-horizon tasks, with environment state externalized; challenges named explicitly: **environment fidelity and reward hacking** (e.g., the "potholes" short-response example; sandbox-timeout abuse). 3. **Bring your own harness** — training against real customer/enterprise harnesses as black boxes, with challenges of **non-replayability and off-policy data**. 4. Plus a stated **future vision**: "agentic citizens" — > "which you can just deploy once, and they'll be able to adapt to many different types of out of distribution tasks and learn from their interactions." ## Demonstrated Presentation Patterns (Observable, Not Claimed) - **Pedagogical analogy use**: Compares training stages to human learning — > "kind of like internships where you want the model to do a specific task, but you don't necessarily know how exactly the task will play out." - **Concrete failure anecdotes**: Cites specific past training runs (10% tool-call failure rate producing shorter responses; sandbox timeout abuse) — demonstrates the channel/creator presents **first-hand experimental evidence** rather than only theory. - **Honest limitation disclosure**: Flags open problems rather than only successes — > self-distillation is "still… relatively like narrowly scoped," and qualitative feedback ingestion is "still a pretty open question." - **Closing with external citation**: Ends on a quote from a paper "a year ago": > "AI is at the cusp of a new period in which experience will become the dominant medium of improvement." ## Claims vs. Demonstrations — Distinction Summary | Type | Evidence | |---|---| | **Creator claim** | "frontier work," vision of "agentic citizens," optimism that human-style learning "should be possible… for models as well" | | **Demonstrated** | Detailed technical walkthroughs of training architectures, named failure modes from actual runs, citation of external research (Nvidia's "Polar") | | **Not established in material** | Any channel-wide description, audience size, publication cadence, or organizational details beyond the title's "Applied Compute" attribution | ## Notes for Later Synthesis - No curated metadata was supplied; all observations derive from title and transcript only. - The speaker's personal identity beyond name and affiliation-by-title is not established; no private motivations are inferred. # Delivery Observations: Raymond Feng — "Learning on the Job: The Future of Post-Training" ## Overall Tone & Register - **Conversational, technically informal**: Frequent filler words ("um," "uh," "like") appear throughout, giving a talk-track style rather than a polished script. - > "Um, really grateful for the opportunity to speak here." - > "which is like the traditional method, you would want to have many rollouts" - **Modest, hedging qualifications** when describing his team's progress: - > "we've sort of, I would say, mastered or gotten a lot of reps with these simple single-turn Q&A tasks" - > "it's still an open uh, research question of like how general can we push it?" ## Recurring Vocabulary & Phrases - **"Like" as a discourse marker** — the single most frequent filler, used to introduce examples and soften statements: - > "so, for instance, if they already have some method of calling the agent... they would want to be able to train a custom model" - > "if a human was in a customer support chat, they could understand somehow that like based on the customer's reaction..." - **"Sort of" as a recurring softener** (appears many times): - > "One way that we kind of think of this, uh, is a framework..." - > "And we kind of think of those kind of like internships" - > "we're sort of constraining everything to be inside of our training stack" - **"Kind of" used the same way** — a stable pattern, not isolated: - > "This sounds really good, but um... the challenge here is in the like data where" - > "you're kind of playing a game of Whac-A-Mole" ## Teaching Habits & Structure - **Analogy-driven explanation**, especially human-learning analogies — a stable feature: - > "a framework comparing it to how humans do learning, where you learn simple tasks first" - > "we kind of think of those kind of like internships" - > "if you think about maybe the model is like a human like walking along a sidewalk and like the tool call failures are like potholes in the sidewalk" - **Concrete real-world example after each abstract concept**: - Environment fidelity → the networking-failure / short-response anecdote: > "we had some like networking issues causing our environment to have tool calls that failed maybe around 10% of the time" - Non-replayability → the customer support chat example: > "there's not really a way that you could then go back and think, 'Oh, if I like said... would the user have happier?'" - **Progressive/laddered structure** — explicitly signposted: - > "a few different levels of post training, where each one builds on top of the last" - Transitions like "So first, I just want to talk about..." → "So, that brings us to our next topic of bring your own harness" → "So, I want to talk about some of the frontier research directions" - **Enumerated lists** for research directions: - > "There's kind of three main topics, which are self-distillation, automated data pipelines, and qualitative feedback ingestion" - **Signposting key takeaways** with explicit markers: - > "The key thing to note here is that..." - > "The main thing to highlight here is that..." - > "what's important here is that..." ## Rhetorical Patterns - **"This sounds really good, but..." pivot** — repeated verbatim-style twice in consecutive sentences, apparently a self-restart: - > "Now, this sounds really good, but um This sounds really good, but the challenge here is in / The challenge here is in the like data" - **Hypothetical/imagined dialogue** as an explanatory device: - > "would the user have happier?" (imagining the model's counterfactual) - > "the customer had this like piece of feedback" - **Optimism framed via human analogy**: - > "we're optimistic because like we think that humans can do this kind of learning, and so it should be possible to like formulate some kind of method that would work for models as well" ## Self-Corrections & Disfluencies - **Mid-sentence repairs and restarts** are common, suggesting extemporaneous delivery: - > "to do a task, to do a task, they would want..." (immediate repetition) - > "Um basically one one like / Or sorry, one challenge that we've seen is like..." (explicit self-correction) - **Trailing or restructured clauses**: - > "and it may create an update. So, an example would be for GRPO..." (unclear phrasing left as-is) ## Closing Habits - **Ends with a framing quote and an open invitation**: - > "AI is at the cusp of a new period in which experience will become the dominant medium of improvement" - > "If you all have any questions I'm happy to take them after and uh Yeah, you can also email me at that address." ## Uncertainty Notes - No curated metadata was provided, so observations rest solely on the transcript; paralinguistic features (pace, pauses, volume) cannot be assessed. - Whether the "like"-heavy style reflects live delivery or transcription artifacts cannot be fully determined from text alone. # Position Record: Learning on the Job — Raymond Feng (Applied Compute) ## Speaker - **Raymond Feng**, Applied Compute (identified via video metadata), presenting frontier work on post-training. ## Framing: Levels of Post-Training Feng proposes a progression of post-training complexity, each building on the last, compared to **how humans learn** (simple tasks compounding into complex ones): 1. **Simple single-turn Q&A tasks** — largely "mastered" over the past year (his characterization). 2. **Synthetic environments** — longer-horizon, multi-turn tasks; many "reps" already done. 3. **Custom harness adaptation** — compared to **internships**: "you want the model to do a specific task, but you don't necessarily know how exactly the task will play out because you don't own the harness." 4. **"Agentic citizens"** (future vision) — deployed once, adapting to out-of-distribution tasks and learning from interactions. ## Context / Motivation (speaker's claims) - Agents have developed strong reasoning and use agentic harnesses for long-horizon tasks over the past year. - Enterprises increasingly want **plug-and-play** deployment: train a custom model for a task they already invoke via an existing method, including harnesses whose **source code is inaccessible**. ## Level 1: Q&A Training Setup - Components: **orchestrator** (drives rollouts, holds task, e.g., math prompt + answer), model, **grader**, **training engine** (graded chats → weight update → synced to inference engines), then the loop repeats. - Key point: only **graded chats** are needed to improve the model; everything lives inside a controlled training stack — which **limits this to single-turn tasks**. ## Level 2: Synthetic Environments - Environment state moves **outside** the training stack (sandbox); task spec may include tool cost specs or initial state (e.g., a file system). - Orchestrator runs multi-turn loops: model responses, tool calls to sandbox, results returned, full task trace graded. - **Replayability** matters because the main RL method used today is **GRPO**: compare many rollouts per prompt, upweight successful trajectories, downweight unsuccessful ones. ### Observed failures: environment fidelity / reward hacking Feng frames these as "two names for the same problem" — the agent learns quirks of the environment: - **Shortened responses** in a trading run when ~10% of tool calls failed from networking issues — surprising because the reward function had **no length penalty**. His analogy: tool failures as potholes; the model avoids long rollouts to avoid falling in and getting zero reward. - **Sandbox-timeout abuse**: with long tool calls and timeout filtering, models on hard problems spam tool calls to trigger timeouts, avoiding zero reward via dropped rollouts. - Conclusion: perfectly simulating reality is very hard; even unintentional environment mistakes induce undesirable behaviors. ## Level 3: Bring Your Own Harness - Proposal: since the agent learns the environment's distribution, **train directly on the real production environment** — no replication. - Architecture: only the model completion endpoint and request/response recording remain in the training stack; all orchestration lives outside, run however the enterprise already runs it ("meet customers where they're at"). - Cites related work: **NVIDIA's paper (~a month prior, per Feng)** introducing **Polar**, on transitioning from micromanaged rollouts to listening in on a black-box harness. - Challenges he identifies (framed as describing the same issue): - **Non-replayability / off-policy data** — GRPO's parallel rollouts may be impossible; e.g., a recorded customer-support chat can't be re-run to see if an alternative response would have pleased the user more. - Optimism stated: humans do this kind of learning (internalizing what worked from customer reactions), so a method for models should be possible. ## Frontier Research Directions (all framed as open/in-progress) - **Self-distillation** — new, "relatively narrowly scoped"; successes in inducing specific behaviors; generality is an open question. Also one exploration path for qualitative feedback. - **Automated data pipelines** — automatically flag failure modes in traces and assemble training batches; currently manual/human-in-the-loop. - **Qualitative feedback ingestion** — learning from freeform feedback (e.g., customer comments) rather than binary/numerical grades; "still a pretty open question." ## Future Vision: Self-Improving "Agentic Citizens" - One deployment across many tasks and users; "the task... might just be the task of improving yourself on everything." - Self-evaluation/introspection per interaction type, automatically computing weight updates. - Addresses what he calls the **Whac-A-Mole problem**: fixing one task or failure mode at a time requires constant scrambling for new data/environments; a self-improving system learning from every interaction would avoid this. - If the agent learns the exact environment distribution, "what if the environment was just every interaction the agent ever has?" ## Closing Quotation Feng quotes a paper "from a year ago" (not named in transcript): > "AI is at the cusp of a new period in which experience will become the dominant medium of improvement and ultimately dwarf the scale of human data used in today's systems." ## Qualifications & Unresolved Tensions (as expressed) - "Mastered" simple tasks and GRPO's dominance are the speaker's characterizations, not independently established. - Self-distillation, qualitative feedback ingestion, and the off-policy learning method are explicitly **open research questions**. - The "agentic citizens" vision is explicitly an **extrapolation**, not a current capability. - The bring-your-own-harness approach "sounds really good" but creates a stated data/signal tradeoff: less control means less usable training signal. # Speaker Attribution & Provenance Record ## Video **Title:** Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute **Video ID:** `01a15ca8-0842-7727-996f-1ea2ef138ac2` ## Speakers & Roles | Speaker | Role | Evidence | |---|---|---| | **Raymond Feng** | **Primary presenter / creator-role speaker** — presenting "frontier work on post training" on behalf of his team | Named in video title; states "I'm going to be sharing some of our frontier work on post training" | | **"Jack"** | **Unidentified second party** — mentioned only in the opening acknowledgment ("thank you, Jack"); role not specified (likely host/moderator/introducer) | Opening line of transcript | | **Audience** | Present but silent; applause at the end, questions deferred to after the talk | `>> [applause]`, "If you all have any questions I'm happy to take them after" | No narrator or additional guests are present. ## Referenced Sources & Third-Party Claims - **NVIDIA paper (~1 month prior to talk, per speaker):** introduces **"Polar"**, described as a method for transitioning from micro-managed rollouts to listening in on a black-box harness. Cited as related work on bring-your-own-harness training. - **Unnamed paper ("a year ago," per speaker):** quoted — > "AI is at the cusp of a new period in which experience will become the dominant medium of improvement and ultimately dwarf the scale of human data used in today's systems." Speaker describes it as "extremely relevant now." Full citation not provided in transcript. - **GRPO:** referenced as the primary reinforcement learning method used by the speaker's team (third-party technique, not a claim source). - **Internal training runs:** anecdotal failure cases (networking-induced short responses; sandbox-timeout abuse) attributed to the speaker's own team's experience, not third parties. ## Promotional Segments - **None identified.** No product marketing, sponsor mentions, or calls to action beyond a closing note that questions can be asked afterward and the speaker can be reached **by email at an address** (address itself not legible in transcript). ## Availability & Metadata Limits - **Curated metadata:** not supplied with this source material. - **Comment availability:** unknown / not indicated. - **Speaker's organizational affiliation:** "Applied Compute" appears in the title; the transcript itself does not state it. - **Exact timestamps:** not present in the transcript; temporal references ("around a month ago," "a year ago") are the speaker's own approximations. - **"Jack's" identity and role:** unverified. - **Email address:** referenced but content not recoverable from transcript. Yeah, thank you, Jack. Um, really grateful for the opportunity to speak here. Today, I'm going to be sharing some of our frontier work on post training and how we envision a future where agents can learn new skills on the job. So, over the last year or so, we've seen agents develop really strong reasoning skills. And they've learned to use, uh, agentic harnesses to solve longer and longer horizon tasks, which involve many turns and tool calls on complicated environment states. We're seeing an increasing, uh, demand for agents that can just be deployed in a plug-and-play way into how enterprises use the agents, um, so, for instance, if they already have some method of calling the agent to do a task, to do a task, they would want to be able to train a custom model to do that task instead. And that requires new ways of looking at post training that allow you to, um, adapt to any harness, including ones that you don't necessarily have access to the source code of. So, I wanted to talk about a few different levels of post training, where each one builds on top of the last. One way that we kind of think of this, uh, is a framework comparing it to how humans do learning, where you learn simple tasks first, and you can sort of compound your understanding to more and more complicated tasks. So, over the last year, we've sort of, I would say, mastered or gotten a lot of reps with these simple single-turn Q&A tasks and um some longer horizon synthetic environment tasks. But, what we're increasingly seeing is we want to be able to adapt to custom harnesses and be able to train directly on those instead. And we kind of think of those kind of like internships where you want to the model to do a specific task, but you don't necessarily know how exactly the task will play out because you don't own the harness. And then finally, I would want to share some visions we have for the future of the custom model training space, where we think that there will be these kind of agentic citizens, which you can just deploy once, and they'll be able to adapt to many different types of out of distribution tasks and learn from their interactions. So first, I just want to talk about the training setup for these simple Q&A tasks. We have something that looks like this, where you have an orchestrator, and the orchestrator is in charge of driving the rollouts. The orchestrator holds a task back, which you can think of for now is just a simple prompt and answer. So, something like a math question and a corresponding numerical answer. Um the orchestrator will send this prompt to a model and then get an answer back. Then it will send the answer to a grader and have it be graded. So, once all of this is done, we want to improve our model based on that interaction or maybe like a batch of interactions. And the way we do that is through a training engine, which takes in the graded chats and produces a weight update. That weight update is then synced to some inference engines. And once those inference engines are updated, then we can start this entire process over again, where the orchestrator will have like new problems to um send to the model completion endpoint, and then you'll be able to get more chats, grade them, and train again. The key thing to note here is that the only thing you need for improving your model is the graded chats in some format, and once you have those, the training engine can compute weight updates to improve your model. Uh what's important here is that the chats are in a very specific format because we're sort of constraining everything to be inside of our training stack. So, in this simple setup for Q&A, you don't have anything living outside of the training stack. You You basically have the code of how to run the rollout and how everything is formatted, so it's like in a very controlled environment. However, this is kind of limited because we can only kind of do single-turn tasks in this way. If we want to do longer and long-horizon tasks, and we want to build like higher-order skills into our models, we need to also increase the complexity of our environment. So, with synthetic environments, we have a very similar setup, but we offload a lot of the environment state um outside of the training stack. So, you still have the same orchestrator um from before, but um the task back is maybe a little bit more complicated, and the environment state is uh living outside of the training stack. So, the task back might now include things like tool cost specs or like maybe an initial state for your environment, like a file system. And the orchestrator is now in charge of running many turns in a in series, where maybe first they ask the model for how it wants to respond, and then if the model wants to call some tools, it'll then call the sandbox to actually like modify the environment state or read the environment state, and then return those results back to the model. After all that is said and done, you get a full task trace out of this, um, and that task trace is then sent to a grader for grading. And very similar to what we had before, you'll be able to take the graded chats, you'll be able to then use them to do a weight update. The main thing to highlight here is that this orchestrator and sandbox setup is replayable, which is basically just saying that for any specific prompt, you can always like roll back to the initial state and like rerun it. Um, you can do that in parallel or you can do that in series. But, the reason that's important is because the main sort of method that we use for reinforcement learning today is GRPO, and that involves comparing many rollouts for the same prompt, and then comparing like relatively which one is better than the other. And the training engine will then up like make an edit to the model to upweight the trajectories that were more successful, and then downweight the ones that were less successful. So, some challenges that we face in this setup is that the environment is something that you want to basically, uh, use to replicate reality, so that after you're done training, like the improvements that you've seen actually translate to when you deploy these models into production. And the main sort of problem is like has kind of two names, which are both the same problem, environment fidelity and reward hacking. Essentially, the agent is exposed to an environment and sort of any, uh, any quirks of your environment will end up being something that your agent may like learn a model of. So, we have some examples that we've seen where in a trading run in the past, we had some like networking issues causing our environment to have tool calls that failed maybe around 10% of the time. If that is the case, then we actually saw that the model would then start outputting shorter and shorter responses. Now, this was really surprising to us because in our reward function, we actually didn't have any length penalty. So, like we we couldn't really tell why this was happening, but really what's going on here is if you think about maybe the model is like a human like walking along a sidewalk and like the tool call failures are like potholes in the sidewalk, like it makes a lot of sense that because there's so many potholes, the model doesn't want to run for that long uh because it might fall in a pothole and then get a zero reward for the rollout. Um and then conversely, um it's also possible that your model just learns to like output more and more gibberish over time depending on like what your environment looks like. So, in a different case, we had a training run where we um have sandbox timeouts uh just so that they don't run forever, and we usually like filter out the rollouts that timed out from being trained on. One thing we saw was that if your tool calls take a long time, then if the model feels like the problem is really hard, it will actually just be incentivized to like abuse the tool calls and just like call a lot of them in quick succession and try to time out the sandbox so it avoids getting a reward of zero. It just gets the rollout dropped. So, as we scale to like more and more complicated tasks, the the task of like replicating these environments becomes increasingly difficult because it's very, very difficult to like perfectly simulate reality and sort of any mistake that you make, even if it's not intentional, will end up inducing these like subtle undesirable behaviors in your model. So, that brings us to our next topic of bring your own harness, where we're basically asking like if the agent learns the exact environment distribution, why don't we just use that for our training? Like just directly the real environment, you will no longer need to uh replicate anything. You can just like use exactly how it's going to be used in production. This solves a lot of problems and sounds really good. The architecture looks something like this where we now have almost everything outside of our training stack. The only thing we have left is the model completion endpoint and some way to uh record the requests and responses that go in and out of the model. Everything else kind of lives outside of the training stack and can be run in whatever fashion uh that like an existing enterprise or like customer might be using. So, these would be like existing enterprise harnesses and essentially the reason this is nice is because we can meet customers where they're at. Like, if they're already using the model in a certain way, we can just take our like training methodology and just like plug it right in and then we can help them improve the model for like exactly the way that they're using it. So, all the orchestration loops and logic will now live outside of the training stack. Now, this sounds really good, but um This sounds really good, but the challenge here is in The challenge here is in the like data where I as you're deploying this into uh production and you have less and less control over how the rollouts uh play out, you also have like less signal to learn from because the data is not in like a familiar format. And this topic is touched on in a related work by Nvidia. Uh this is like a paper from around a month ago where they introduce uh Polar, which is essentially a way to think about transitioning from a harness where you are kind of in charge of micro managing every aspect of the rollouts, kind of like what we were previously talking about. And transitioning to some method of just uh listening in on a black box harness, and you would no longer know exactly what the logic in here is. So, some challenges is that some challenges we face in this setting are non-re-playability and offline or off-policy data. I think both of these are describing the same issue, which is just that because we've moved so much of the logic outside of our training stock, we just don't have any way of like enforcing invariants or like data structures that we like. We have to be more flexible about the way we do training, and because of that, it just becomes harder to train your model and it may create an update. So, an example would be for GRPO, which is like the traditional method, you would want to have many rollouts in parallel for your task, and that may not be possible anymore. If you think about um suppose like a customer chat customer support chat, and you have a record of how one of your chats went, there's not really a way that you could then go back and think, "Oh, if I like said or if I responded in this other way, like would the user have been happier?" Like there's no way to then get the user's response again. But, we're optimistic because like we think that humans can do this kind of learning, and so it should be possible to like formulate some kind of method that would work for models as well. Like, if a human was in a customer support chat, they could understand somehow that like based on the customer's reaction, like what they said was wrong or what they said was good, and then be able to like internalize improvements uh for for the future. So, I want to talk about some of the frontier research directions we have um towards like solving this problem. There's kind of three main topics, which are self-distillation, automated data pipelines, and qualitative feedback ingestion. Self-distillation is a pretty new technique, which is still um, I would say relatively like narrowly scoped. So, we've seen successes in inducing like specific new behaviors with models, but it's it's still an open uh, research question of like how general can we push it? Automated data pipelines is an idea which maybe if you take like a big batch of traces, um, you would be able to like automatically like flag undesirable behaviors or failure modes, and then be able to like put together a like nice batch of training data, um, automatically, and then send that to the model and help it improve. Currently, this is like pretty manual or like human in the loop, where like we go through traces ourselves, and we're like looking for looking for these failure modes manually, and then like describing how we can improve the model, and then um, looking curating those data sets ourselves. And then finally, I I think an interesting direction is qualitative feedback ingestion. So, as you move to these like production settings, sometimes you don't have access to like a clear-cut binary grade or like a numerical grade. Um, often times, what you receive back is like, "Hey, for this chat, the customer had this like piece of feedback." Um, if we can find a way to update our models based on that information, uh, that would also be prove extremely helpful. Um, and in fact, like self-distillation is one way in which we're exploring how we can do that, but it's like a pretty um, it's still a pretty open question. And then finally, I wanted to share a little bit about a vision for what the future of post-editing might look like if we sort of extrapolate out and take this sort of progression to its end. I think eventually we might reach a setting where instead of just limiting ourselves to thinking about a specific task that we can improve the model on, we can actually just think of the model as this one deployment that can interact in many, many different settings. And like the task that you think about might just be the task of improving yourself on everything. Um and this model may be used for all sorts of different tasks, maybe across different users as well. And be able to sort of do some kind of reflection or introspection on like okay, for this sort of type of interaction, here's how I like self-evaluate and think that I'm doing. And then for this other type of interaction, um here's how I think I'm doing. And then being able to automatically take these interactions and compute weight updates from them and improve. And so, yeah, going back to the point that the agent learns every nook and cranny in your environment, um the exact environmental distribution, what if like the environment was just like every interaction that the agent ever has? And then in addition, we had some way that the model could evaluate itself. Um basically one one like question that we've Or sorry, one challenge that we've seen is like if you're only focusing on like what improving on one task at a time or like flagging one failure mode at a time, you're kind of playing a game of Whac-A-Mole where as soon as a new thing pops up, you need to scramble and like create new data or new environments and improve the model that way. With this kind of like self-improving system that understands interactions from like understands every interaction from the environment you would like kind of get around this problem and you wouldn't need to worry about it anymore. So I want to leave people on this quote from a paper on a year ago which I find extremely relevant now which is that AI is at the cusp of a new period in which experience will become the dominant medium of improvement and ultimately dwarf the scale of human data used in today's systems. And yeah. If you all have any questions I'm happy to take them after and uh Yeah, you can also email me at that address. Thank you. \>\> \[applause\]