Talk by Varun Singh, who leads pre-training at Arcee AI, arguing that the traditional view of a base model as a mirror of web text is outdated. He explains how instruction data and synthetic reasoning traces have moved earlier into training, how a mid-training stage has emerged for longer, downstream-oriented data, and how recent open recipes (Nemotron, Kimi K2) reflect raw web text taking a smaller role. He discusses how RL-driven reasoning raises the bar for the base model, which must now carry priors RL builds on, and shares lessons from training Arcee's Trinity series, including tuning balancing coefficients and establishing stable representations early. # Channel Identity Observations — "The Base Model Is Dead — Varun Singh, Arcee AI" ## Explicit Self-Description Evidence - **Speaker role stated directly**: The speaker introduces himself as **"the pre-training lead at RCAI [Arcee AI]"** and delivers a named technical talk, "the base model is dead... but not really." - **Affiliation claimed**: The speaker references "our own model RC [Arcee] training" and "our own model Trinity Lodge," explicitly associating the channel content with **Arcee AI's internal pre-training work**. - No broader channel mission statement or channel-level self-description appears in the transcript itself. ## Observable Presentation Patterns ### Intended Audience - The content assumes **deep familiarity with LLM research**: terms like *MOE load balancing*, *next token prediction*, *SFT*, *RL*, *mid-training*, and *agentic traces* are used without basic explanation. - The talk cites research papers and models by name — **GPT-3, Llama 3, GLM 4.5/5, DeepSeek R1, OpenAI o1, MAI Thinking 1, NeMoTron 3 Ultra, Kimi K2, Composer 2.5, AlphaGo** — indicating an audience of **ML researchers and practitioners** rather than general viewers. - Slide references ("these screenshots," "the slide," "the pie chart") and closing **applause** indicate this is a **recorded conference talk**, not a studio-produced explainer. ### Claimed / Demonstrated Scope - **Scope demonstrated**: The evolution of LLM pre-training — specifically the **declining role of web text** in training mixes and the shift from "general human knowledge priors" toward "reasoning and agentic behavior priors." - **Claimed insider perspective**: The speaker presents Arcee AI's own training experience ("in our own model Trinity Lodge, we had a large amount of web-scale synthetic data") as evidence, positioning the channel as a venue for **first-party lab research perspectives**. ### Stated Personal Positions (Claims, Not Demonstrations) - **Opinion on synthetic data**: The speaker states, "I've got my opinions on like synthetic data being the way forward" — an explicitly flagged personal view, contrasted against MAI Thinking 1's no-synthetic-data stance. - **Proposed reframing**: The speaker argues a "better way to understand the current phase of LM training" is as two paradigms — supervised next-token prediction and RL — with RL growing in importance; this is presented as his interpretive claim, not established fact. ## Notes on Distinguishing Claims from Evidence - Speaker's factual claims about training mixes (e.g., "web text... now all the way down at 15%" in MAI Thinking 1; "WebText... roughly 85%" for GPT-3) are attributed to cited papers — these are **reported citations**, not independently verifiable within this transcript. - No private motivations beyond stated opinions were inferred; the speaker explicitly labels which views are his own. # Delivery Observations — "The Base Model Is Dead — Varun Singh, Arcee AI" ## Tone - **Casual, conversational academic style.** The talk reads like an informal research talk rather than a rehearsed keynote: the speaker opens with a hedge on his own title — *"the base model is dead. Um but not really."* - **Frequently hedged and qualified claims.** He explicitly signals uncertainty when the evidence is unclear: *"it seems like not super clear yet. Um I have my opinions on like synthetic data being the way forward, but I'm I've got like two contrasting uh perspectives here."* - **Self-aware caveats.** He flags his own simplification: *"of course, that's kind of, uh, reductive in the in a way."* ## Filler and Disfluency Patterns - **Very heavy use of "um" and "uh"** throughout — appears in nearly every transcript segment, e.g., *"Um the way that uh older base models were trained was, like I said, mostly on uh things."* - **"like" as a verbal connective**, used constantly: *"built on uh this idea of like training on super large-scale web text"*; *"which is like a commonly available web scrape."* - **Repeated words and self-restarts**, e.g., *"I've I've I've just got this slide here"*, *"the the famous graphs from 01"*, *"be be useful in an interaction."* ## Recurring Phrases and Vocabulary - **"from the very beginning"** recurs as a framing phrase for the talk's central argument (appears at least four times): - *"um from the very beginning of the pre-training process"* - *"all the way back in pre-training and uh allowing the model to like learn those task representations from the very beginning"* - *"the model should learn stable representations from really early on"* - **"kind of"** is a stable verbal tic: *"kind of is like built on"*, *"kind of a reflection"*, *"RL was mostly just a cherry on top, um shaping the, you know, flavor."* - **Domain vocabulary reused consistently**: *pre-training, post-training, mid-training, base model, synthetic data, web text, RL, SFT, MOE, load balancing, next token prediction, agentic*. ## Teaching Habits - **Slide-anchored narration.** He repeatedly references visuals as evidence rather than describing them fully: *"I've taken the top one from uh GLM 4.5, the bottom one from GLM 5"*; *"these screenshots were taken from, uh, Zephyr's Ziya 1 paper."* - **Analogies to convey technical shifts.** Notably: *"RL was mostly just a cherry on top"* — later explicitly inverted: *"no longer a cherry on top, but it can dramatically improve the performance."* - **Concrete numbers to anchor claims.** He quotes specific data-mix percentages: *"up to 85% of the train data in GPT uh 3, is now all the way down at 15%"*; *"50% of the tokens corresponding to general knowledge."* - **Contrastive framing.** He structures points as opposing pairs — two papers with opposite data philosophies (*"MAI Thinking 1... really try to, you know, filter"* vs. *"NeMo Tron series leans so heavily into synthetic data"*) and two paradigms (*"supervised learning to next token prediction and RL"*). - **Named-source citations as habit.** GPT-3, Llama 3, DeepSeek R1, GLM 4.5/5, Kimi K2, NeMoTron 3 Ultra, AlphaGo, Xiaomi MiMo Labs, Composer 2.5 — grounding nearly every claim in an external paper or lab. - **Classic-reference appeal.** *"I had to put in the classic AlphaGo graph there"* — signaling shared cultural touchstones with a technical audience. ## Communication Patterns - **Cumulative narrative structure.** The talk moves chronologically ("Back then… / Now… / Um, however, uh, this kind of changed a lot last year"), building a then-vs-now argument. - **Rhetorical questions to pivot.** *"Um, is your standard base model still uh, what the best uh, what will be the best um, prior"*; *"but what is actually uh like how is synthetic data used?"* - **Brief, low-key closing.** *"And yeah. Thanks for Thanks for listening. Thanks for your time."* — consistent with the informal register maintained throughout. ## Stability vs. Isolated Moments - **Stable patterns** (high repetition): filler words ("um/uh"), "like" and "kind of" as connectives, source-citing habit, contrastive framing, "from the very beginning" phrasing. - **Isolated moments** (single occurrences): the "cherry on top" analogy, the "it gets a bit muddy" phrasing (*"It all gets a bit muddy that way"*), and the AlphaGo aside — memorable but not repeated, so their status as habits is **uncertain**. # Observation Document: "The Base Model Is Dead" — Varun Singh, Arcee AI ## Speaker Identification - **Varun (Varon) Singh**, self-identified as **pre-training lead at Arcee AI (RCAI)**, is the sole speaker. ## Core Thesis - The talk title claims "the base model is dead," but the speaker immediately qualifies: **"but not really."** - The classical base model — trained as a broad reflection of "the whole knowledge of the human internet" — is being replaced by models built around **reasoning and agentic behavior priors**. ## Historical Framing: The Old Base Model - Pre-training was the stage where models accumulated world knowledge via **next-token prediction on web text**. - Evidence cited: - **GPT-3 paper**: Common Crawl, WebText-2, books, Wikipedia; web text (incl. Wikipedia) made up ~85% of the training mix. - **Llama 3**: web text still a majority at ~50% general-knowledge tokens. - **GPT-3 paper** also cited for in-context learning emerging from next-token prediction. - Training diagrams from **GLM 4.5** and **GLM 5**, and Arcee's own model, shown as examples. - Post-training was historically a thin layer: adapting to chat templates and question-answer formats. **"RL was mostly just a cherry on top."** ## The Shift: RL Becomes Central - Catalysts cited: - **OpenAI o1 (2024)** — pioneering reasoning models; famous AIME performance graphs. - **DeepSeek R1 (January 2025)** — showed the world how to build such models. - **Claude Code** — enabled terminal-based agentic use; models learned to interact with software environments via end-to-end RL. - Central question posed: **Is the standard base model still the best prior for large-scale RL?** The speaker states the answer is "not super clear yet." ## Two Contrasting Data Paradigms ### 1. MAI Thinking 1 — No Synthetic Data - Deliberately excluded synthetic data and other models' outputs, filtering web scrapes to do so. - Yet its data mix is still radically different from classical recipes: **web text dropped from up to 85% (GPT-3) to 15%**, displaced by code and STEM. ### 2. NeMo Tron 3 Ultra — Synthetic/SFT Data Pulled into Pre-training - SFT-style question-answer datasets (labeled with "SFT" prefixes in the recipe) introduced during pre-training. - Purpose: let the model learn the **shape of conversations and downstream tasks from the very beginning**. ## Synthetic Data Usage - Speaker acknowledges the concern that blind synthetic data can cause **model collapse** and performance degradation, but notes large-scale successes: - **Arcee's Trinity Lodge**: web-scale synthetic data via **rephrasing** — upsampling a seed item so the model sees the same information multiple ways. - **Kimi K2**: applied this broadly across the pre-training dataset. - **Swallow Code / Swallow Math** papers as early examples. - Claimed benefits: more tokens, cleaner/higher-quality tokens, tokens shaped like instruct/agentic tasks from the start. - **Speaker's stated opinion**: "I have my opinions on like synthetic data being the way forward" (explicitly flagged as his view). ## MOE Load Balancing Argument - **Problem**: post-training data distribution differs drastically from pre-training distribution, causing expert imbalances in mixture-of-experts training (illustrated, per the speaker, by the **MAI Thinking 1 report**). - MAI's fix: cranking up the load-balancing coefficient during SFT. - Speaker's position: ideally the model should **learn stable representations early** rather than disturbing balance late in training — an argument for pulling post-training data into pre-training. ## Mid-Training - Defined as exposing the model to post-training/RL distributions at **longer context**, including agentic traces. - Speaker notes many models now train with long context in pre-training, so "there's no reason" these datasets can't be pulled earlier. ## Reframing: Two Paradigms, Not Three Phases - The speaker argues the pre/mid/post-training framing "gets a bit muddy"; better viewed as **supervised learning (next-token prediction)** and **RL**, with RL growing in importance. - Compute evidence: - **Xiaomi MiMo Labs** interview: roughly equal compute allocation between pre-training and post-training. - **Composer 2.5** (attributed to "Kosar"): more RL compute than supervised learning — the extreme case. - Implication: supervised learning should be viewed as **preparation for RL** — building useful representations — rather than the bulk of the model's capability. ## Supervised Learning → RL Interaction - Cites a paper whose takeaways: the base model needs **exposure to atomic skills** it must compose during RL; with a sufficiently difficult environment, the model can extrapolate during RL. - Cites the **AlphaGo graph** where RL eventually overtakes supervised learning — but qualifies that it's **unclear** this will replicate for LLMs because human language is "such an insane distribution" to learn via RL alone. ## Novel Data in Supervised Learning - **Novel data**: shapes the model wouldn't naturally see, e.g., **reasoning traces** (unlike typical human output). - **Test-time compute schemes** warmed up during SFT or pre-training. - Sources: screenshots from **Zephyr's Ziya 1 paper** (speaker's attribution), framed as interesting ways data can shape RL exploration skills. ## Conclusion (as stated) - Base models have shifted from **general human knowledge / world priors** to **reasoning and agentic behavior priors**. - Speaker concedes this is reductive — reasoning/agents are simply the dominant current use pattern. - Conditional claim: if a new interaction paradigm emerges, the base model should be understood as **building a prior for that paradigm**, not as a massive corpus of web text. ## Preserved Qualifications and Tensions - Title thesis ("dead") immediately softened ("but not really"). - "It seems like not super clear yet" where data trends are heading. - Synthetic-data position is the speaker's **opinion**, contrasted with MAI Thinking 1's no-synthetic approach. - AlphaGo-style RL overtaking SL is **uncertain** for language models. - Final conclusion explicitly labeled **reductive**. # Video Attribution & Provenance Record **Video ID:** `01a15ca8-0a2d-7feb-bfcc-1b76f479e989` **Title:** The Base Model Is Dead — Varun Singh, Arcee AI ## Speakers & Roles - **Varun (Varon) Singh** — **Creator / sole speaker**. Self-identified: *"My name is Varon. I'm the pre-training lead at RCAI"* (Arcee AI). Delivers the full talk; no other speakers present. - **Audience** — implicit, non-speaking; **[applause]** at the end. No guests, narrators, or other identified roles. ## Roles Attributed to Third Parties (named in transcript, not speaking) - **Head of Xiaomi's Mimo Labs** — quoted via interview screenshot regarding compute allocation between pre-training and post-training. - **Kosar (Composer 2.5)** — cited for RL-heavy training approach. ## Referenced Sources ### Papers & Models Cited - **GPT-3 paper** — in-context learning; training data mix (Common Crawl, WebText-2, books, Wikipedia) - **GLM 4.5** and **GLM 5** — training pipeline diagrams - **Arcee's own model ("RC" / "training large thinking"; "Trinity Lodge")** — creator's own work, web-scale synthetic rephrasing data - **Llama 3** — data mix percentages - **OpenAI o1 (2024)** — reasoning models, AIME performance graphs - **DeepSeek R1 (January 2025)** — open reasoning-model recipe - **Claude Code ("Cloud Code" in transcript)** — developer terminal tool - **MAI Thinking 1** — no-synthetic-data approach; load balancing findings; 15% web text - **NemoTron 3 Ultra** — SFT-prefixed data in pre-training - **Kimi K2** — large-scale synthetic data screenshots - **Swallow Code / Swallow Math** datasets — early synthetic data work - **Zephyr's Ziya 1 paper** — novel data / test-time compute screenshots - **AlphaGo** — RL vs. supervised learning graph - **Xiaomi Mimo Labs interview** — compute allocation screenshot ### Third-Party Claims (uncorroborated in transcript) - o1 AIME performance gains from RL - MAI Thinking 1's 15% web text and load balancing coefficient claims - Equal pre-/post-training compute at Xiaomi Mimo Labs - NemoTron 3 Ultra's SFT-in-pre-training data recipe ## Promotional Segments - **Promotion of creator's own work:** Arcee's models ("RC," "Trinity Lodge") and its synthetic-data approach are highlighted: *"in our own model Trinity Lodge, we had a large amount of web scale synthetic data."* Flag as **self-promotional context**, presented as technical example. - No other sponsor, product, or call-to-action segments present. ## Comment Availability - **Not specified** — no metadata supplied; transcript contains no information on comments. ## Uncertainty Notes - Speaker name rendered inconsistently in transcript ("Varon" vs. title's "Varun Singh"); affiliation transcribed as "RCAI" vs. title's "Arcee AI" — treated as the same entity. - Speaker explicitly states personal opinion: *"I have my opinions on like synthetic data being the way forward."* - Claims about whether RL will overtake supervised learning are marked by speaker as speculative: *"it's unclear if we'll see something like [that] for language models."* Hi everyone. My name is Varon. I'm the pre-training lead at RCAI. And the talk I'm going to be giving today is called the base model is dead. Um but not really. Um the idea of the base model that we have um kind of is like built on uh this idea of like training on super large-scale web text and the base model kind of being a reflection of like the whole knowledge of like the uh human internet. Um you can see in like uh these I've I've taken these uh from a bunch of uh different papers on like the entire LM training process. Um our own model RC uh training large thinking, um the process looked kind of like the simplified diagram on the left. I've taken the top one from uh GLM 4.5, the bottom one from GLM 5. Um all these have a pre-training phase and uh pre-training is like the stage where the model look accumulates world knowledge, builds useful representations uh all through next token prediction on um web text. Um I've got a simplified uh transformer diagram um decoder-only transformer. And um a screenshot from the GPT-3 paper that talks about how um language models can uh learn how to do in-context learning through um unsupervised or self-supervised or um some some even just call it supervised um learning on uh through next token prediction. Um the way that uh older base models were trained was, like I said, mostly on uh things that reflected the entirety of human knowledge. Uh so, Common Crawl, uh which is like a commonly available web scrape, uh made up most of the training data set for GPT-3. Um WebText-2, another web scrape data set. Um some sources from like uh books as well. Um and Wikipedia as like a high-quality representation of human knowledge. Um you can see that uh WebText alone here, including like Wikipedia, makes up like roughly 85% of the whole training mix. Um looking at uh the bottom with Llama 3, um WebText still kind of makes up a majority of of the um model's training data with like 50% of the tokens corresponding to general knowledge. Back then, post-training was mostly shaping the model to use um the parts it to like surface uh the knowledge that it accumulates and accumulated in pre-training uh in like a chat interface. So, mostly allowing the model to adapt to a chat template, to the question-answer format, um and be be useful in an interaction that way. Um RL was mostly just a cherry on top, um shaping the, you know, flavor of the interactions more than conferring extra um knowledge or quality onto the base model itself. Now, in in like this realm of of how language models used to be, pre-training um and the base model kind of defined how good you were able to get a model. Um it was like the bulk of the compute uh budget and it was um, the the like core of the training process. Um, however, uh, this kind of changed a lot last year when um, OpenAI I guess 2024 actually. OpenAI released 01, um, pioneering reasoning models and DeepSeek uh, also released R1 in January 2025, um, allowing the whole world to know how to build these types of language models. And now we have this new uh, new um, use for reinforcement learning, which is no longer a cherry on top, but it can dramatically improve the performance of of the model on various different tasks. Um, the the famous graphs from 01 there talking about uh, AIME performance, um, competitive math contest. Um, and then even later in the year we we saw Cloud Code um, coming to being as a way for developers to easily um, kind of use uh, language models in a in a terminal to build out applications as models got stronger and stronger on things like function calling. Um, and then people realized you could uh, you know, RL this end-to-end. Um, and now models could learn how to interact with software environments and build software and uh, perform really useful work. And so, um, the question then becomes like, is your standard base model still uh, what the best uh, what will be the best um, prior for the for the this large-scale reinforcement learning phase that uh, research knows and um agentic models now use. And we can kind of see like in the in a few open research papers what the trend is where the trend is going. Um and interestingly enough, the um it seems like not super clear yet. Um I have my opinions on like synthetic data being the way forward, but I'm I've got like two contrasting uh perspectives here kind of in the slide. The top image is from the MEI uh Thinking 1 paper where they make a make it really large point to not use any synthetic data or any uh data from any other language model. Um and they really try to, you know, filter their web scripts for this as well. Um in order uh to kind of adhere to like the previous um paradigm of like uh using human knowledge as a way to bootstrap uh model representations and like uh capabilities. Um But I would say that this is also, like even even though they stuck with no synthetic data, the data mix that they've chosen here is still um totally different from what you'd expect in like a um in a in a classical uh language model. And I mean, the the main reason for that is that web text, which used to make up like up to 85% of the train data in GPT uh 3, is now all the way down at 15%. And that I mean, that just shows that uh the value of like web text contributing to like the downstream um performance of like the models on RL and stuff is kind of uh it it's still important, but taking a backseat to things like code and stem abilities as the models kind of gain more real-world use cases related to to those. Um the other approach uh is to bring um post-training data and large-scale synthetic data back uh through pull it back through the process into the pre-training phase. Um the the bottom chart I've taken from NeMo Tron 3 Ultra. Um what they reveal there um the data recipe and not sure how readable it is, but these top three um on the left uh pie chart, the top three on the kind of right right side of it, uh they're all labeled SFT with with SFT as a prefix. And that's the type of question and answer kind of chat data set that you'd you'd expect to see only in post-training, but by pulling it back into the process, they're able to like get the model to learn um the shape of these conversations and what kind of tasks they might be expected to do downstream um from the very beginning of the pre-training process. Um and uh this follows like uh a similar um trend in like diminishing uh the amount of web text used in the model. Um Yeah. Um it's really interesting to see the the NeMo Tron series leans so heavily into synthetic data, but uh MAI Thinking 1 kind of lean in the opposite direction. Um I've I've I've just got this slide here as like a con uh easy contrast that people can see on the like amount of web text and the amount of books and stuff um being less of a percentage here. And GPT-3 didn't even used to have any specific code data sets, but now code is like the dominating um data data subset that we have in uh pre-training recipes. Um so I mentioned synthetic data, but what is actually uh like how is synthetic data used? There's a lot of uh talk around synthetic data that, you know, blindly tossing it into a model can cause the model to collapse and uh and performance to tank, but there's been a lot of work and uh even at like a large scale, you know, example of this uh turning out really well. Um so in our own in our in our own uh model Trinity Lodge, we uh had a large amount of uh web uh web scale synthetic data um mostly through rephrasing, where you take a seed data item and you sort of upsample it in the mix by uh generating synthetic rephrases of the same information. So, the model sees the same information in like multiple ways. Um the bottom two uh screenshots are from Kimikay2, um an even larger scale model that uh broadly used this um across the whole pre-training data set. Um the top uh top right um screenshot is from a paper that uh resulted in the data sets Swallow Code and Swallow Math, which are early examples of this. But, the trend seems to be that um synthetic data not only allows you to get more and more tokens, uh but also, you know, clean up tokens, get higher quality tokens, and have um tokens that are shaped more like instruct or agentic tasks all the way back in pre-training and uh allowing the model to like learn those task representations from the very beginning. Um another reason that it's uh beneficial to add post-training data early in pre-training is now with MOEs, um one of the biggest uh pain points in training an MOE is dealing with load balancing. Um, where experts can specialize over the course of training and um they uh and load balancing objectives aim to uh achieve broadly equal utilization of the experts um in a given batch or sequence depending on the objective. Um without uh post-training data in uh early in pre-training uh with an MOE one really easy pitfall uh that we can fall into is this is kind of illustrated in the MAI thinking one report, which is that the data distribution that the model sees uh in post-training is really really different uh compared to what it sees in pre-training. And this can cause massive imbalances and um MAI overcame it by uh really cranking up the load balancing coefficient during the SFT stages. Um but I mean, ideally you don't want to mess with the balance that far into training and the model should learn stable representations from really early on. Another uh interesting thing that um is changing in base models now is that you is this whole advent of mid-training, uh which is exposing the model to the distribution that it would see during post-training in RL and at a longer context, so for things like agentic traces to be allowed into the mix and uh to kind of help prepare the model that way. Um a lot of models though are training with much longer context in pre-training and there's no reason that these data sets can't be pulled back into the mix to allow for more stable representations from the very beginning. Um I think uh a better uh better way to understand the current uh phase of LM training uh isn't so much like pre-training, mid-training, post-training, RL. It all gets a bit muddy that way, but there's two broad paradigms that are like that really help build a LM today, and that's supervised learning to next token prediction and RL. Um and RL is becoming more and more important. Um The bottom thing is a screenshot from interview um with uh the head of Xiaomi's Mimo Labs, uh where she talks about how they allocate compute between research, pre-training, and post-training. And pre-training and post-training in the final model have a roughly equal compute allocation. Um Composer 2.5 takes us to the extreme where um Kosar really uh sank much, much more RL compute into the model than the model had ever seen in supervised learning. But with RL dominating such a massive amount of the compute budget, uh it makes sense to view supervised learning as a way specifically to prepare the model for to build useful representations for for RL instead of it being the bulk of like um what the model would be used for like previously. Um there's been some some work on how supervised learning affects RL. Um I really like this one paper where the main takeaways are basically that uh the base model needs to have some exposure to like uh like the atomic skills that it would need to compose during RL, and um the model can learn to extrapolate from there during RL given like the environment has a sufficient level of difficulty. Um I had to put in the classic AlphaGo graph there where RL eventually overtakes supervised learning. Uh it's unclear if we'll see something like for language models because of course, you know, uh, human language is such an insane distribution to have to like learn through reinforcement learning alone. Um, but it's definitely possible that we might see diminished supervised learning in more and more RL, uh, which makes this kind of thinking of a base model as, um, atomic skills for RL more and more valuable. Another thing that some labs are doing, um, is kind of introducing novel data, uh, during supervised learning. And by novel, I mean something that the model really wouldn't have seen the shape of before. Uh, easy example is, you know, reasoning traces. They don't really look like a like a ton of what, um, humans output. And, um, another interesting thing is like training for test time compute, um, schemes, um, by kind of warming the model up to them during SFT, um, or even pre-training itself. Um, this These screenshots were taken from, uh, Zephyr's Ziya 1 paper. Um, and I think if they're, uh, very interesting ways of thinking about how data can affect, um, like, uh, the skills needed to explore well in RL. Um, in conclusion, um, base models have kind of moved from general, uh, human knowledge and world priors to reasoning and agentic behavior priors. Um, of course, that's kind of, uh, reductive in the in the in a way that reasoning reasoners and agents are like the main way we see, uh, bots, uh, the main way we see these chatbots used now. But, if a new paradigm were to take off, like a new way of interacting with the models, um, it makes sense to like think of a base model as building a prior for that instead of, um, just building off like a a massive, uh, script of web text. And yeah. Thanks for Thanks for listening. Thanks for your time. And \>\> \[applause\]