Diogo Almeida, a GPT-4 co-author formerly at OpenAI and now CEO of TypeSafeAI (creators of Jev), argues that RLHF optimizes models for pleasing the human in the loop — driving engagement and overpromising rather than truthfulness. He distinguishes two regimes where this matters: models as assistants with a human catching mistakes (where RLHF shines) versus autonomous operation with real stakes (where the same people-pleasing instinct becomes a liability). Drawing on Sutton's bitter lesson, he frames what comes next as reinforcement learning with verifiable rewards (RLVR) — optimizing for real automation instead of human approval — while cautioning that pre-trained models are already highly capable and the trap is bolting preference optimization on top. Timestamps cover the state of the field, how RLHF works, preference vs. truth, why assistance isn't automation, pre-training, and RLVR. # Channel Identity Observations ## Speaker / Channel Self-Description - The speaker identifies himself as **Tiago Almeida**, presenting a talk titled around *"what's next after RLHF"* / *"what's next after the ChatGPT era."* - > "So, my name's Tiago Almeida. I'm talking about what's next after RLHF." - He claims authority via co-authorship on major OpenAI releases: - > "Co-authored to GPT-4, chat GPT, RLHF / instruct GPT. Um the team I was part of basically invented post-training as a concept." - He positions himself as a critical insider: *"I'm one of the few people at OpenAI who actually hates on chat GPT"* — then qualifies this as criticism of its limitations, not the product. **Note:** The video title ("Jev CEO: I made ChatGPT, now I'm building what's next") is a curator/upload framing; the transcript itself gives the speaker's name as Tiago Almeida and the CEO-type claim is consistent with him describing his company TypeSafe. Treat "Jev" as channel-level branding not corroborated in the transcript. ## Claimed Scope & Venture - He describes founding or leading a startup, **TypeSafe**, currently in stealth: - > "That is what we are working on at TypeSafe. We are still kind of stealthy." - > "Our core question is what if the AI stack was redesigned for reliability and automation?" - He claims a novel third post-training approach distinct from RLHF and RLVR: - > "we are doing a third thing that is optimized for calibrated decision-making." - He teases future content: > "I will post something later today that I guarantee will be very spicy. The hint is that the original scaling laws were incorrect." ## Intended Audience - A **technically expert audience** familiar with ML fundamentals — he polls the room: - > "can I have like a vague show of hands of who knows what RLHF is? Oh, excellent. I might be able to skip through that part." - An audience "adjacent to AI" professionally: > "anyone who works adjacent to AI should have an answer to this." - He invites interactivity and disagreement throughout ("yell out," "I love questions, I love the interactivity"), consistent with a **conference/meetup talk format**. ## Demonstrated Presentation Patterns (observable, not claimed) - **Analytical framing**: maps the AI discourse into two "cults" ("insanely well" vs. "insanely poorly") and proposes a unifying explanation (assistance vs. automation divide). - **Structured argumentation**: explicit "lessons" (Lesson one: LLMs excel at human-in-the-loop; Lesson two: AI designed for assistance via preference optimization; Lesson three: tomorrow's AI will be for automation). - **Promotional integration**: the talk pivots to a product/company pitch with calls to action — sign up for the mailing list or careers page, follow him on Twitter. - **Epistemic positioning**: distinguishes settled claims ("this is not like a controversial take") from speculation ("I believe that eventually the field will write..."), though this self-labeling is itself a creator claim. ## Audience Engagement Evidence - Live audience interaction: a questioner prompts discussion of classifier heads and Bengio's suggestions; another asks whether "the reward is injected through the whole process" — indicating a technically sophisticated live crowd. - Speaker explicitly runs overtime taking questions, reinforcing an interactive talk format. ## Claimed vs. Demonstrated Summary | Type | Evidence | |---|---| | **Claimed credentials** | GPT-4 / ChatGPT / InstructGPT co-author; team "invented post-training" | | **Claimed scope** | TypeSafe: automation-native AI stack, "third" post-training method, upcoming launch | | **Demonstrated** | Technical talk to expert audience; analytical framing; startup promotion; audience Q&A | | **Not established in material** | Any specifics of TypeSafe's product; "Jev" branding; private motivations | # Delivery Observations: "Jev CEO: I made ChatGPT, now I'm building what's next" ## Overall Delivery Character - **Casual, conversational conference-talk style** with frequent filler words ("um," "uh," "like") throughout, e.g., *"Um, I will say that um I might speed run through this."* - **Fast-paced, self-aware pacing**: he announces *"I might speed run through this"* and later notes *"I'm over halfway done with the talk and I haven't even answered it."* - **Enthusiasm expressed through casual exclamations**: repeated use of *"Hell yeah," "Heck, yeah," "Cool,"* — e.g., *"I can repeat the question. You don't have to worry about the mic. Hell yeah."* ## Strong Preference for Interactivity This is the most repeated delivery theme: - Opens by inviting interruptions: *"Feel free if you don't disag- agree with something to yell out. It's way more fun for me if things get interactive."* - Uses show-of-hands polling early: *"can I have like a vague show of hands of who knows what RLHF is?"* - Ends with the same energy: *"I love questions. I love the interactivity."* and *"I would love questions, feedback, disagreements, strong stuff."* ## Recurring Teaching Habits - **Numbered lessons** structure the talk: *"lesson one for my talk is..."*, *"Lesson number two is..."*, *"Oh, lesson three."* - **Explicit framing questions** then answering them: *"what is the sane view of AI?"*, *"Why do all LLMs require a human in the loop?"* followed by *"The simple answer is we literally put them in the loop."* - **Repeated "this is/by design" definitional phrasing**: *"overpromising is a feature. This is by design."*; *"This is like it's like in the name. This is not like a controversial take."* - **Pre-emptive complexity flags**: *"This is a little advanced, but..."*, *"This is a very advanced topic."*, *"maybe this is a little bit philosophical for you guys."* ## Humor and Self-Deprecation - Labels opposing camps as *"cult one"* and *"cult two."* - Disarms a joke: *"I I I love that you're laughing at this."* - Self-critical credential framing: *"I'm one of the few people at OpenAI who actually hates on chat GPT"* — immediately qualified: *"I don't hate chat GPT as a product, to be clear."* - Uses an absurd concrete example to make a point (the fart-sound-effects tweet). ## Credibility Establishment - Front-loads credentials: *"Co-authored to GPT-4, chat GPT, RLHF {slash} instruct GPT. Um the team I was part of basically invented post-training as a concept."* - States uncertainty where honest: *"As far as I can tell by usage, 100% roughly of LLMs are trained with RLHF"* and *"the numbers probably have changed"* (on the meta study). ## Repetitive Verbal Patterns - **Filler clusters and stutters on emphasis**: *"the the the the way it converses"*, *"And that That to me is like tragic."* - **Contrast pairing as rhetorical structure**: *"not just going well, it's going insanely well"* vs. *"not just going poorly, it's going like insanely poorly"*; *"smarter software"* vs. *"cheaper to write."* - **Superlative intensifiers**: *"super boring," "super obvious," "super fun nuanced discussion," "super valuable."* - **Repeated "kind of / like" hedges** before strong claims: *"this is like a wild state of affairs."* ## Handling Constraints and Q&A - Explicitly negotiates time: *"I've been told to not take Q&A until after"*, *"I believe I'm overtime cuz this red thing is is beeping, but please find me afterwards."* - Defers when a question is too deep: *"that's complicated. And I actually think I don't have the time to answer that particular question"* — then gives a *"simplified view."* ## Promotional Close - Pitches his company and social channels with playful framing: *"follow me and I will post really spicy things. I actually will post something later today that I guarantee will be very spicy."* ## Uncertainty Notes - The video title ("Jev CEO") does not match the speaker's self-identification as **"Tiago Almeida"** in the transcript; no explanation is observable in the source, so this discrepancy is reported, not resolved. - Filler-word frequency and stuttering are strongly evidenced across the transcript; tonal qualities (volume, pace in seconds) beyond word patterns cannot be observed from text alone. # Position Extraction: "Jev CEO: I made ChatGPT, now I'm building what's next" ## Speaker - **Tiago Almeida**, self-introduced. Co-author of **GPT-4, ChatGPT, RLHF/instruct GPT**; claims his team "basically invented post-training as a concept." Now working at **TypeSafe** (stealth-stage startup). ## Core Thesis - The talk is about "what's next after the ChatGPT era" — which he says is **not** the "Claude Code era," because Claude Code belongs to the same assistance era as ChatGPT. ## Two Conflicting "Cults" - **Cult one**: AI is going "insanely well" — benchmarks crushed, allegedly exponential growth in autonomous operation time. - **Cult two**: AI is going "insanely poorly" — a bubble, circular financing, everything is a chat app; old-guard "transformative AI revolution" language replaced by "massively valuable B2B SaaS" framing. - His question: what single explanation reconciles superhuman math performance with humans still needed for "dumb" tasks like customer service decisions? ## Key Positions ### Assistance vs. Automation - **Lesson one**: Today's AI (all RLHF-derived) is excellent at **human-in-the-loop assistance** tasks (where the goal is to please the human in the loop) but not automation tasks (where the goal is to remove the human). - Business pattern observed: "make sure that all of the costs are to the user and not to your business" — he calls this "horrible." - Claims ~100% of LLMs by usage are trained with RLHF: "collect human preferences, optimize for human preferences." ### Overpromising is by design - **Lesson two**: RLHF models optimize for human preference, not results, so "overpromising is a feature." - Examples: the fart-sound-audio praised as "eerie vibe atmosphere piece." - Consequence: models "will look right" even when wrong, due to asymmetry in the reward model. - On hallucination (Q&A): pre-training is "phenomenal" and not the problem; hallucination is "intrinsic to optimizing for human preference" — reward-model asymmetry similar to GANs encourages mode-dropping and confidence. ### Critique of the software trajectory - SaaS has "not changed since 2019" except chatbots latched on — predictable given AI is "assistance native." - Early AI pioneers expected software to get **smarter**, not just cheaper to write. Quotes Garry Tan's "golden age of just-in-time software" — he reads it as a double-edged sword. - **Lesson three**: the field will eventually see the current path as a "weird detour"; tomorrow's AI will be for automation and "smarter software." Currently automated work is "a rounding error." ### TypeSafe / what's next - Core question: "what if the AI stack was redesigned for reliability and automation?" - Their approach: **definitely not RLVR** — a third thing optimizing for **calibrated decision-making**, "mainlining the intelligence of pre-trained models." - Even the **shape of the API** differs across paradigms; new post-training branches look "totally alien" then obvious in hindsight. ### On the bitter lesson - Disagrees with Sutton as applied here: "algorithms matter more than compute" is true in games but not reality; his hierarchy: **right task matters more than data matters more than compute**. ### Teaser claim - Announced intent to post that "the original scaling laws were incorrect" (assertion of intent, not argued in the talk). ## Qualifications & Contradictions Preserved - He says he "hates on ChatGPT" yet immediately qualifies: he doesn't hate it as a product — it's "world-changing" and he'll keep using it (same for Claude Code). - Describes himself as "one of the few people at OpenAI who actually hates on ChatGPT" — framing, not a private-belief inference. - The meta study on overpromising is old; "the numbers probably have changed" but he asserts the effect holds "by construction." - OpenAI charter characterization ("not about making profit") is his recollection, presented as an aside. # Speaker Attribution Record ## Video - **Title:** Jev CEO: I made ChatGPT, now I'm building what's next - **Video ID:** `01a15ca8-0741-720d-979a-d7d012b02c3e` - **No curated metadata was supplied**; all facts below derive from the transcript. ## Speakers ### Primary Speaker — **Tiago Almeida** - **Role:** Guest presenter / talk speaker (self-identified: "my name's Tiago Almeida"). - **Claimed credentials (speaker's own claims, unverified):** - Co-author of **GPT-4**, **ChatGPT**, and **RLHF / instruct GPT** ("OpenAI's greatest hits"). - Member of the OpenAI team that "basically invented post-training as a concept." - Currently working at **TypeSafe** (described as "still kind of stealthy"), building an AI stack "redesigned for reliability and automation." - **Delivery context:** Appears to be a live conference/meetup talk with audience interaction, ending with shouted audience questions (no Q&A mic; speaker repeats questions). ### Secondary Voices - **Unidentified audience members:** Interact verbally during the talk (laughter, shouted questions). One question references **Yoshua Bengio's** suggestion about training a classifier head with pre-training; another asks about RLVR and reward injection. - **No host, narrator, or other identified speakers** appear in the transcript. ## Referenced Sources & Third-Party Claims - **OpenAI blog post** on RLHF ("we as in we the OpenAI team had this great blog post") — no title given. - **Meta study** on RLHF preference-vs-results gap — "an old meta study," no citation provided. - **Tweet** about ChatGPT responding to fart-sound audio as "a very eerie vibe atmosphere piece" — attributed to no one. - **Garry Tan** — quoted phrase: "golden age of just-in-time software." - **Yoshua Bengio** — referenced via audience question. - **Sutton's "bitter lesson"** — cited by speaker (algorithms vs. compute). - **OpenAI charter** — referenced for early wording about "doing tons of work." ## Promotional Segments (Flagged) - **TypeSafe promotion:** Speaker promotes his stealth company ("what we are working on at TypeSafe"), announcing an upcoming release and inviting sign-ups on its **mailing list** and **careers page**. - **Personal account promotion:** Repeated calls to "follow me on Twitter" with a teaser about "spicy" upcoming claims ("the original scaling laws were incorrect"). ## Evidence Channel Availability - **Comments:** No comment availability data present in transcript or metadata — **unknown**. - **Visuals referenced by speaker** (slides, mouse pointer, red overtime indicator) are not available in transcript form; slide content is unverifiable. Excellent. I will say that um I might speed run through this. Feel free if you don't disag- agree with something to yell out. It's way more fun for me if things get interactive. Um otherwise, I will go through this. Uh first, can I have like a vague show of hands of who knows what RLHF is? Oh, excellent. I might be able to skip through that part quickly and get into the interactive stuff. So, my name's Tiago Almeida. I'm talking about what's next after RLHF. More accurately, I think this should be called what's next after the chat GPT era that I think we're all in. And my hint for you guys is it is not the Claude code era. I will justify this later on, but I actually believe them to be part of the same era. Why should you listen to me? I was co-authored to what what is basically OpenAI's greatest hits, at least published hits. Co-authored to GPT-4, chat GPT, RLHF \{slash\} instruct GPT. Um the team I was part of basically invented post-training as a concept. So, um very qualified on a lot of this stuff. But what makes me somewhat unique here is that I'm one of the few people at OpenAI who actually hates on chat GPT. Uh thank you. Uh I don't hate chat GPT as a product, to be clear. I think chat GPT is a world-changing product that will probably stay with us for the rest of time unless something better comes up. But I also acknowledge its limitations and I I I think a lot of what's happened in the state of the field can be traced back to minor decisions we made in making the algorithms behind chat GPT. Um I feel like the question that's relevant to everyone in AI right now is what's actually going on. Um there's a lot of like differing opinions, and I think it's really useful to like map out the spectrum and figure out how can smart people have like such different opinions. There's cult one. Um, AI is not just going well, it's going insanely well. Every single benchmark we surpass human level, and as far as we can measure, we are continuously surpassing human performance. You know, like basically every new benchmark, and it's only getting faster and accelerating. You have uh, you know, every Can I see my mouse? Excellent. Basically every like NLP benchmark is getting crushed, and not only that, allegedly the time that LLMs can operate autonomously is growing exponentially. On the other hand, you have AI is not just going poorly, it's going like insanely poorly. AI is a bubble, it's basically generating no value, it's just circular financing deals, etc., etc. And, you know, if AI is so great, why is why is everything just like a chat app right now? Or like a cloud go thing? Um, and a a lot of the people have actually kind of given up on what was the old guard's terminology of a transformative AI revolution. People aren't really talking about that anymore. They're talking about it being like massively valuable like B2B SaaS. So, the only thing that everyone agrees on is like there's just a these extreme points of view and like nothing in between. And everyone basically thinks AI is insane, but like for different reasons. And what I would want to talk about is what is the sane view of AI? Let's take all the evidence of like cult one, it's going super well. Take all the evidence of cult two, it's going super poorly. Like, uh, you know, map them out and try to explain what what what explains that divide. Like, what is the simplest possible explanation of why some things are too good to be true, and some things are not just bad, they are so bad that we would still employ human workers to do like, you know, like kind of like dumb tasks. Um, no offense to any of them. A lot of these tasks on the right seem way, way, way easier than the stuff on the left. Like, how can we be solving like, you know, unsolved math problems, but still customer service requires like humans in the loop in order to actually like make decisions? This I think is like a kind of like a wild state of affairs. And in my opinion, anyone who works adjacent to AI should have an answer to this because this is like the evidence in the field right now. Um, I would normally pause and ask people if they want to like yell out their thoughts in this, but uh, that I don't think we have time for that and I've been told to not take Q&A until after. Um, but I'll just give you my answer to this, which is, in my opinion, the simplest explanation. All the stuff on the left is not just a task that happens to have a human in the loop. In the left, the task The goal of it is to please the human in the loop. These tasks are intrinsically human in the loop tasks. The like Claude code's job is not to just make code work. Um, the the the the way it converses would be totally different. The goal is to please the human in it. And on the other side, all of these tasks that seem way more basic, the goal is to not have remove the human loop. Ideally, it would be running in the background in a server that you never even look at and ideally it eventually becomes like legacy software that you don't really worry about. So, and this is the divide between assistance and automation. Um, lesson one for my talk is that today's AI, everything inherited from our LHF, is incredible at the human in the loop stuff, but not for automation tasks. Tasks. This is a longer side, but the lesson basically every business has learned is do not use AI for decisions with stakes to your business. Um, a common pattern is make sure that all of the costs are to the user and not to your business. So, um, it's oh, totally okay to throw the user at infinite docs in customer service, but it is not okay to make it make expensive decisions. Horrible pattern, but that is the state of AI right now. Uh I can I can blitz through the what is RLHF part cuz you all seem to know what it what it is. Um it's the algorithm behind not just ChatGPT, but basically every LLM today. As far as I can tell by usage, 100% roughly of LLMs are trained with RLHF. And we have this we as in we the OpenAI team had this great blog post on how it worked. Um I will not get into that because you all know it, and this is super boring. Um the summary of this is it is just collect human preferences, optimize for human preferences. Um and if you want to see like an annotated version of this, you can see which parts are collecting human preferences, which ones are optimizing for them. And \[snorts\] this, I think, provides a really clear answer to everyone in the field asking, "Why do all LLMs require a human in the loop?" The And the simple answer is we literally put them in the loop. The goal of the loop is to optimize for human preference. It is not to run software autonomously. It's kind of super obvious. Thank you, my man at the back. The Yeah. I I I love that you're laughing at this. Um and because of that, overpromising is a feature. This is by design. This is an old meta study. Um and the the numbers probably have changed, but by construction, every RLHF model will always have a big difference between human preference and results, even if the results are good, because the main objective you're optimizing for is for human preference. This is just like natural to how LLMs work. Um I love this tweet of um uh sending ChatGPT an audio file of fart sound effects and asking like what What do you think of the music I made? Here's a straight honest reaction. It's a very eerie vibe atmosphere piece. And this is just how RLHF works. If it doesn't know, it will err on the side of doing what it thinks is best for human preference. And this makes total sense if you are a user in the loop because like the end game for all RLHF models is optimizing for engagement. But what you really want if you want automation is for it to just like not give a about the humans and just do the task correctly in a calibrated way. Um Lesson number two is that today's AI was designed for assistance through optimizing for human preference. This is like it's like in the name. This is not like a controversial take. And the consequences are maybe more controversial, but it's like very obvious if you think about what we really are optimizing for, which is no matter how wrong the models are, they will look right because of the asymmetry within the reward model in RLHF. Um and this is where a lot of like the dilemma in the field stems from because people really want automation to happen. Cool. So, back to the original question. I'm over halfway done with the talk and I haven't even answered it. I was just talking about what's RLHF. But this was a framing to talk about what RLHF is to talk about what's next. And I would say the real question is what's next after AI's assistance era, which I think that we are like very firmly in right now. And back to the original clue of why it's not Claude code. It's actually a super fun nuanced discussion, but it's not Claude code because Claude code is still part of that assistance era. Claude code is still RLHF and it'll it would look very very different if it was purely This is a little advanced, but if it was purely RLVR, it would look very very different. And this is why you get like this dilemma with models where sometimes it gets really good at agentic stuff, but it stops following what you actually want. This This like the trade-off in optimization space that keeps dancing, but both of these trade-offs in optimization space do not add to the automation component. And like that leads to what I think the the logical answer of what's next after assistance is real automation. Um to talk about a little bit about the automation and how that would work, I want to talk about software. Um maybe this is a little bit philosophical for you guys, but I think it's when it clicks and hopefully it clicks if I do a good job. It it I I hopefully it'll be like really clear, which is I'm a lover of software. I assume everyone here loves software. Software is like super valuable. See all the SaaS. And kind of like the craziest part of software in my opinion is that all of the SaaS basically has not changed since 2019. Like SaaS is not really changed in the LLM era, except sometimes a chatbot is like latched on, which is like kind of insane if you think about like the progress made in AI, but is actually very predictable when you think that AI is assistance native, right? Like AI is made for assistance. What can you do in SaaS? Just provide an assistant on the side. And this is not what early AI pioneers used to think would happen. Like when you see like the early wording in opening eyes uh charter, it's about like doing like tons of work, not about like making profit or anything like that. And we used to think that software would get a lot smarter, not just cheaper to write, which is kind of the direction we're going down right now. And I actually really like this phrasing from Garry Tan. Um I think he means this as a compliment to what's going on right now. We're entering the golden age of just-in-time software, but I actually think that this is like a like a double-edged sword. Like I don't just want just-in-time software, which is cool. I I love cloud code, to be clear, just like I love ChatGPT. I would keep using it. But like what I want is smarter software. Why can't like B2B Why can't software just be more expressive? Like why are the like the building blocks of software actually still the same? And um I think this is a question that the whole AI industry should ask itself. And basically every time you're thinking about we want to do automation, it is not about like, you know, an amalgamation of like automating a person's work. It's about like, "Hey, there's this extremely rote work. It's so simple that we can like communicate to someone else that this thing should be done." And ideally it's like it it's so basic that it could be done repeatedly for basically free. Um or it could be done by computers. And that's really not happening right now. What we're doing is we're just automating the writing of the software. But then it its expressibility is the same. And that's That to me is like tragic in the state of the world. Um Cool. Oh, lesson three. Um This is something that I believe strongly in. I believe that like eventually the field will write the I wouldn't say Arlatech is a wrong, but it was like a weird detour and one that we didn't expect. Tomorrow's AI, I believe, will be for automation. And we will eventually have a world with smarter software. Like there will start to be actual work that is automated, which I, you know, right now it's a rounding error despite LLM's intelligence. And that is what we are working on at TypeSafe. We are still kind of stealthy. Like I'm willing to give these talks, but these are like some of the early ones. Um Our core question is what if the AI stack was redesigned for reliability and automation? Like how would that all change? What what would you do? And actually there's a lot It's a It's a very interesting fork in the road for what's go you know, like from basically every LLM that's built today. And I think it's one of the most satisfying things I've worked on, and I've worked on some pretty cool stuff. We are releasing soon. So, um if you want to work with us or you want to like you know, be the first one of the first to build smart software, please sign up on either our mating mailing list or careers page. And I am trying to start a Twitter. So, follow me and I will post really spicy things. I actually will post something later today that I guarantee will be very spicy. Uh the hint is that the original scaling laws were incorrect. Cool. Um that uh that's it for my prepared stuff. I would love Do I have time for for people yelling out questions? I would love questions, feedback, disagreements, strong stuff. I can repeat the question. You don't have to worry about the mic. Hell yeah. Uh cool. Uh the question was roughly what if you trained like a classifier head with pre-training as well? Uh roughly uh like Yoshua Bengio is suggesting. Um I will say that that's complicated. And I actually think I don't have the time to answer that particular question. I will give like my simplified view on this. And it the answer is I actually don't think that pre-training is the problem. I think pre-training is uh phenomenal. Like the fact that we compress the knowledge of the internet into like this core of intelligence that then can be utilized is incredible. And the pre-trained models are incredibly intelligent. Uh and I believe that the problem is like how we unearth it. And hallucination \[clears throat\] to me is intrinsic to um optimizing for human preference. Like there's an asymmetry in the reward model kind of like a GANs have. Oh, I really should not get This is a very advanced topic. But there's an asymmetry in the reward model like what GANs have that allow for um that encourage the models to drop modes and be confident because it's very easy to see when the model is not confident and to punish that from a reward model perspective. It's very complicated, but uh I'm happy to chat afterwards if you want to jam. Cool. Oops. Um I have other slides from other talks as well that I could go into more about that. I have a minute left. Hell yeah. Say it again. It is definitely not RLVR. So, it is a new thing. Every single optimization stack I will actually go into an old presentation that I have because I think this is super important. Um in terms of like to me what the like Sutton's bitter lesson is that algorithms matter more than compute. This is true in games, but not true in reality. I actually think that the full stack is that data matters more than compute and doing the right task matters way more than data. And basically every single branch of LLM post-training if you want to call it has its own North Star of what it's optimizing for. So, RLHF is optimizing for human preference. RLVR is optimizing for like log error rates of pure correctness, but we are doing a third thing that is optimized for calibrated decision-making and like basically mainlining the intelligence of pre-trained models into like being actually useful for software, which I think is like quite different. Uh could you say that again? Uh they're asking if the the reward is injected through the whole process. I will actually say that even the shape of the API is different because the shape of the API for RLHF is different from RLVR, which is different from what we are doing. So, we are like thinking about it from scratch just like no one thought about instruction following before we made instruction following happen. Um usually when there's a big branch in new ways to post-train, like it it it just looks like totally alien, and then in hindsight becomes super obvious. Cool. I believe I'm overtime cuz this red thing is is beeping, but please find me afterwards. I love questions. I love the interactivity. Um and uh follow me on Twitter for spicy stuff. Heck, yeah. Oh, oh yeah, it's over here. Complete skeptic. Um it's it's on brand for me. Cool. Heck, yeah. Thank you. \>\> \[music\]