Alessandro Cappelli (Adaptive ML) argues that GenAI pilots fail to reach production because of a feedback integration problem rather than a deployment or prompt engineering problem. The talk outlines a production-grade reinforcement learning pipeline used at Fortune 500 scale, covering synthetic data generated as a byproduct of environment training, mock environments where agents can fail safely before touching real systems, and LLM judges that replace annotation campaigns with rubric-definition exercises. It also explains why agents — with more tokens, less error tolerance, and live database access — make RL especially suited to the problem. # Channel Identity Observations ## Explicit Self-Description (Creator Claims) - **Company and product**: Speaker identifies as **Alessandro Cappelli, co-founder and chief customer officer at Adaptive ML**, which builds an **"RL Ops platform"** (reinforcement learning operations) for enterprises. > "At Adaptive ML we build an RL Ops platform… that allows large enterprises like AT&T, Manulife, CCS to build, evaluate, and serve in production their own specialized large language models." - **Named product**: The **Adaptive Engine** is described as a "holistic platform where you can observe, train, and serve at once," built on open-source base models (Gemma, Mistral, Qwen mentioned). > "In the last 2 years at Adaptive, we built the Adaptive Engine that is an RL Ops platform to evaluate, tune, and serve the best LLMs for your business." - **Claimed credentials**: Speaker claims to have been on the team that trained **Falcon**, "one of the most widely adopted open-source models" ~3 years ago, and says his team is the founding team of Adaptive ML. - **Customer claims**: Named enterprise clients: **AT&T** (transcript summarization at scale), **Manulife** (pre-existing agent workflows), **CCS** (medical supply customer support, containment-rate optimization). ## Intended Audience - **Enterprise ML/AI decision-makers** — the talk frames problems in terms of enterprise scale economics, production deployment, and ownership: > "When you are a big enterprise like AT&T, any use case… will cost you millions of dollars." - **Teams stuck between MVP and production** — the central framing targets organizations whose GenAI pilots fail ("95% of GenAI pilots fail to reach production"). - **Technically informed listeners**: references PPO, GSPO, RLHF, rejection sampling, reward models, and a Cursor blog post from an audience Q&A — indicating a conference/workshop audience of practitioners. ## Claimed Scope - **Domain**: Post-training / reinforcement learning for LLMs, productionization ("last mile"), agent training, reward modeling, evaluation. - **Service scope**: End-to-end RL Ops — evaluation, tuning, serving, pre-built RL recipes, reward-model training from production feedback. - **Breadth claim**: Applicability asserted across "summarization… classification… OCR… anything you can think of," plus agents. ## Observable Presentation Patterns (Demonstrated, Not Claimed) - **Sales-oriented conference talk**: Talk is structured as problem framing → product pitch ("all of that is possible with the adaptive engine"), including a reference to colleagues' workshop on "our engineering website." - **Vendor perspective**: Every technical point converges on the Adaptive Engine's value; complexity of RL is framed as "the catch" the platform solves. - **Q&A engagement**: Speaker handles a technical question on implicit vs. explicit human feedback, describing workflows (prompted large judge models → trained reward models at production scale) — demonstrating practitioner-level fluency beyond pure marketing. - **Unverified specifics asserted as fact**: The "95% of GenAI pilots fail" statistic is cited without a source; client anecdotes (AT&T costs, CCS use case) are speaker claims, not independently evidenced in the material. ## Evidence Limits - No curated metadata was supplied; identity observations rest solely on title and transcript. - Client relationships, Falcon involvement, and platform capabilities are **creator claims** — no external verification available in the source material. # Delivery Observations — Alessandro Cappelli, Adaptive ML ## Tone & Register - **Conversational, sales-oriented technical talk**: mixes engineering terminology with accessible framing, e.g. *"Reinforcement learning unlocks, let's say, outsized performance."* - **Persuasive, thesis-driven**: repeats a central claim throughout — *"RL is not just any other algorithm for post-training"* ... *"RL not being just one algorithm, but the one algorithm that industrializes bringing model to production."* - Uses **rhetorical questions** to structure the talk: *"Why that is the case?"*, *"What about the reward?"*, *"Where is the human in the loop?"*, *"what's the catch with RL?"* ## Recurring Verbal Habits - **Filler phrases "you know" and "let's say"** appear very frequently throughout: *"which uh I believe it's false"*, *"let's say, a time description"*, *"Might be a systematic reward when it comes from, you know, does the code run?"* - **Hedging with "I believe" / "in my experience"**: *"I believe it's what we call the myth of the last mile"*, *"In my experience, nobody wants to run an annotation campaign."* - **Self-corrections and restarts**: *"at its core will bring models to production"* preceded by *"is not just any other algorithm"*; *"a rel-enabled"* (apparent abbreviation of "RL-enabled"). ## Vocabulary & Framing Devices - **Business/economics vocabulary**: "tokenomics," "MVP," "KPIs," "containment rate," "production," "at scale," "ownership" — e.g. *"the tokenomics of your use case will eventually make sense."* - **Numbered/listed structure when unlocking benefits**: *"Another things you unlock is that smaller model will be faster... The last thing you unlock is ownership."* - **Named frameworks coined on the spot**: *"the myth of the last mile"*, *"first mile vs. the real marathon"* — *"Getting to an MVP is not easy but it's just the first mile."* - **Concrete customer anecdotes as evidence**: AT&T summarization costs, Manulife's existing agents, CCS containment rate, and a relatable latency example: *"When you're talking to someone and you get and you have to wait half of a second, that's already weird."* ## Teaching Habits - **Compare-and-contrast method**: repeatedly pits RL against prompting and instruction fine-tuning: *"Reinforcement learning is disproportionately more effective than instruction fine-tuning, and likewise versus versus prompting."* - **Question-answer scaffolding**: poses an objection then resolves it — *"So, where does the human in the loop come in the equation that I just showed you?"* - **Concede difficulty, then resolve with product**: *"The only catch with RL is that reinforcement learning is actually hard"* → follows with how the Adaptive Engine handles it. - **Redirects to external resources**: mentions colleagues Letizia and Joao's workshop on the engineering website. ## Q&A Delivery - **Confirms engagement with brief acknowledgments** (*">> Yeah."* repeatedly) while the audience member asks. - Restates the question before answering: *"what what you ask is how do we, you know, uh leverage a human feedback."* - Answers with **scenario-splitting structure**: *"I would say there's two, let's say, scenarios."* - Closes politely: *"You're welcome. Okay. And then, thank you very much."* ## Uncertainty Notes - Transcript lacks reliable timestamps; the bracketed line numbers suggest sentence segmentation, so pacing and pauses cannot be assessed. - Tone beyond words (vocal inflection, gestures) is unobservable from text-only transcript. - Some transcript artifacts (e.g. "Wen"/"Gwen" for Qwen, "Gemma 4", cut-off sentence *"easily mocked by"*) appear to be transcription errors; no delivery conclusion should be drawn from them. # Position Record: "Lessons from Trillion Token Deployments at Fortune 500s" — Alessandro Cappelli, Adaptive ML ## Speaker - **Alessandro Cappelli**, co-founder and Chief Customer Officer at **Adaptive ML** (identified in transcript). - A brief **audience question** is posed near the end (speaker unidentified). ## Background & Credentials Claimed - Adaptive ML builds an **RL Ops (reinforcement learning operations) platform** serving enterprises including **AT&T, Manulife, CCS**. - Cappelli was part of the team that trained **Falcon**, described as "one of the most widely adopted open-source models" ~3 years ago. ## Core Thesis > "RL is not just any other algorithm for post-training, but is an algorithm that at its core will bring models to production." - Claimed reasoning: the gap between open-source models and proprietary frontier models (e.g., OpenAI) is **reinforcement learning**. ## Central Claims and Reasoning ### 1. "The Myth of the Last Mile" - Cites a claim that **95% of GenAI pilots fail to reach production** (attributed as a general statistic, no source given). - The common timeline view — that building an MVP is the hard part and production is just "the last mile" — is, per Cappelli, **false**. - Real journey: MVP-to-production is the "real marathon"; the secret is **accelerating model lifecycle** via continuous retraining driven by client feedback, business metrics, and environmental reward. ### 2. Why Proprietary Models and SFT Fall Short - **Proprietary models**: only recourse for defects is changing the system prompt — no systematic, mathematically integrated way to improve. - **Instruction fine-tuning (SFT)**: iterating on datasets is expensive and unsustainable post-production ("Will you keep creating a new data set every single week?"). - **Claim**: RL is "disproportionately more effective" than SFT and prompting; equal performance achievable with a **much smaller model**. ### 3. Unlocked Benefits of RL (as claimed) - **RL-enabled scale/tokenomics**: smaller models are cheaper to serve. Example: AT&T summarizes every customer–agent transcript, which he says "cost them millions of dollars." - **Speed**: latency constraints — he argues speech-to-speech support requires ~⅓ second, "something you will never get if you're using large language models"; need small models (~10B family: Gemma, Mistral, Qwen/Wen). - **Ownership**: own the data and the solution; no risk of vendor model updates shifting performance. ### 4. Agents - Agents raise complexity, token counts, error tolerance, and question the tokenomics further ("Imagine if you scale an agent to actually 10x the number of tokens"). - Claim: RL "naturally fits" agents because it "was actually made to train robots… to live in environment." - Two scenarios: **existing agent environment** (e.g., Manulife) — plug a trained model directly in; or **build a mocked environment** (mock tools, mock user as an LLM). - Reward can be any business outcome, KPI, or LLM-as-judge. ### 5. Data Problem and Synthetic Pipelines - Claim: **agent training data doesn't exist in the wild** — nothing scrapeable of an agent using tools. - With environment + reward, a **synthetic dataset pipeline** emerges as a byproduct; rejection sampling creates data to bootstrap first training. - Existing enterprise data (e.g., real transcripts) can train the mock user for realism — e.g., medical supply customers in panic; correct behavior might be escalating to a human or calling 911. ### 6. Human in the Loop - **RLHF** became famous via ChatGPT's blog post, but he argues it often hides **expensive annotation campaigns**: "nobody wants to run an annotation campaign. It is either expensive, or it is really useless." - Human's actual role (per Cappelli): **defining rubrics, judge system prompts, and scenarios** — minutes to hours, not weeks. - Reward sources: systematic rewards (does code run, containment rate — CCS example), KPIs, LLM-as-judge for open-ended criteria (tone, business guideline compliance). ### 7. Adaptive Engine (Product Pitch) - RL Ops platform to **evaluate, tune, and serve** LLMs — observe, train, serve at once; built on open-source models (Gemma, Mistral, Qwen/"Gwen"). - **Acknowledged caveat ("the catch")**: "reinforcement learning is actually hard" — e.g., **PPO requires orchestrating four large language models simultaneously**. Adaptive Engine offers pre-built recipes (e.g., GSPO) to absorb this complexity. ## Q&A Exchange ### Audience Question (unidentified speaker) - References Cursor's blog post on using production human feedback (accepted/rejected top completions) to update models. - Asks: with a **single implicit production signal** (vs. multiple rollouts per prompt), do you do replays/variations, or is single implicit feedback effective as a reward signal? ### Cappelli's Response - **Two scenarios**: - **Small feedback sets (~10–20, pre-production)**: used to improve LLM-as-judges; initially a large prompted model (e.g., "Qwen 235B" — transcribed "when 235B") serves as judge. - **At production scale (thousands of feedbacks)**: data used to **train reward models** to scale human feedback. - For **implicit feedback**: "it really depends on the specific use case"; reward models built accordingly; with data available, multiple training scenarios can be compared against evaluation to pick the best. ## Qualifications & Tensions Preserved - The 95% pilot-failure statistic is asserted without citation. - "Gemma 4" and "Wen/Gwen" naming appears as spoken in the transcript, potentially imprecise. - Cappelli's claim of RL's superiority is framed from his company's position (Adaptive ML sells the platform described). - He simultaneously argues annotation campaigns are "useless or expensive" while insisting humans stay in the loop via rubric/scenario definition — the tension is his own framing, not resolved. # Speaker Attribution & Provenance Record ## Video Identification - **Title:** Lessons from Trillion Token Deployments at Fortune 500s — Alessandro Cappelli, Adaptive ML - **Video ID:** `01a15ca9-4fc2-723d-b970-5a6da7406b13` - **Format:** Single presenter talk followed by one audience Q&A exchange ## Speakers & Roles ### Primary Speaker — Alessandro Cappelli - **Role:** Creator / presenter - **Self-identified role:** Co-founder and Chief Customer Officer at **Adaptive ML** - **Provenance note:** Was part of the team that trained **Falcon** (~3 years prior), described as part of the founding team of Adaptive ML - **Function:** Delivers the full presentation and answers the audience question ### Audience Questioner (unnamed) - **Role:** Audience member - **Identification:** Not named in transcript; no metadata supplied - **Function:** Asks a single question about human feedback in RL, citing an external source (see below) ### Referenced Third Parties (non-speaking, named) - **Letizia and Joao** — identified as Cappelli's colleagues at Adaptive ML; recorded a workshop on training models against existing environments, hosted on the company's engineering website ## Referenced Sources & Third-Party Claims ### Claims by third parties - **Cursor blog post** (cited by the audience questioner, "last year"): describes using human feedback from production data (e.g., whether top completions are accepted) to continuously update models ### Claims/figures attributed to others by the presenter - **OpenAI blog post:** credited with popularizing RLHF in the LLM world ("we did RLHF") - **"95% of GenAI pilots fail to reach production"** — statistic presented without a cited source ### Named clients / organizations - **AT&T** — customer; cited example of transcript summarization costing "millions of dollars" - **Manulife** — customer; described as already having agent workflows in place - **CCS** — described as a medical supply company customer with a customer-support system maximizing containment rate ### Referenced models/technologies - Falcon, ChatGPT, Sonnet, Gemma (incl. "Gemma 4"), Mistral, "Wen"/"Gwen" (spelling varies in transcript), Qwen 3.5 / "235B" (transcript spelling), PPO, GSPO, SFT, RLHF ## Promotional Segments — Flagged - **Entire talk carries promotional framing** for Adaptive ML's product, the **Adaptive Engine** ("RL Ops platform to evaluate, tune, and serve the best LLMs for your business") - **Explicit promotional conclusion:** RL "is possible with the Adaptive Engine"; pitch of pre-built recipes handling RL complexity (e.g., PPO's four-model orchestration) - **Company introduction** at the opening promotes Adaptive ML's enterprise clientele (AT&T, Manulife, CCS) - **Cross-promotion:** referral to colleagues' Letizia and Joao's workshop on the company engineering website ## Evidence Channels — Availability - **Comment availability:** Not indicated; no comment metadata supplied - **Curated metadata:** None supplied beyond title; speaker identity comes solely from the transcript - **Visuals referenced but unavailable:** presenter references a plot (RL vs. SFT performance) and time-to-production diagrams not contained in the transcript ## Uncertainty Notes - Audience questioner's name and affiliation unknown - "95% of GenAI pilots fail" has no stated provenance within the talk - Some model names appear with ambiguous spelling in the transcript (Wen/Gwen, Gemma 4, "one 3.5," "235B") Hello everyone. I am Alessandro Cappelli. I'm co-founder and chief customer officer at Adaptive ML. At Adaptive ML we build an RL Ops platform as in reinforcement learning operation that allows large enterprises like AT&T, Manulife, CCS to build, evaluate, and serve in production their own specialized large large language models. I'm I'm here to show you how reinforcement learning RL is not just any other algorithm for post-training, but is an an algorithm that at its core will bring models to production. Around 3 years ago uh I was part of a team that trained Falcon. Falcon around 3 years ago was one of the most widely adopted open-source models and we realized with, you know, my team that is actually the core founding team of Adaptive ML that the gap that was missing between bringing an open-source model to production versus proprietary models like of Frontier Labs like OpenAI was actually reinforcement learning. 95% of GenAI pilots fail to reach production. Why that is the case? Uh I believe it's what we call the myth of the last mile. Here you see, let's say, a time description of what does it get to get to production, which uh I believe it's false. And this idea that the hard part is to get to an MVP is to you know, come up with a demo that looks nice in front of a stakeholders in front of your colleagues and that is the the hard part. And then you know, there will be just the last mile where the last mile will actually get the model into production. The issue is that most MVP are built on top of proprietary models or are built on top of open source model using instruction fine tuning. Both of these solutions won't let you systematically improve your solution. They don't integrate in a really systematic and mathematical way what are the defects you might find in the journey to get to production. I will give you an example. If you test, let's say your solution and there will be some defects and you're using a proprietary models, all you can do is to change the system prompts. Now you change the system prompt in one direction, you might have other defects and there's no really like a really scientific systematic way to improve that system prompt nicely in a way that you can easily monitor. Likewise for instruction fine tuning. Best you can do is to iterate over the data set that might be expensive and what after production? Will you keep creating a new data set every single week? This is what I believe is a more realistic view of what getting to production and beyond actually looks like. Getting to an MVP is not easy but it's just the first mile. What actually the real journey, the real marathon is to get from an MVP to production and beyond. And the secret to do that is to accelerate model life cycle. It's to be able to integrate every single feedback you can get from a variety of sources to keep improving your solution. This continuous retraining, refinement and improvement driven by real client feedback, business metrics and environmental reward is unlocked in a systematic way only by reinforcement learning. Reinforcement learning, as I before, my entire point is that by almost by design, by nature, allows to integrate feedback in a almost a mathematical way. But, reinforcement learning is not just like not just that. Compared to other post-training techniques or steering behavior techniques like, you know, prompting and instruction fine-tuning, they all, let's say, reach the same goal, which is steer a model's behavior, but they're not equally effective. Reinforcement learning is disproportionately more effective than instruction fine-tuning, and likewise versus versus prompting. Reinforcement learning unlocks, let's say, outsized performance. So, what does it mean? What you can see in the plot that, you know, you can get the same performance with a rel with respect to SFT with a much smaller model. What it unlocks that actually help you to get to production. A rel-enabled scale at adoption. What do I mean by that? As you train a smaller model, those models will be cheaper to serve at scale, and the tokenomics of your use case will eventually make sense. When you are a big enterprise like AT&T, any use case, any feature that you want to be a commodity either for internal employees or any any you know, facing customer features you might think at scale will cost you millions of dollars. As an example, AT&T they summarize every single transcript that might happen between, you know, a customer and an agent. Just summarizing that cost them millions of dollars. If you can train a model that is much smaller than, you know, uh you know, uh uh a chat GPT or a Sonnet, you will save money. Another things you unlock is that smaller model will be faster. Now, not all use cases requires speed, but many of them they have a threshold of latency that is not just something nice to have, is a constraint that will prevent you to get into production. Let's say you have a model for customer support that is powering, you know, a speech-to-speech system. You can go above half of a second, and I would say half of a second is already weird. When you're talking to someone and you get and you have to wait half of a second, that's already weird. Ideally, it should be a third of a second, and a third of a second is something you will never get if you're using large language models. You need to use small models. Maybe, you know, the latest Gemma, the later Mistral, the later Wen of, you know, in that 10B family, but you can't use much larger models. The last thing you unlock is ownership. You will own the data that you give to the model cuz the model will be trained on your own business data, and you will own the solution. So, you don't need to worry about, you know, the latest update of the model that may shift performance underneath your feet. Everything I've said so far, it is true for any use case you might think. I mentioned summarization, could have been classification, uh could have been OCR, could have been anything you can think of. And reinforcement learning is already the better choice. But now we are in the era of agents, and agents actually make everything more complicated. Agents require more tokens, more complexity. There is less room for errors because now agents will have access to the data, will change things in the database connected to, you know, either, you know, internal employees or client to you might have. So, all of that raises the standard of what can be brought into production, and it raises further question on whether the tokenomics of an agent actually makes sense or not. I mentioned before, you know, just for a summarization use case, you might spend millions. Imagine if you scale an agent to actually 10x the number of tokens. A real advantage that already existed only widens when it comes to, you know, training agents. Cuz a real at its core was actually made to train robots, you know, to to train agents to to live in environment. And environments are where, you know, agents actually behave. So, a real naturally fits in a narrative where you want to train a model to be a good agent. Now, there are two scenarios. Uh either you already have an agent in place. Uh as an example, you know, like um we work for a Manulife and Manulife already had agents. So, they already have an actual workflow that has been settled. Like, we don't need to recreate it on our side. You can directly plug a model that you might train. Can be, you know, the latest one 3.5 and you can directly train the model on an environment that already exists. If such environment doesn't exist, it can still be built. You can still mock the tools and you can mock, if you need one for the specific case, a mock user. Like, if you want to create a chatbot that has access to tools, a mock user might be an LLM. What about the reward? The reward will be just, you know, any business outcome, any KPIs, any LLM as a judge that might define what success looks like to you. Like, was the agent helpful? Was the agent useful? Was the agent using, you know, a tone and a vocabulary that is following business guidelines? On this topic, two colleagues of mine, Letizia and Joao, they recorded a workshop that you might find, you know, on our engineering website that will show you exactly how you can actually train a model by plugging into an existing environment. When I talk to clients, one of the main, you know, sources of doubts on whether they will ever get to an MVP or to production is because they don't have the data to do it. Data was already an issue before agents. After agent is even more of an issue because agents' training data doesn't exist in the wild. Like, there's no such a data that you can scrape from the web where, you know, an agent is using tool. You don't have such such a data set. The nice thing is that when you train a model with reinforcement learning and you have an environment and you have a reward in place, basically, you just build what is uh as a byproduct of your of your environment, you created a synthetic data set pipeline. Basically, as you have an environment, you can literally create trajectories that are good because the reward that you put in place will tell you what is good and what is not. So, basically, you can do rejection sampling and create a data set that you can use to bootstrap, you know, the first training of a model. And the nice thing is that even though many companies don't have the data uh exact the exact data that is required to train agents, they still have a lot of data sets that can be leveraged to improve the entire experience in the environment. Such data might be, you know, real transcript between a customer and an agent that can be given to the mock user. The mock user can even be trained on that to be the actual realistic person that might be annoying, that might ask things three times in a row. We work with, you know, like with with customers like medical supply where people might call them might be in panic. So, the correct behavior might be, you know, I will escalate you uh to a human agent or will call 911 for you. And that kind of, let's say, dirty real conversation is something that can be easily mocked by Where is the human in the loop? Um IRL became famous uh in the LLM world thanks to ChatGPT, because OpenAI published a blog post where we're saying we did RLHF, so reinforcement learning from human feedback. But, sometimes that human in the loop, which is nice to hear, sometimes what actually hides behind are expensive annotation campaigns. So, in my experience, nobody wants to run an annotation campaign. It is either expensive, or it is really useless, because the reality is that people don't want to do it. But, you still want to keep a human in the loop. So, where does the human in the loop come in the equation that I just showed you? When you train with RL, the most important thing you want to do is to build a reward signal. A reward signal might come from different sources. Might be a systematic reward when it comes from, you know, does the code run? Is the syntax is correct? It can come from direct KPIs or bit and business outcomes. One of our client, CCS, the medical supply company I was mentioning before, as as a customer support system that as any other customer support system, what it's trying to maximize is containment rate. Like, how many calls are actually brought end-to-end by the model. And that reward, that percentage of calls that actually brought end-to-end is something you can directly maximize. Many other things, like was the tone correct? Were the business requirements followed? It's a bit of an open-ended question when it comes to systematic reward, but that issue can be solved with LLMs as judges. So, the human in the loop is helping just by defining the rubrics, defining the system prompt to these LLMs judges, and defining the scenarios. As in, making sure that it's just aligned with what they see. But, this, let's say, activity that that the human will do will take from few minutes to hours, but it will not take weeks, and you don't have to do it iteratively dozens of times. For everything I mentioned before, RL not being just one algorithm, but the one algorithm that industrializes bringing model to production. In the last 2 years at Adaptive, we built the Adaptive Engine that is an RL Ops platform to evaluate, tune, and serve the best LLMs for your business. The Adaptive Engine is an holistic platform where you can observe, train, and serve at once. When I mentioned at the very beginning that the goal is to accelerate the life cycles, that doesn't mean to accelerate training per se, right? You also want to evaluate, be sure that the model is actually behaving, and you want a systematic way to find defect pre- and post-production and to act accordingly. This is something you can do only if you have a systematic, holistic approach. Our models are built on top of the best open-source models. Any, let's say, most open-source model you can think of that are available, like the latest Gemma Gemma 4 that you heard a few days ago. The latest Mistral, the latest Gwen, they're all, you know, available in your company, and depending on, you know, the model of your preference, you can start building on top of it. And finally, what what's the catch with RL? The only catch with RL is that reinforcement learning is actually hard. Reinforcement learning is not as easy as changing a system prompt, and it's not as easy as just building a data set for instruction fine-tuning. One of the most famous RL algorithms, which is a PPO, uh requires orchestrating not one, but four large language models at the same time. That is where, you know, Adaptive Engine shines because we let you define the rubrics and the rest, but we we take care of the complexity of reinforcement learning by exposing a series of pre-built recipes for you. So, you don't need to implement the latest algorithm, say GSPO, and you don't need to build like, you know, the training recipe to run an actual training. So, once again, RL is the one algorithm that will let you bring model into production in a systematic and industrialized way, and all of that is possible with the adaptive engine. Thank you very much FOR YOUR ATTENTION. \>\> I HAVE A QUESTION ABOUT UM UH SOME OF THE HUMAN feedback portion that can get incorporated into RL. \>\> Yeah. \>\> Uh for example, um last year Cursor had a blog post where they outlined how they take human feedback from production data, such as whether or not the top completion is accepted or not. \>\> Yeah. \>\> And they constantly use it to update the model. \>\> Yeah. \>\> Um in settings like this where an LLM is at play, um like in a more traditional LLM RL style, you would do several rollouts per prompt, and pick the ones that uh work, and train on those few samples. \>\> Yeah. \>\> When it's human feedback, and there's a single signal uh single signal, um do you do uh like replays to um have many variations of outputs for that problem and train on it, or is it effective to just have a single uh implicit feedback from production train on as a reward signal? \>\> So, I would say, you know, like what what you ask is how do we, you know, uh leverage a human feedback. Uh I would say there's two, let's say, scenarios. There's a scenario where, you know, sometimes the human feedback is like uh it doesn't come from production, right? It comes from, you know, uh like 10 to 20 feedbacks. In that scenario, what we do is that we basically usually use it to improve the LLM as judges, as in that is good, that is bad, like how does it fit into the current description of, you know, of what you are trying to do. And the nice thing is that as you go to production, then you will have thousands of such feedbacks. So, what we do is that we usually, rather than using for LLM as judges, at the very beginning we just use prompted a really big large language model, say when 235B. As we move to production, we have so much data that what we do is that we use this data to train reward models so that we can basically scale that human feedback in like to actually train actively the the And then, you know, with respect to, you know, to the question on when these feedbacks are not as explicit, but more like implicit, I think it really depends on the specific use case, and then we can build a reward model accordingly. You know, because we already have the data, we can have do two different scenarios to see which kind of training actually gives you the best output give at the best performance given a certain evaluation. You're welcome. Okay. And then, thank you very much.