Talk by Maximilian-David Rumpf (SID.ai) on applying reinforcement learning to search for AI agents. He notes that agents spend a large share of tokens (roughly 30–50%) on searching, and that agentic search finds the right documents about twice as often as traditional query pipelines but at 100–1000x the cost and far greater latency. He argues classical pipelines fail because their decisions are fixed at design time with uniform compute budgets, even when the reranker can tell results are inadequate, producing a long tail of failures patched with non-exhaustive edge cases. Drawing parallels to computer vision and chess, he contends search is well suited to RL because the reward is verifiable and the environment supports rapid iteration; reported results show a specialized model completing the same task ~20x faster (5s vs 2min) at roughly 1/100th the cost of a frontier model. Topics include where the classical retrieval pipeline breaks, machine vs human design, why search suits RL, and keeping bad results out of agent context. # Video Identity Observations — "Where RL Will Take Search — Maximilian-David Rumpf, SID.ai" ## Speaker / Channel Self-Description - The speaker identifies himself as a **founder and CEO**: > "I'm the founder and CEO of Citi. We're a stealthish AI lab for search. We're backed by some pretty amazing people. Uh, and we're hiring." - Note: the transcript's "Citi" appears to refer to **SID.ai** (per the video title), suggesting a transcription variance; no correction should be assumed beyond the title evidence. - Explicit channel/company scope claim: an **AI lab focused on search**, described as "stealthish" (early/low-publicity stage). - Explicit recruiting signal: **"we're hiring."** ## Intended Audience - The talk addresses a **technical audience familiar with search infrastructure**: > "This is the pipeline that many of you guys will be familiar with. A question comes in, you might have an LLM that rewrites the question..." - Assumes familiarity with concepts like rerankers, vector search, BM25, PageRank, token budgets, and RL training — indicating an audience of **ML/search engineers and AI practitioners**. - References to Alpha Zero, MuZero, Deep Blue, Stockfish, and the CPU/GPU/ASIC analogy suggest a **research-adjacent technical audience**. ## Claimed Scope of Work - Core thesis: **reinforcement learning applied to search/agentic retrieval**, replacing hand-designed pipelines: > "This new paradigm of pure RL where we actually don't bake any design decisions into the model." - Domains benchmarked (self-reported): **legal, finance, knowledge bases, science, email**, plus "some academic benchmarks, some internal benchmarks." - Claimed results for their model ("SID one"): - ~**20x faster** ("instead of taking around 2 minutes, you take around 5 seconds on average") - ~**100x cheaper** than a frontier model for the task - Forward-looking claims (creator claims, not demonstrated): RL scaling will yield "arbitrarily good search in any domain," enable voice and e-commerce use cases, and "unlock more knowledge work tasks," with the example that enterprise data ("how to run JP Morgan... deep inside of the databases at JP Morgan") is more valuable than web data. ## Demonstrated Presentation Patterns (Observable) - **Founder-led product/mission talk format**: personal background slide, company intro, hiring mention — consistent with a startup pitch/demo talk rather than pure academic lecture. - **Evidence-driven argumentation style**: repeated benchmark charts ("This is the same chart as before"), baselines (reranker, vector-only), and quantified cost/latency comparisons. - **Analogy-heavy exposition** to ground the RL thesis: computer vision's progression (edge detection → narrow models → VLMs), chess (Deep Blue → Stockfish → Alpha Zero/MuZero), and hardware specialization (CPU vs. GPU vs. ASIC). - **Positioning pattern**: frames the work as the next step in a known historical progression ("machine design outperforms human design"), positioning the company at the frontier of that shift. - **Practical framing for developers**: production-focused discussion of agent context pollution and delegating search to a cheap sub-agent ("the main agent only ever sees great results"). ## Claims vs. Demonstrations — Distinctions - **Creator claims (stated, unverifiable here)**: backing quality ("backed by some pretty amazing people"), result figures (20x/100x), the "no ceiling yet" observation, and future capabilities. - **Demonstrated (observable in the material)**: technical fluency in search pipelines and RL training dynamics; a structured narrative linking their product to established ML-history trends; a hiring-stage, stealth-mode company posture. - No metadata was supplied beyond the title; all identity evidence comes from the transcript and title. # Delivery Observation Report **Video:** *Where RL Will Take Search* — Maximilian-David Rumpf, SID.ai --- ## Overall Communication Style The delivery is a compact technical conference talk (~9–10 minutes based on transcript length) structured as a classic **problem → pattern → solution → results → outlook** arc. The speaker moves quickly, spends almost no time on self-introduction (two sentences), and dedicates the bulk of the talk to argumentation and quantitative claims. --- ## Vocabulary & Word Choice - **Heavy use of intensifiers with informal register:** "vastly higher quality results," "incredibly expensive," "extremely slow," "extremely high recall," "incredibly fast and cheap," "completely incomparable." Nearly every major claim is paired with an absolute adverb. - **Hedging appears sparingly and is immediately contrasted with confidence:** > "we don't know whether this method has no ceiling, but we're definitely not yet seeing a ceiling to this approach." - **Conversational fillers and softeners** punctuate the technical content: "Okay," "Uh," "you know," "kind of like," "like yeah." Examples: > "it's about a hundred times cheaper than using a frontier model for this task. Uh there's some more detail here, but like yeah, the cost and kind of like speed are just completely incomparable." - **Domain vocabulary used without definitions:** "reranker," "BM25," "page rank," "vector only," "context window," "grindable" — assuming a technical audience. --- ## Recurring Phrases & Linguistic Patterns - **Repeated comparative cost/speed framing** — the most consistent rhetorical habit. Concrete numbers appear throughout: > "a hundred to a thousand times more expensive" > "30 to 50% of their tokens on searching" > "you're about 20 times faster... about a hundred times cheaper" > "just you know 100x cheaper search sub agent" - **"And this means that..." / "And what this means is..."** used as connective tissue after nearly every data point. - **Scaled/absolute quantity claims:** "thousands of times per second," "you can't design infinite edge cases," "arbitrarily good search in any domain." - **The phrase "search" itself** is used both as product name and generic activity, sometimes ambiguously (e.g., "SID one" rendered as "search one" adjacent in the transcript — likely a transcription artifact worth noting). --- ## Teaching & Explanation Habits 1. **Analogy to prior paradigm shifts** — the central teaching device. The speaker builds the argument twice through historical parallels: > "machine design outperforms human design. Um, and we saw this in computer vision... You saw this again with chess with IBM Deep Blue..." 2. **Concrete grounding of abstractions.** Verifiable rewards are explained by immediately operationalizing them: > "Verifiable here means for a given question, did you find the correct document?" 3. **Hardware analogy for specialization:** > "a CPU in theory can do anything that a GPU can do, but you would never want to use a CPU to do LLM inference" 4. **Vivid concrete example for the closing point:** > "how to run JP Morgan is nowhere on the web but it is deep inside of the databases at at JP Morgan." 5. **Talk-scaffolding signposting:** "And the question here that we'll answer today is...", "And let's quickly look at classical search.", "And these are the results.", "So this is the same chart as before." --- ## Structural & Interactional Patterns - **Chart-driven narration.** Multiple segments consist of walking the audience through a visual: "You see reranker and vector only performance at the bottom and you can see frontier models essentially kind of like go through here..." - **Rhetorical question → answer pattern:** > "And the question here that we'll answer today is how much cheaper and faster can we make this with reinforcement learning?" - **Brief audience acknowledgment** appears once: "Uhhuh." after presenting the RL search loop — possibly reacting to audience response; evidence is too thin to characterize interaction style. - **Rapid pivot closers:** transitions often occur mid-sentence with "And" as the primary connective throughout ("And this is really our target." "And what helps us here is..."). --- ## Stable Patterns vs. Isolated Moments **Stable (supported by multiple instances):** - Quantified comparative claims (5+ instances) - Intensifier + noun construction ("incredibly/expremely/vastly + adjective") - Historical-analogy argumentation (three separate domains) - Signposting of talk structure **Isolated / low confidence:** - The audience-interaction moment ("Uhhuh") — single occurrence - The "stealthish" self-description and hiring mention — one-off framing at the open, no follow-up - Hedged uncertainty statement about the ceiling — appears once; cannot confirm it is a habitual practice vs. a single calculated concession --- *Note: all quotations are taken verbatim from the supplied transcript, including filler words and repetitions; timestamps refer to transcript segment markers, not verified wall-clock times.* # Observation Document: "Where RL Will Take Search" — Maximilian-David Rumpf, SID.ai ## Speaker - **Maximilian-David Rumpf**, founder and CEO of SID.ai (also referred to as "Citi"/"SID" in transcript; stealth-ish AI lab for search). He notes the company is hiring and backed by unnamed "amazing people." ## Core Claims ### Agents as a new paradigm for search - Agent-based search yields **vastly higher quality**: "twice as likely to find the right documents." - But it is **100–1000x more expensive** than classical search and **extremely slow** (minutes vs. milliseconds). - Agents spend **30–50% of their tokens on searching**, usually at the start of a task, to gather context. - Proposed solution: offload searching to a **trained sub-agent**; the talk's central question is how much cheaper/faster RL can make this. ### Limitations of classical search pipelines - Standard pipeline: LLM query rewrite → search backend → reranker → results. - Characterized as **chained, locally optimized models** with all decisions **baked in at design time** and **fixed compute per question**. - Key limitation: the reranker may know results are insufficient but **cannot take action** — it can only return bad results. - Result: a **"long tail of failure"** on unexpected questions; practitioners patch with edge cases, but "you can't design infinite edge cases." ### Machine design outperforms human design (stated pattern) - **Computer vision**: hand-coded edge detection → narrow box-around-dog models → general VLMs. - **Chess**: Deep Blue (human rules) → Stockfish (bridge) → AlphaZero/MuZero (all inside the model). - Prediction: search follows the same arc — BM25/PageRank → vectors/rerankers → **pure RL with no baked-in design decisions**. ### The RL search agent design - One model interacting iteratively with the database: search, read, iterate, re-search, set **metadata filters on the fly**, constrain or expand search, then output a ranked list. - Adapts compute per question; spends more on hard questions. - Why search suits RL: rewards are **verifiable** (did you find the correct document?) and **grindable** (thousands of attempts per second during training). - Specialization argument: most LM components aren't needed; analogy to CPU vs. GPU vs. ASIC — a CPU could do GPU work in theory but is far less effective. ### Results claimed - Search quality **increases predictably with compute**; latency and retrieval-strategy rewards can be mixed in. - Explicitly does not tell the model what to do — wants it to discover its own strategies, like AlphaZero. - Qualification: "we don't know whether this method has no ceiling, but we're definitely not yet seeing a ceiling." - **SID one** results: ~**20x faster** (~2 minutes → ~5 seconds average), ~**100x cheaper** than a frontier model. - Qualification: **not yet at vector+reranker latency**, but he believes they "can get there quite quickly." ### Production behavior - Bad search results **pollute the main agent's context window**. - A search sub-agent absorbs all searching/thinking/iterating; the main agent "only ever sees great results," improving correctness and cost (search tokens shift to a 100x-cheaper sub-agent). ### Predictions for the future - Scaling RL → "arbitrarily good search in any domain." - RL models will get faster (enabling voice, e-commerce) and cheaper. - Better search unlocks more knowledge-work tasks; the **web is small relative to total data** — "the most valuable information is not on the internet," e.g., how to run JP Morgan exists only in JP Morgan's internal databases. ## Qualifications & Tensions Preserved - Presenter's own uncertainty: ceiling of the method unknown (though none observed yet). - Speed claim is comparative, not absolute: not yet matching vector+reranker latency. - All performance figures (2x quality, 100–1000x cost, 30–50% tokens, 20x/100x improvements) are presenter-stated claims from internal/academic benchmarks, without independent detail shown in the transcript. # Speaker & Provenance Record ## Video Overview - **Title:** *Where RL Will Take Search — Maximilian-David Rumpf, SID.ai* - **Video ID:** `01a15ca7-5d70-7f63-a962-9bbc9f88cc9d` ## Speakers & Roles | Speaker | Role | Evidence | |---|---|---| | **Maximilian-David Rumpf** | **Creator / Presenter**; self-identified **Founder & CEO of SID.ai** | Title attribution + statement: *"I'm the founder and CEO of Citi"* (transcript rendering of "SID" is uncertain — see Notes) | - Single-speaker presentation; no guests, interviewers, or narrator distinct from the presenter. - A second voice indicator (`>>`) appears only at the closing `[music]` cue; **no audible second speaker content** is present in the transcript. ## Self-Identified Affiliations - Founder and CEO of the company referenced as "Citi" in the transcript, contextually **SID.ai** per the title. - Company described as: *"a stealthish AI lab for search"*, backed by unnamed investors (*"backed by some pretty amazing people"*). - **Promotional segment flagged:** the closing *"and we're hiring"* is an explicit recruitment/promotional statement within the talk. ## Referenced Sources & Claims (all made by the speaker, not independently verified) ### Third-party systems & named entities - **Benchmarks:** mixed set spanning *legal, finance, knowledge bases, science, email*, plus academic and internal benchmarks (internal ones not published in the transcript). - **Named products/organizations cited as comparisons:** - IBM Deep Blue, Stockfish, AlphaZero, MuZero (chess/CV analogy) - BM25, PageRank, vector search, rerankers (classical search pipeline) - "Turboroper" (mentioned as a fast classical search reference; unclear spelling — possible "TurboPuffer") - **JP Morgan** (example: internal databases vs. public web) ### Quantitative claims made by the speaker - Agentic search: **~2× more likely** to find right documents; **100–1000× more expensive**; minutes vs. milliseconds latency. - Agents spend **30–50% of tokens on search**. - Trained model (SID 1): **~20× faster** (~2 min → ~5 s) and **~100× cheaper** than a frontier model. - Training throughput cited as *"thousands of times per second"* per question during training runs. ## Availability & Provenance Limits - **No cited papers, datasets, URLs, or external references** appear in the transcript; all charts ("this is the same chart as before") are described verbally only. - Internal benchmark details and investor identities are **not disclosed** ("stealthish"). - The transcript itself is available; **comment availability is not stated** anywhere in the source. - Spelling of "Citi" vs. "SID" and "turboroper" is **uncertain** from transcript text alone; title metadata supports "SID.ai." ## Uncertainty Flags - 🔶 Company name transcription ambiguity ("Citi" vs. SID.ai). - 🔶 Benchmark composition (which academic/internal benchmarks) unspecified. - 🔶 Charts/results described but their underlying data sources are not identified in the transcript. Okay, today I'm going to talk about where reinforcement learning will take search and some some background on me. Uh I'm the founder and CEO of Citi. We're a stealthish AI lab for search. We're backed by some pretty amazing people. Uh, and we're hiring. Okay. Agents are a new paradigm for search. You can now get vastly higher quality results. Um, twice as likely to find the right documents. But it's incredibly expensive. um about a hundred to a thousand times more expensive than what you'd get out of a classical search query and it is extremely slow. You're looking at minutes and not milliseconds like turboroper. And what this means is that agents spend 30 to 50% of their tokens on searching. And this is usually at the beginning of some task. It finds the right context to then do whatever you ask it to do. And the idea here is quite simple. First, instead of having the main agent do the searching, you pass the searching to a sub agent and you train a model to be a great sub agent. And the question here that we'll answer today is how much cheaper and faster can we make this with reinforcement learning? And this is really our target. So this is a benchmark across legal, finance, knowledge basis, science, email, a bunch of different tasks, some academic benchmarks, some internal benchmarks. And this is where you currently are. You see reranker and vector only performance at the bottom and you can see frontier models essentially kind of like go through here at the cost of spending many many you know minutes per question. And can we get a model to kind of like be extremely accurate um have extremely high recall but also be incredibly fast and cheap. And let's quickly look at classical search. This is the pipeline that many of you guys will be familiar with. A question comes in, you might have an LLM that rewrites the question. You then execute that on a search backend. You might have a reranker. Um and you get your results at the end of the day. Um, it is essentially a pipeline of chained locally optimized models and all of the decisions are baked in at design time and you expend a fixed amount of compute per question. And this one is really important. The re-ranker might know that the results are insufficient at answering the question, but the re-ranker can't take action. It can only essentially return the results even when they're bad. And what this means is that in practice a pipeline like this acrrues a long tail of failure where unexpected questions come that you know the designer didn't have something for. And in practice this usually means people add lots of edge cases um to essentially fix these. But of course you can't design infinite edge cases and you can't add infinite tweaks. And so the strategy here is one that we've seen before. machine design outperforms human design. Um, and we saw this in computer vision where you had your, you know, primitive edge detection algorithms. You then had the box around a dog generation of models um that were very good at this like very narrow task and locally optimized for it. And then you had VLMs that were extremely good at all parts of the search pipeline. You saw this again with chess with IBM Deep Blue being largely a collection of human written rules. um stockfish bridging the two and then alpha zero and mu0ero essentially completely just putting it all inside of the model and we're going to see something similar happen to search where we have our existing algorithms like BM25 and page rank um then we had an evolution from that with small models that did some task very well like vectors and re-rankers and now essentially this new paradigm of pure RL where we actually don't bake any design decisions into the model. And what this looks like in practice is um something like this. Um so you have one model uh it goes back and forth with the database. It can search, it can read results, it can iterate. Uh it can search again until it is happy. It can set metadata filters on the fly. It can constrain its search. It can try as much as it wants. Uh and in the end, it produces a ranked list of results. Uhhuh. And what you get is a model that makes all of the decisions and can adapt to any question on the fly and for example use much more compute if a user asks a very difficult question. And what helps us here is that search is verifiable. Um and reinforcement learning needs rewards that are verifiable and grindable. Verifiable here means for a given question, did you find the correct document? And we can design this and tell this quite easily. Um, and is there an environment where the model can attempt this question loads and loads of times and in practice for us this means uh thousands of times per second during a training run. Uh, and the second part is kind of like can we turn the models that are currently very general and very general purpose into something that is much more specialized and it turns out we don't actually need most of the parts of a language model to be extremely performant at search. Uh and similarly with like CPUs and GPUs and AS6, um a CPU in theory can do anything that a GPU can do, but you would never want to use a CPU to do LLM inference, for example. Um because the much more specialized version is much more effective. And this really makes search an ideal target for RL. Um and this is what happens when you train a model on this task. And so again, here we added the vector and reranker only baselines. This is of an earlier task. And what we see is that search quality increases very predictably with compute. And we can mix in other rewards like latency um and different kind of like retrieval strategies to make it even more performant. Importantly, we don't really tell the model what to do. Much like in Alpha Zero and chess, we want it to discover its own strategies and its own tricks to essentially search. Well, uh, and we don't know whether this method has no ceiling, but we're definitely not yet seeing a ceiling to this approach. And these are the results. Uh, so this is the same chart as before. And this is SID one and then SID one um with some parallel execution on the left hand side. And so what this ends up meaning is you get you're about 20 times faster. So instead of taking around 2 minutes, you take around 5 seconds on average. And it's about a hundred times cheaper than using a frontier model for this task. Uh there's some more detail here, but like yeah, the cost and kind of like speed are just completely incomparable. Uh you can yeah um it's not quite at the latency of a vector and reranker pipeline, but in practice we think we can get there quite quickly. And how does this look like in production? So this is a usual kind of like agent execution trace. The agent does some searching here. It finds some good stuff. It finds some bad stuff. Um, but all of the bad stuff that it finds is essentially polluting its own context window. And what we can instead do is use um a sub agent here that does all of the searching and thinking and iterating for the main agent. And the main agent only ever sees great results. And this means that the main agent sees more good stuff, which means it's more likely to be correct. And it is also um extremely cost- effective uh where those 30 to 50% tokens that were earlier used by the main agent to do searching can now be passed off to this just you know 100x cheaper search sub agent. Uh and where will this take us? Scaling RL will give us arbitrarily good search in any domain and RL models will become even faster which will allow them to be used in things like voice and e-commerce. They'll become even cheaper than we are currently. Um so the charts that you saw there but like I think we can move even further. Uh and better search will unlock more knowledge work tasks. Uh the web is actually quite small uh in comparison to the entirety of data that is there that is out there. Uh and the most valuable information is not on the internet. For example, how to run JP Morgan is nowhere on the web but it is deep inside of the databases at at JP Morgan. That's it for me. Thank you. \>\> \[music\]