Talk by Erina Karati on Project Paradox, a framework for game agents with memory, emotions, and trust scores built with Arunachalam Manikandan at Supercell's AI Innovation Lab. In an AI agent village, a rumor about a mango sale degrades as it spreads — agents lose attribution and treat uncertainty as fact. The framework worked in short scenes but broke down over long horizons, so the team wrapped the agents in an autoresearch loop: controlled scenarios (public facts, rumors, plan changes) with collected traces, scored on a balanced scorecard covering reach, source retention, uncertainty preservation, replanning, and privacy. The loop can only modify a small frozen policy surface, and changes are kept only if scores improve. Broader lessons: memory alone isn't enough, facts need provenance, rollback is required, and long-horizon agents need experiments rather than prompt tweaks. # Channel Identity Observations — "Long-Horizon Agents Need Experiments, Not Just Prompts — Erina Karati" ## Creator Self-Description (Explicit Claims) - **Professional background claimed in the talk's opening:** *"Hi everyone, I'm Arena, former engineer at Microsoft and Supercell."* (Note: transcript self-identifies as "Arena" while the title credits "Erina Karati" — an identity discrepancy observable in the source.) - **Project attribution:** *"We developed project paradox at supercell's AI innovation lab"* with a named collaborator (*"Me and my teammate... natural money"* — likely a transcription artifact). - **Scope of project:** *"a modular AI framework that allows any developer to plug in intelligent autonomous agents within a video game"* — i.e., positioned as a general-purpose developer tool, not just an internal demo. ## Intended Audience (Observable Evidence) - **AI engineers and agent builders**, stated directly: *"the broader question is one I think many AI engineers are starting to run into. How do we evaluate and improve agents that carry state over a long period of time?"* - **Practitioners beyond gaming:** the talk explicitly generalizes to *"support agents," "personal assistants," "research agents," "coding agents," "workflow agents"* — indicating an audience building long-running stateful agent systems of any kind. - **Assumed technical fluency:** unprompted use of terms like *RAG, policy surface, guardrails, scenario suites, scorecards* suggests a professional/engineering audience, not a general consumer one. - **Conference-style closing inviting engagement:** *"please do connect with us. We'd love to talk if you have any questions."* ## Claimed Scope and Positioning - **Core thesis:** *"Long horizon agents need experiments and not just prompts"* — positioning the channel/talk around **evaluation methodology** (controlled scenarios, scorecards, rollback) rather than prompt engineering. - **Explicitly bounded claims:** the speaker self-limits scope: *"I kind of want to be careful about our claims here... I wouldn't say the system just generally improved"* — indicating a norm of hedged, evidence-based framing. - **Generalizability claim:** *"we definitely believe this is not only relevant for game agents"* — one game example offered as a stand-in for a broader engineering pattern. ## Creator Claims vs. What the Material Demonstrates | Creator claim | Demonstrated evidence | |---|---| | Project Paradox exists at Supercell's AI innovation lab | Demonstrated via described capabilities and an in-talk demo of character "Blossom" planning and responding in context | | RAG memory + emotion vectors + trust matrices | Described architecture; no external verification in source | | Auto-research loop improved the "mango rumor" failure | One anecdotal before/after (*"the agent was able to respond within context this time"*); creator themselves notes this is insufficient evidence | ## Boundary Conditions - Single-talk evidence only; no channel-wide patterns (upload cadence, recurring series) can be inferred. - No curated metadata was supplied beyond the title. - Transcript quality is degraded (e.g., "aent," "auto research"/"otter research"/"art research" for what appears to be an eval/research loop concept, likely referencing Karpathy's work as cited: *"Karpathy posted out auto research"*); quotes reflect the transcript as given. # Delivery Observation — Erina Karati, "Long-Horizon Agents Need Experiments, Not Just Prompts" ## Overall Delivery Profile The speaker delivers a technical conference talk in a **conversational, thinking-aloud register** rather than a polished scripted one. The delivery combines structured slide-driven explanation with frequent verbal restarts and self-corrections. --- ## Tone & Register - **Conversational and self-correcting.** The speaker repeatedly restarts or repairs sentences mid-flow, which creates a live, unscripted feel: > "Um yeah in this context uh oh yeah in this context art research is" > "And al together these agents make our multi- aentic framework. Um yeah yeah one second so the architecture was intentionally stateful behind this." - **Collaborative framing over authoritative claims.** Frequent use of hedged group-voice language: > "we believe," "we're trying to say," "I wouldn't say the system just generally improved," "a bit more systematically," "to some extent at least." - **Explicit epistemic caution.** Notably, the speaker calls attention to the limits of their own claims: > "This is also where I kind of want to be careful about our claims here." --- ## Recurring Phrases & Vocabulary Patterns observable across the transcript (several appear multiple times, others once but stylistically consistent): - **"So" as a pivot word** — used constantly to transition between points: > "So the question here became…" / "So at this point…" / "So again in brief…" - **"We believe" / "we're trying to"** — recurring hedge constructions. - **"Like" as an explanatory connector**, often introducing analogies: > "You can think of this as a trust matrix. Basically like after the interaction happens…" > "let's say you had dinner a few days ago, you probably wouldn't remember what you had for dinner, right?" - **"Right?" as a rapport-check tag question**: > "And of course, it wouldn't be complete if agents can't start conversations, right?" - **Domain vocabulary**: *policy surface, guardrails, traces, scenario suite, provenance, scorecard, ratchet, revert, harness* — consistent technical lexicon. - **"Um yeah" / "Uh"** filler words appear frequently at sentence boundaries. --- ## Teaching Habits 1. **Running example discipline.** A single concrete example — the **mango rumor** — is introduced, dropped, and deliberately returned to as proof: > "And talking about our Mango example again, after running one of our auto research loops…" 2. **Everyday analogies for technical mechanisms.** The memory-importance score is explained via a dinner-vs-murder comparison: > "if someone was murdered a few days ago, you definitely remember that." 3. **Enumeration as a structural habit.** Numbered and parallel lists organize nearly every explanation: > "First we define a control scenario… Then we run the simulation… Then we score this behavior…" > "For diffusion, you might measure reach… For provenance, you measure source retention… For rumors, you can measure uncertainty preservation…" 4. **Anticipated-question style**, posing questions then answering them: > "Did the information spread as we expected it to? Did the source attribution survive? … Did uncertainty stay uncertain?" 5. **Explicit "lesson" framing** near the end, turning the talk into transferable advice: > "And the biggest lesson for me perhaps was that memory is not enough here." 6. **Rhetorical contrast pairs** to sharpen the thesis: > "not by trusting one demo and not by endlessly hand-tuning prompts, but by running control experiments…" --- ## Communication Patterns - **Paired claim + caution**: nearly every strong statement is followed by a qualification ("it should not rewrite the whole application of course", "these exact scenarios are not universal here"). - **Negative-space teaching**: repeatedly defines concepts by what they are *not*: > "this is not only about improving rag retrieval" / "auto research is not another agent in the village" / "roll back also is not optional." - **Audience-facing closing ritual**: an invitation to connect plus a summarized takeaway, bookending the title claim: > "Long horizon agents need experiments and not just prompts. And I hope that's the takeaway that you get from this talk." --- ## Uncertainty Notes - **Self-corrections and verbal stumbles are frequent** ("aent", "sen a scenario suit", "auto/otter research" transcribed inconsistently). Whether this reflects habitual delivery style or transcript/transcription artifacts is **not fully determinable** from this single source. - The "So"-pivot and hedged-claim habits appear repeatedly enough to count as stable patterns; the analogy usage ("dinner vs murder") is observed only once and may be an isolated choice rather than a habit. # Long-Horizon Agents Need Experiments, Not Just Prompts — Position Record ## Speaker Identification - The transcript introduces the speaker as "Arena, former engineer at Microsoft and Supercell" (note: the video title attributes the talk to **Erina Karati**; the transcript itself uses "Arena" and also references a teammate "natural money" — these discrepancies are preserved as expressed). ## Core Thesis > "Long horizon agents need experiments and not just prompts." The speaker argues that evaluating and improving agents carrying state over long horizons requires an automated experimental loop (citing Karpathy's "auto research" post from "a few months ago" as inspiration), not manual prompt tuning or single demos. ## Project Paradox (Supercell AI Innovation Lab) **Claim:** A modular AI framework allowing developers to plug autonomous agents into video games as dynamic companions. **Stated agent capabilities:** - Move with intent, guided by memories, emotion, or curiosity - Interact with the world (pick up/drop objects, environment awareness) - React to events, which update beliefs and emotions "on the fly" - Initiate and hold conversations, stored in memory and affecting beliefs/goals **Architecture (intentionally stateful):** - **Per-agent memory**: separate RAG-backed namespaces; memory does not bleed between agents - **Emotion vector**: joy, sadness, fear, anger, disgust updated after events - **Belief/trust scores**: an LLM decides whether trust goes up, down, or stays unchanged - **Importance scoring**: LLM scores events; above-threshold memories stored in a separate cache (analogy: dinner vs. murder memory) ## The Problem Observed **Claim:** The architecture worked for short-term gameplay but "social consistency started to get weaker" over long horizons. Failure modes cited: - Rough topic remembered but source lost - Rumors hardening into facts stated as certain - Known facts failing to be recalled during action planning Example given: a mango-sale rumor spreading agent-to-agent was not retrievable in context when the player later asked about it. ## Proposed Solution: Auto Research Loop **Position:** Project Paradox is "a lab bench"; auto research is the experimental loop around it. Key framing: it is **not** another agent in the village but "a meta system outside the village." **Loop as described:** 1. Define a control scenario (e.g., public-fact learning, rumor hearing) 2. Run the simulation and collect structured traces (observations, conversations, memory writes, retrievals, belief updates) 3. Score behavior (spread, source attribution, uncertainty preservation, action-on-knowledge) 4. Propose a **small, constrained** change to the agent protocol / policy surface 5. Rerun; keep improvements only if the score improves **and guardrails hold**; otherwise revert **Key shift claimed:** "We were no longer evaluating one answer. We were evaluating an entire run." ## Scorecard Argument **Position:** Avoid a single vague metric ("agent quality") — it "will hide all the interesting failures." Instead use a balanced scorecard: - Diffusion → reach - Provenance → source retention - Rumors → uncertainty preservation, false-certainty rate - Planning → action consistency, time to replan - Privacy → containment **Stated rationale:** single-metric optimization creates bad behavior (e.g., diffusion-only → oversharing; recall-only → noisy/stale memories). The scorecard "keeps the system honest" and prevents gamification. ## Engineering Lessons (as stated) - **Keep the editable surface small.** Freeze the harness, scenarios, and metrics; expose only memory-writing policy, retrieval policy, communication prompt, trust rules, source attribution, replanning triggers, etc. This is "the difference between the LLM writing random patches versus the LLM actually searching within a controlled policy space." - **Controlled scenarios are necessary.** Free-roaming agents "might look cool" but are "very hard to evaluate." - **Memory is not enough.** Agents need provenance (firsthand/secondhand/verified/uncertain), separation of raw episodic memory from current belief, and scenario-based testing "not just through vibes." - **Rollback is not optional.** A change improving one dimension can damage another (e.g., faster fact spread may leak private info). The loop "should basically be like a ratchet." ## Explicit Qualifications and Hedges > "This is also where I kind of want to be careful about our claims here because... I wouldn't say the system just generally improved." - The speaker explicitly declines to claim general system improvement; the claim is limited to identifying **the right kind of surface** to expose to an auto research loop. - The specific example scenarios (fact diffusion, rumor uncertainty, replanning) are stated as **not universal**: "The point is not that these exact scenarios are universal here." - The exact scoring formula is "less important than the shape of the scorecard." - The mango example improvement is reported anecdotally ("we did find that the agent was able to respond within context this time") without repeated-run claims. - "Auto research gave us a way to approach this **a bit more** systematically" — modest framing of improvement. ## Generalization Claim **Position:** The pattern applies beyond game agents to any system maintaining state over time that affects future action: - Support agents (policy provenance/supersession) - Personal assistants (commitments, corrections) - Research agents (provenance, citations, contradiction handling, hypothesis updates) - Coding agents (long-running context across issues/files/teammates/requirements) - Workflow agents (access controls, handoffs, replanning) **Proposed recipe:** "Freeze the harness, define scenarios, log traces, score behavior, and expose only a small policy surface." ## Closing Framing The deeper problem of Project Paradox was "not animation or dialogue... it was the state" — who knows what, who told whom, what is true/uncertain/outdated, and whether agents act on what they remember. The real question: **"across controlled runs, does the system behave better?"** # Video Attribution Record ## Speakers & Roles | Speaker | Role | Notes | |---|---|---| | **Erina Karati** (title) / self-introduces as **"Arena"** | Speaker, presenter, solo | Transcript audio/self-intro says *"I'm Arena, former engineer at Microsoft and Supercell"* while the title names **Erina Karati**. Discrepancy flagged; treat as likely transcription variance, identity per title metadata. | | **Natural Money** (as heard in transcript) | Mentioned collaborator | Described as *"my teammate are natural money"* — likely a garbled name from transcription; flagged as uncertain, not confirmed as a speaker. | - **No guests or interviewers** appear; the video is a single-presenter conference-style talk. - Speaker role: **creator/practitioner** presenting work from **Supercell's AI Innovation Lab**. ## Speaker Background Claims (self-reported) - Former engineer at **Microsoft** and **Supercell**. - Co-developed **Project Paradox** at Supercell's AI Innovation Lab with the named teammate. ## Third-Party Sources Referenced - **Karpathy (Andrej Karpathy)**: *"few months ago, Karpathi posted out auto research"* — the speaker credits a Karpathi post as the inspiration for applying auto research to their system. No link, date, or title provided. ## Claims Attribution Summary - All technical claims (agent architecture, RAG memory, emotion vectors, trust matrix, mango rumor example, scenario suites, scorecards) originate **solely from the speaker**, describing their own project. - Speaker explicitly limits claims: *"without repeated current loop results... I wouldn't say the system just generally improved"* — a hedged claim about generalization. - Examples of agent capabilities (picnic/Blossom demo) are described verbally; no on-screen verifiable evidence is cited in the transcript. ## Promotional Segments - **Closing call-to-connect**: *"please do connect with us. We'd love to talk if you have any questions."* — mild self-promotion/networking invitation, not a product pitch. - No paid sponsorship, product advertisement, or third-party promotion detected. ## Comment Availability - Not stated in transcript or metadata. **Unknown.** ## Uncertainty Flags - Speaker name mismatch between title ("Erina Karati") and self-introduction ("Arena"). - Teammate's name ("natural money") is a probable transcription artifact; identity unconfirmed. - Karpathy reference has no verifiable citation details in the transcript. Hi everyone, I'm Arena, former engineer at Microsoft and Supercell. And today I want to talk about auto research in a multi- aent AI village. I will use a video game like AI Village as a running example here, but the broader question is one I think many AI engineers are starting to run into. How do we evaluate and improve agents that carry state over a long period of time? Before I get into the auto research layer, I want to talk a bit about project paradox. We developed project paradox at supercell's AI innovation lab. Me and my teammate are natural money. We built a modular AI framework that allows any developer to plug in intelligent autonomous agents within a video game that can interact, compete or cooperate with other players or agents as well and place them uh and make them into dynamic game companions. Now to give examples of what these agents can do, the agents can move with intent. They can go to any location or person and they're guided by their own memories, emotion, or curiosity. These agents can interact with the world. They can pick up objects, drop them anywhere, and they're also aware about the context in their own environment, such as the objects or other characters or agents as well. I would also like to note that game developers can also add new actions for these agents to accomplish within our framework as well. Instead of just dropping or uh placing objects, agents can also obviously react to what's happening around them. And these events that happen around them affect their own beliefs and emotions on the fly as well. And of course, it wouldn't be complete if agents can't start conversations, right? agents can in this scenario approach other agents or even the player as well and this makes the game feel more alive. And of course these conversations are stored within their memory and is according to their own um and affect their own emotions and beliefs or goals as well. And al together these agents make our multi- aentic framework. Um yeah yeah one second so the architecture was intentionally stateful behind this. The first important part was per agent memory. Each agent has its own memory namespace backed by rag. So memory did not bleed between agents. Second, we tracked emotion as a small vector. So after an event or conversation, the system could update values like joy, sadness, fear, anger, or disgust. Third, agents had belief scores towards other agents and the player. You can think of this as a trust matrix. Basically like after the interaction happens, the LM basically decides whether the trust score should go up, down, or whether it shouldn't change at all. And fourth, every memory receives an important score. Um to to explain this better, like let's say you had dinner a few days ago, you probably wouldn't remember what you had for dinner, right? But um if someone was murdered a few days ago, you definitely remember that. So the agent will evaluate or the LM will evaluate uh an important score of an event and if it crosses a threshold, it will store that specific memory uh in a separate cache so that important context can be retrieved better later on. And here's an example of it just working. Um, we going to ask one of the characters to go on a picnic with us. Here, uh, our character Blossom um, decides to pick up a pastry and go to the picnic area because we asked her to do so. Keep in mind during the conversation in the background, she plans all of these sequence of actions to accomplish. And one when we talk to her afterwards, she will also reply within context as well. Yeah. But this is where an interesting problem actually started. As you saw in the last example, like for shortterm game play, this our architecture worked pretty well. like a character could make a plan, move around, talk and remember the recent interaction and respond to us or other characters as well. But over longer horizons, this is where we notice the social consistency start to get weaker. So in this example, we have one agent spreading a rumor about a sale on mangoes to another agent and that agent receives that information and goes and tells another agent about it. Later on, after a number of events that occurred in between, when the player asks one of the agents about the mangoes, it doesn't exactly store that context that we were expecting or it doesn't give us the context that we kind of wanted to. And this is where things are starting to get messy naturally. Like the system may remember the rough topic but lose the source of the topic. A rumor may become certain instead of just a rumor like the agent might state it as a fact or um an agent might know a fact but fail to execute fail to remember it while creating a plan for its actions. So the question here became how do we improve a multi-agentic system over longunning social behavior and not just over one response. And this is where we wanted to bring in auto research. As you all know, uh few months ago, Karpathi posted out auto research and this this made us immediately very curious. Uh perhaps we can make the system run experiments uh on itself and can we use this for our system as well. So what we understood is instead of manually tuning a prompt or watching one nice demo, we could define sen a scenario suit, run the agents, collect traces, score the behavior and change a small policy surface and only keep the changes that actually improve the score. And this is where we're trying to bridge project paradox with auto research. So at this point basically our multi-agentic framework project paradox is more like a lab bench and auto research becomes the experimental loop around it. And importantly this is not only about improving rag retrieval. The broader framing is optimizing the agent protocol like how do agents write memories, retrieve them, communicate uncertainty, update trust attribute sources and replan around new facts. basically um yeah in this context uh oh yeah in this context art research is not another agent in the village like I said it's a meta system outside the village the villagers have local perspectives of course they only know what they saw heard remembered or inferred because there isn't a common memory database in between them. Information only travels once uh other agents communicate them properly. The auto research layer has a different job here. It reads the full traces of a run, compares what happens against the scenario ground truth, uh scores the behavior and proposes a constrained change to the agent protocol or cognitive policy. Then it reruns the scenario and asks society level behavior like did society level behavior get better. This is the key shift we were trying to look for. So we were no longer evaluating one answer. We were evaluating an entire run. And this is what one of the loops would look like. Like first we define a control scenario which I'll elaborate a bit more about later. For example, one agent learns a public fact or one agent hears a rumor. Uh that could be a controlled scenario. Then we run the simulation. During the run, we collect structured traces, observations, conversations, memory rights, retrievals, belief updates, whatever is relevant to us in that case, we collect. Then we score this behavior. Did the information spread as we expected it to? Did the source attribution survive? Such as, does the agent remember who started the rumor? Did uncertainty stay uncertain? Did agents act on what they actually knew? And then the auto research layer here proposes a small policy change. And this is important. It should not rewrite the whole application of course. it should only edit a controlled policy surface and then we rerun. If the score improves and the guardrails hold, we we keep the improvement and if not we simply just revert back. And talking about controlled scenarios, the reason why uh scenario design matters is that social behavior is otherwise a bit fuzzy. uh in general in the sense if you just let the agents in our environment wander around it might look cool and you might get nice interactions but it's actually very hard to evaluate on whether the system actually improved. So this is why we believe you need controlled scenarios. For example, one scenario could test a public fact diffusion. Let's say agent A learns uh the bakery will close tomorrow. Do the right agents learn it? Do they remember who said what? Do they rem do they change their plans based on this fact? Another scenario could test rumor uncertainty. Agent let's say agent A hears that agent C might leave the village. When this rumor spreads, does might leave suddenly become is leaving or does it stay as might leave? Like does it become a fact or does it still stay as a a rumor? Another scenario could test replanning. The group has a plan, but one agent learns, let's say, the route they wanted to take is blocked. Do agents update this and communicate this uh with each other to avoid uh a improper plan or scale actions? The point is not that these exact scenarios are universal here. The point we're trying to make is that long horizon agent behavior needs scenario suits. And talking about our Mango example again, after running one of our auto research loops, this time after uh a a long pro period of time, when the player finally asked one of the agents about the sale on mangoes, we did find that u the the agent was able to respond within context this time like compared to last time. Um yeah and for this talk the form the exact formula we believe is less important than the shape of the scorecard. Uh you do not want a single vague met metric like agent quality. This will hide all the interesting failures. Instead you want a balanced scorecard. For diffusion, you might measure reach like how many agents know the fact after end steps. For provenence, you measure source retention among agents who know it. How many remember it, where it came from, etc. For rumors, you can measure uncerny preservation and false surn rate. For planning, you can measure action consistency and time to replan. And for privacy, you can measure containment. This matters because optimizing only one metric can create bad behavior. Because let's say if you only optimize for diffusion, the agents may learn to overshare everything. And let's say if you only optimize for memory recall, you might create noisy or steal um like memories. So this scorecard is what keeps the system honest and prevents the auto research agent from gamifying the system to just increase one specific score. The other important engineering lesson that we learned over this project is that um it's important to keep the editable surface really small. The auto research layer should not have permission to randomly rewrite the whole codebase. Instead, it's really important to freeze the harness, the scenarios, and the metrics. So, we're only exposing the part of the system that we actually want to optimize. Here in project paradox for us that meant things like memory writing policy, retrieval policy, communication prompt, belief, trust rules, source attribution, replanning triggers, etc. This gives the search pro process room to improve behavior, but also prevents it from gaming the evaluation directly as we mentioned before. And this is the difference between the LM writing random patches versus the LM actually searching within a controlled policy space. And here are here are examples of the kind of changes I want this kind of loop to search over. If if source attribution disappears, the policy change might be preserve source in memory and uh write uh memory rights and summaries. If rumors harden into facts, the policy change might be stored confidence mark firstand versus secondhand and require hedging when retelling uncertain claims. If if facts if public facts stay local, the policy change might be classify useful public facts differently and make agents proactively share important source evidence. The key is that these are small changes to the agent protocol, but they can have larger effects on a society level behavior for multi-agentic systems. This is also where I kind of want to be careful about our claims here because with we believe without repeated current loop results like I wouldn't say the system just generally improved. We're trying to say this is the right kind of surface to expose to an auto research layer uh loop because it is small enough to control but it's still rich enough to change the social behavior to some extent at least. And the biggest lesson for me perhaps was that memory is not enough here. You can add a rag memory to an agent and still not get the current long-term uh horizon behavior that you were looking for. Um because agents need to sometimes know where that information came for uh came from. You need to preserve whether it was firsthand, secondhand, verified or uncertain. Sometimes you need to separate raw episodic memories from what the agent currently believes too. And you need to test behavior through scenarios not not just through vibes. So the other lesson is that uh roll back also is not optional. When you optimize social behavior, a change can improve one thing and damage another. So, a policy that spreads public facts uh faster might also leak private information. A policy that increases recall might increase stale memory usage. So, the loop should basically be like a ratchet. Try a change, score it, keep it only if the scorecard improves and guard rails whole. And we we definitely believe this is not only relevant for game agents cuz although I gave you an example using a game village um we believe like let's say for example support agents support agents need to know which policy update comes from where right and whether it supersedes an older answer. Personal assistants for example need to remember commitments that they previously made and h make corrections if uh if the user u wants to change those personal commitments. Research agents need pro uh provenence citations contradiction handling and hypothesis updates. Coding agents need longunning context across issues, files, teammates and changing requirements. Workflow agents need access controls, handoffs, and replplanning when the world changes. All of these systems have the same underlying problem. They maintain state over time. And that state affect affects future action. So they need control scenarios and behavioral scorecards is what we are proposing. So again in brief, a recipe for long horizon agents. If there's one practical recipe I want you to take away, freeze the harness, define scenarios, log traces, score behavior, and expose only a small policy surface. Search over these changes, keep only changes that survive your measurement. And this is an engineering pattern that we believe would uh make sense for longunning agents. The real question we believe is across controlled runs, does the system behave better? To close, project paradox start as an attempt to make game agents feel alive in a 3D world. But the deeper engineing problem was not animation or dialogue for us. It was the state such as which agent knows what, which agent told whom, what is true, uncertain or outdated, and do agents act on what they remember. Otter research. Otter research gave us a way to approach this a bit more systematically. Not by trusting one demo and not by endlessly handtuning prompts, but by running control experiments and keeping only the changes that survived our measurement. Long horizon agents need experiments and not just prompts. And I hope that's the takeaway that you get from this talk. And yes, please do connect with us. We'd love to talk if you have any questions. Thank you so much for listening. Yeah.