Talk by Yohei Nakajima (creator of BabyAGI) on ActiveGraph, an agent runtime built around an immutable event log rather than the LLM. In this architecture, every action and state change flattens into one typed log, which projects a graph serving as the agent's state, enabling replays, rollbacks, and forks — demonstrated when an interrupted 350-question eval resumed at question 353 after an API key failure. The talk covers behaviors that react to graph changes, policies governing what the agent may modify on its own, and swappable packs for memory, tools, and chat. Nakajima draws on 1970s blackboard systems and Kafka for the design where LLMs communicate only through shared state. He also presents a self-improvement loop that forks the agent, proposes patches, gates them behind sandbox tests, and keeps changes only if accuracy improves, plus ActiveGraph Lab, which reads his blog posts, runs its own experiments, and once opened a PR fixing a bug it found in its own code. # Channel Presentation Observations: Active Graph Agent Runtime (BabyAGI 4) ## Speaker / Channel Identity Evidence - The presenter **explicitly self-identifies** as **Yohei Nakajima** of **Untapped Capital** (also stated in the video title). - Self-description quote: *"I'm Yohei, VC by day, builder by night."* - He claims dual roles: - **Investor**: *"I've actually gotten to invest in a good number of agentic companies... through my funds Untapped Capital and I also have an agent fund."* - **Builder**: Creator of BabyAGI and follow-on open-source experiments. ## Explicit Self-Description of Work / Scope - Claims a **3-year research theme**: *"Let's build the simplest thing that can build itself has basically been kind of my research theme for the last 3 years since I did baby AGI back in March of 2023."* - Claims **nine iterations** of BabyAGI: *"over the course of 3 years I've done nine iterations of baby AGI with less fanfare."* - Points to a public wiki as evidence trail: *"if you go to baby AGI wiki, you can see earlier experiments."* - Presents **ActiveGraph** as *"my new open-source experimental approach to building agents"* — explicitly framed as **experimental and inspirational**, not production guidance: *"it's definitely experimental. The idea is more to give you... inspire you with some potentially new ideas."* - References published work: *"I have a paper that was my first archive paper called the log is the agent"* and a second paper (the "Regimes" project). - Claims everything shared has public artifacts: *"everything I've shared has a blog, a GitHub repo."* ## Intended Audience (Demonstrated) - The talk is aimed at a **technical AI engineer / agent-builder audience**, evidenced by: - Opening: *"AI engineer warfare has been so fun meeting everybody"* (apparently a conference for AI engineers). - Assumes familiarity with ReAct agents, LangSmith, Claude Code, Replit, blackboard architecture, Kafka, embeddings, and long-context evals. - Direct invitation to experiment: *"I'd love for you to try it... have it explain if it was helpful or not. Let me know if you try it. Or hate it."* - Audience addressed as peers who build agents themselves: *"looks a little bit different than maybe you've been building agents."* ## Claimed Scope of Demonstrated Expertise - **Agent architecture**: contrasting "LLM-centric" vs. "log-centric" agent construction. - **Event-sourced runtimes**: *"ActiveGraph is an event-sourced graph runtime for building auditable agents."* Explicitly distinguished from a harness: *"this is not a harness. It's a runtime."* - **Self-improvement loops**: controlled self-modification experiments with acceptance gates ("Regimes"). - **Benchmarks**: experiments on LongMemEval. - **Honest epistemic caveats** (demonstrated, not just claimed): - *"I don't actually know how to train models or anything. So I'll caveat that."* - *"Candidly, I'm not the one writing the code."* - *"it might not make sense if you don't play Pokémon"* — acknowledges tangents. - Acknowledges BabyAGI's failure: *"People thought it was going to work. It didn't work at all."* ## Observable Presentation Patterns - **Show-and-tell structure**: concept explanation → code concepts (behaviors, policies, views, packs) → experiments → speculative hypotheses. - **Concrete experimental evidence** cited for claims (recovery from API key failure at question ~350 of LongMemEval; Pokémon Kaggle competition with ~80 passes, ~20–30 accepted changes, "27%" score). - **Speculation clearly flagged as hypothesis**: the "experiential world model" and hippocampus analogy introduced with *"Here's where I might lose some serious researchers"* and framed as *"kind of the new hypothesis that's growing."* - **Open-source, community-invitation posture**: repeated calls to try the project and report back, with reference to documentation (*"all this is in the documentation"*). ## Summary of Observable Positioning The channel/video presents a **builder-VC hybrid** speaking to **practicing AI engineers**, claiming scope around **experimental, open-source autonomous-agent research** with an emphasis on **self-improving agents and auditability**, consistently framing work as experimental, caveating speculative claims, and inviting community replication and feedback. # Delivery Observations — Yohei Nakajima, "Active Graph Agent Runtime (BabyAGI 4)" ## Tone & Overall Style - **Casual, self-deprecating, and conversational** throughout. The talk opens warmly ("Hi everybody. Thanks for coming. I'm excited to be here") and mixes technical depth with informal asides. - **Hedged confidence**: he repeatedly qualifies claims as personal or preliminary — "It's definitely experimental," "It's just my personal experience," "some people you'll have to try it yourself to see." ## Recurring Phrases & Verbal Habits - **"Candidly"** as a repeated disclosure marker (at least two uses): > "Candidly, I'm not the one writing the code, but AI seems to be pretty good at figuring out how to do it." > "Was able to do it, but candidly, um I could put more effort into try to increase that..." - **"It just worked / just worked / just came natively"** framing for surprises: > "I didn't have to think about it. I just had to ask my coding agent to use ActiveGraph and this event log and graph came natively." - Frequent **"um," "uh," "you know," "right?"** fillers, plus filler pairs like "and and," "it it," suggesting a loosely scripted, improvised delivery. - Repetition of the phrase **"event log"** and "typed event log" as his anchor concept: > "you get this beautiful typed event log, which gives you replays. It gives you rollbacks and it gives you forks." - The adjective **"beautiful"** recurs for describing outputs ("beautiful typed event log," "beautiful report," "this is what it ended up looking like... beautiful"). ## Teaching Habits & Structural Patterns - **Old-way vs. new-way framing**: > "So, the old way, you got the while not done if loop. The new way, or at least my new way..." > "So today most people build agents around the LLM... but ActiveGraph asks, what if you build around the log?" - **Self-correction on the fly**, made explicit to the audience: > "Oh, actually, well, shoot, I I jumped ahead." - **Deferring detail to documentation**, repeatedly: > "I'm I'm flying through some of this cuz I want to get to the fun parts and all this is in the documentation." > "I'll come back to it... I'll show you some code you code examples." - **Concrete experiment narratives** as evidence: the long mem eval API-key failure, the Kaggle Pokémon competition, ActiveGraph Lab finding its own bug — each told as a short story with outcomes. - **Humor via understatement and self-deprecation**: > "It didn't work at all." [laughter] > "As you can see, I'm all over the place." > "You might recognize this face more than this face." - **Analogies to established systems** to legitimize a novel idea: blackboard architecture, Kafka, and the hippocampus. ## Handling of Uncertainty - He **explicitly flags limits of his expertise**: > "Here's where I might lose some serious researchers, at least in the language I use. I don't actually know how to train models or anything. So I'll caveat that." - Speculation is labeled as hypothesis: "my hypothesis is that that's in the training data," "is is kind of the new hypothesis that's growing." ## Closing Pattern - Ends with a **low-pressure call to action** inviting the audience to test and report back, including negative results: > "Let me know if you try it. Or hate it. Or building something relevant. Thanks for listening." ## Uncertainty Notes - Fillers and restarts are pervasive and stable across the transcript; this is clearly characteristic, not isolated. - Some patterns ("beautiful," "candidly") appear only 2–3 times — recurring but possibly situational rather than signature habits. # Active Graph Agent Runtime (BabyAGI 4) — Position Record **Speaker:** Yohei Nakajima, Untapped Capital (self-introduced in transcript: "VC by day, builder by night"; creator of BabyAGI, March 2023) ## Core Thesis: Log-Centric vs. LLM-Centric Agents - Most agents today are built **around the LLM** (model → response API → tools → memory → logging). - **ActiveGraph inverts this: build around the log.** Nakajima describes it as an "event-sourced graph runtime for building auditable agents," referencing his first arXiv paper, *"The Log is the Agent."* - The log captures not just what the agent does, but **every change to the agent** — typically tracked separately, here flattened into a single **immutable event log** that serves as ground truth. - The log **projects a graph** (the agent's state); **behaviors** react to graph changes and emit events, which update state and may trigger further behaviors. - **LLMs don't talk to each other** in ActiveGraph — all communication goes through shared state. ## Key Concepts as Presented - **Policies:** rules governing how the graph can be modified (e.g., adding a source article freely; changing a prompt requiring human-in-the-loop; changing a fact requiring a contradiction check). - **Views / context management:** context is a graph query — "grab a subset of the graph" made available to a behavior. Nakajima calls this "really elegant." - **Packs:** bundles of object schemas, tools, behaviors, and a pack policy; assembled to build harnesses. Modular and swappable (e.g., replace one memory pack with another). - **Positioning:** ActiveGraph is "not a harness. It's a runtime" — common harnesses (e.g., ReAct) can be rebuilt on top; he shows a ReAct agent implemented as non-communicating behaviors. - **Architectural lineage:** inspired by **blackboard architecture** ('70s–'80s) and more recently **Kafka**-style micro-workers sharing state. Old challenge: unintuitive to write, thin deterministic workers. Now: "AI writes the code, and the workers can be very powerful." ## Honest Qualifications and Contradictions (Preserved) - On BabyAGI's original launch: "People thought it was going to work. **It didn't work at all.**" - ActiveGraph is **experimental** — "the idea is more to give you… inspire you with some potentially new ideas." - **"I would never write code myself with ActoGraph, but again, AI seems really good at it."** He notes it is "actually pretty unintuitive." - On context management: "Candidly, I'm not the one writing the code." - Memory experiment caveats: semantic ingestion improved scores, but he "jumped onto the next experiment" rather than optimizing. - On his speculations: "Here's where I might lose some serious researchers… I don't actually know how to train models or anything. So I'll caveat that." - Kaggle results remain modest: "I'm still at like 27%, so I don't know if I can get higher than that." ## Experiments and Claimed Results ### Log as Memory - Not pure vector RAG: leverages structured log ordering plus embeddings of messages. - On LongMemEval: no semantic ingestion, fact extraction, or entity extraction — only embedded queries plus surrounding messages; "it actually did pretty well." - **Surprise finding:** mid-run API key failure at question ~350; after key update, the system rolled back one step and resumed from question 353 — "that just has not been my experience" with prior agents. ### Reference Agents - Asked Replit's coding agent to build on ActiveGraph; event log + graph "came natively" without him designing for it. - Same for a deep research agent, which produced evidence-provenance and contradiction graphs. He concedes this is possible with tools like LangSmith, but required no thought. ### Regimes (Paper 2) — Controlled Self-Modification - Failure-classified agent allowed to edit only specific parts of itself. - Loop: ~20 questions → identify failures → self-modify → test on 50 questions → accept only if accuracy increased. - Process: propose patch → fork → static gate check → sandbox gate check → verify impact → accept. - Looped 8–13 times; accepted only 4–5 patches; "modest, but… statistically significant improvement" on LongMemEval. Agent also "knew what didn't work." ### ActiveGraph Lab - Reads all his blog posts and GitHub repos, proposes ideas, asks permission, runs experiments, writes blog posts. - Found an error in its own code, wrote a PR, which Nakajima merged. - Discovered packs are modular across repos — a capability Nakajima says he didn't know about. ### Kaggle Pokémon Deck Competition - Deterministic decks in an Elo-style competition; ~80 improvement passes using Claude Code and Replit. - Agent proposed gating changes on ~200 simulated games vs. three reference agents with win-rate and Wilson-score thresholds. - ~20–30 of 80 passes accepted; score "slowly improved." - Most interesting outcome: the agent retained knowledge of **what didn't work**, unlike his prior "YOLO agents" — attributable to the acceptance policy. ## Stated Hypotheses (Tentative, Speaker-Flagged) - **Why AI is good at architecting these systems:** decades of blackboard/Kafka/micro-worker patterns exist in training data; LLM-agent patterns are only ~3 years old. "My hypothesis is that that's in the training data." - **Experiential world model:** long-running agents may need, beyond a predictive world model (priors), an **experiential world model**. Analogy: the hippocampus functions "like an immutable state event log" feeding back into priors through replay, dreaming, and sleep. - **Against the "harness disappears" view:** as models improve, some suggest harnesses vanish; Nakajima is "starting to think that's not true. I think we need both." - **Agent identity:** humans are defined less by reasoning capability than by beliefs, knowledge, and behaviors from lived experience; analogously, "maybe that identity of the agent is derived from its own log." ## Additional Facts Stated - Background: 3 years since BabyAGI (March 2023), nine iterations with less fanfare; earlier graph projects (Instagraph, Mindgraph — "pre-graph-rag rag", code/function/log graphs); investor in agentic companies via Untapped Capital and a separate agent fund. ## Closing Invitation - Asks listeners to try ActiveGraph with their own coding agent and report whether it helped. # Speaker & Provenance Observation Record ## Video **Title:** Active Graph Agent Runtime (BabyAGI 4) — Yohei Nakajima, Untapped Capital **ID:** `01a15ca8-67f1-7cf2-a2fb-cf5a72c8eb30` ## Speakers & Roles - **Yohei Nakajima** — sole identified speaker; **creator/presenter** and primary content source. - Self-identified roles in transcript: - Creator of **BabyAGI** (March 2023) and nine subsequent iterations - **VC** ("VC by day, builder by night") — General partner, **Untapped Capital**; also runs an "agent fund" - Author of an arXiv paper, *"The Log is the Agent"*, and a second paper ("Regimes" project) - Creator of prior projects: **Instagraph**, **Mindgraph**, code/function/log graphs - **Audience** — unnamed; audible reactions only (**[laughter]**, multiple **[snorts]**); no speaking role. ## Referenced Sources & Third-Party Material - **arXiv paper:** *"The Log is the Agent"* (speaker's first arXiv paper) - **Second paper:** Regimes project (controlled self-modification) - **babyagi.wiki** — cited for earlier experiments - **Benchmarks:** LongMemEval (long mem eval); Kaggle Pokémon trading card competition - **Named tools/platforms:** Replit, Claude Code ("no cloud code" — possible transcription artifact), LangSmith, Kafka, blackboard architecture ('70s–'80s), React agent, OpenClaw/Hermes (as "open claw or Hermes") - **Neuroscience reference:** hippocampus analogy (speaker's own framing, no cited source) ## Self-Reported Claims (uncorroborated) - LongMemEval results: query embedding + local message context "did pretty well"; Regimes self-modification produced "modest but statistically significant improvement" (~8–13 loops, 4–5 accepted patches) - Pokémon Kaggle: ~80 experiment passes, 20–30 accepted, ~27% score - ActiveGraph Lab found a bug in its own code, wrote a PR the speaker merged ## Promotional Segments - **None flagged.** The talk promotes the speaker's own **open-source project (ActiveGraph)** and closes with an invitation to try it, but no third-party commercial promotion is present. ## Availability & Limitations - No curated metadata supplied beyond title. - **Comment availability:** not indicated. - **Uncertainty notes:** transcript contains apparent speech-to-text artifacts (e.g., "ActoGraph"/"Auto graph" for ActiveGraph, "no cloud code"); no timestamps present; all quantitative results are speaker self-reports without external verification. \>\> Hi everybody. Thanks for coming. I'm excited to be here. AI engineer warfare has been so fun meeting everybody. Um, but I'm here to talk about active graph, which is my new open-source experimental approach to building agents, which looks a little bit different than maybe you've been building agents. Uh, it's definitely experimental. The idea is more to give you inspire you with some potentially new ideas. Um, agents are awesome, but long-running agents break. And if they're so awesome, why am I still building them? Why they should build themselves. Let's build the simplest thing that can build itself has basically been kind of my research theme for the last 3 years since I did baby AGI back in March of 2023. So that's over 3 years ago. If you were there at the time, it was crazy. It went wild like it was covered by media. People thought it was going to work. It didn't work at all. Um, \>\> \[laughter\] \>\> Uh, over the course of 3 years I've done nine iterations of baby AGI with less fanfare, but you know, every time just experimenting on like how do we get autonomous agents to actually work? Usually with the theme of self-improvement. Uh, if you go to baby AGI wiki, you can see earlier experiments. Um, in the in this process I kept coming up to coming back to graphs and I've had a couple of projects. Earlier I did a one called Instagraph and Mindgraph. That was like pre-graph rag rag. I did some code graphs, function graphs, log graphs. And since then it seems like a lot of people have started using graphs to build agents. Um, and so and in addition to that I've actually gotten to invest in a in a good number of, you know, agentic companies, some of which you I'm sure you'll recognize through my funds Untapped Capital and I also have an agent fund. Uh, but yeah, that's that's me. Yohei, VC by day, builder by night. You might recognize this face more than this face. Um, ActiveGraph is an event-sourced graph runtime for building auditable agents. I have a paper that was my first archive paper called the log is the agent, but I'm here to explain it. So today most people build agents around the LLM. You start with the LLM, you add a response API, you give it tools, you add memory, and then you make sure you log everything correctly, which can give you you know all the benefits that ActiveGraph will give you, but ActiveGraph asks, what if you build around the log? Now what does that mean? Um means not everything the agent does, but more importantly, every change to the agent, right? Nobody here is using the same agent they were using a year ago, and the agent you're going to use a year from now is going to be different. And a lot of people, what the agent does and how the agent changes are tracked in two different places, but I'm saying let's flatten that down into a single immutable event log, and this is the ground truth of the agent. And this projects a sort of graph. This is the state of the agent. And what I mean by that is, for example, a prompt can be edited multiple times, but you might have, you know, a master prompt that gets used when you're, you know, when you query the graph. And then on top of this, you attach something that I'm calling behaviors. Behaviors are reacts to graph changes. And then they emit events, which then in turn updates the state of the agent, which might trigger new behaviors. Um LLMs don't talk to each other in ActiveGraph. They all communicate through this shared state, and that's what makes it a little bit different. Behaviors can be deterministic, or they can include LLMs, which is which is how you build this agent. And you get this beautiful typed event log, uh that's the source of truth about everything the agent did and everything every change that's happened, which means Oh, actually, well, shoot, I I jumped ahead. So in addition to that, there's a concept called policies, which determine how the graph can be modified. I'll come back to it, but for example, things like a source article that you found in research, you might be fine with adding, but if you're changing a prompt, maybe you want human in the loop. Or if you're changing a fact, you might want to make sure there's no contradicting facts. So there's these thing called policies and again, I'll show you some code you code examples. But yeah, in the end you get this beautiful typed event log, which gives you replays. It gives you rollbacks and it gives you forks. And this becomes natively when you start building agents with ActiveGraph. So this is the kind of LLM-centric versus log-centric way of building agents that I'm going to be talking about or showing you code for. And and again, I'll specify that this is not a harness. It's it's a runtime and you can actually rebuild most of the common harnesses on top of it. You're just forcing every single communication to communicate through the shared state. \>\> \[snorts\] \>\> So at the highest level, right? When you're building with agents, messages feels like the kind of core unit that you're seeing often, but we're going to replace that with logs. Logs are going to be the core unit you're going to build around. That's what you're going to be reading, that's what you're going to be looking at. \>\> \[snorts\] \>\> These are typed logs, again, immutable, clean. You can't edit the graph. These are just kind of basic rules. Just emit events. You can have the add object, which is an event emitting. But yes, you can query over the graph. And again, I'm I'm flying through some of this cuz I want to get to the fun parts and all this is in the documentation. Behaviors listen to graph changes and emit events. So this is a behavior called a planner that triggers on a goal created, right? And then it adds an object to adds two task objects and a relationship object, a research, write memo, two tasks. And then actually behaviors can actually live on edges as a something called relation behavior. This one has an unblock relation. So basically, when the research is done, you can write the memo, right? Um And behavior subscriptions can be pretty complex. They can be graph queries. So this one says on object created, if the if the object type is a claim, and this claim contradicts another claim, we're going to trigger the contradiction detector. And these can be more complex than this. It can have, you know, uh confidence percentages baked into it um and and then on top of that, there's another concept called views. Uh context management can be done programmatically as basically a graph query. It's a It's You basically grab a subset of the graph, which makes it available to that behavior. You can still do other types of context uh context management, but I felt like this kind of graph query as context management just felt really elegant. Candidly, I'm not the one writing the code, but AI seems to be pretty good at figuring out how to do it. Um and you know, I earlier I talked about policies, so some graph changes require a proposed patch before approval. Again, this is how you these these policies kind of give it the control on what it's allowed to change by itself, what uh what kind of changes require certain tests, um and I'll give a few examples in a bit, um or if you want human in the loop, right? And you have these kind of policies that that determine or define uh what the uh what these rules are. And when you bring it all together, you got these kind of object schemas, tools, deterministic LLM behaviors can be assembled into a something called a pack, right? With a pack policy. And that's how you build Harness on top of ActoGraph. These And all of this together is and I'll have a couple examples later um are are modular and they can be combined, uh but you're not just adding skills. It is much more complex about actually pretty unintuitive. I would never write code myself with ActoGraph, but again, AI seems really good at it. Um and just on like event types are fixed. You can add custom events, objects are user-defined. I only added that cuz someone asked me that question when I was showing them these slides, but think that makes sense. So, the old way, you got the while not done if loop. The new way, or at least my new way, uh you have a whole bunch of behaviors that don't talk to each other, that just monitor the state. So, it's it's it's inspired by uh blackboard architecture from the '70s or '80s or more recently Kafka, whole bunch of micro workers communicating through a shared state. One of the challenges, at least back when uh blackboard was was that it was really unintuitive to write, and the workers were very slim and deterministic. But now, AI writes the code, and the workers can be very powerful because they have reasoning capability. This is a React agent on Auto graph. React agent was one of the earlier agent kind of architectures. As you can see, it's it look it actually works the same way, but on goal created, you add a thought. On thought created, you trigger the reason function. So again, this is just to show that you can build any harness on top of Auto graph. It does look different um because they're not communicating with each other. \>\> \[snorts\] \>\> And so, to see if how well this can work, I've been running a lot of experiments. Um the first one I did was can I use the log itself as memory? So this is not pure vector rag. It's actually leveraging the structured log, so it knows which which message was before what, plus uh vec uh plus embedding the actual messages within the log. Uh I did this on long mem eval. I embedded the query. There was no semantic uh ingestion, no fact extraction, no entity extraction, but I just embedded the query, looked for relevant messages, grabbed a couple messages before and after, made sure it fit into the context, and it actually did pretty well on long mem eval, right? Like a lot of the data in your memory is actually overlaps with the memory uh the data in your log. Actually having them the same actually kind of makes sense and makes sure they don't separate. Um I did try another couple other experiments on adding kind of semantic ingestion to improve the score. Was able to do it, but candidly, um I could put more effort into try to increase that, but I jumped onto the next experiment. Actually, but in that process, one of the biggest fun surprises was, I don't know if you've run long mem eval, but you have to like 500 questions. And then one of the runs, uh my API key ran out like 350. I was like, "Oh Okay." So I updated the API key, said, "Okay, let's let's rerun it again." And it just like rolled back one and was like, "All right, we'll just start from, you know, question number 353." And I don't know if you have, but like I've built a lot of agents that like where the API key did broke or something, and I had to rerun the long agent from the beginning. and that just has not been my experience since building with ActiveGraph, uh which was a very fun surprise. Uh I built a couple reference agents, uh right? Like now I have this runtime, let's see what I can build. I asked Replit to build a coding agent on top of ActiveGraph and and as a result it came with, you know, a event log graph. And again, I'll say this, you can do this with, you know, things like LangSmith. Thing is, I don't know I didn't have to think about it. I just had to ask my coding agent to use ActiveGraph and this event log and graph came natively. Same thing with research agent. I just had just build a deep research agent on ActiveGraph and it came with this beautiful, you know, had a event event log and a graph of like where the evidence came from, what contradict to each other. And again, I didn't have to think about it. I just asked my agent to build a research agent on top of ActiveGraph and this is what it ended up looking like. \>\> \[snorts\] \>\> Um and then I realized, okay, how do we get this closer to like an open claw or Hermes? So I I played around, I have this thing called uh ActiveGraph packs, which is a core pack, a tool pack, a secret pack, a memory pack, an identity pack, a communication pack, a chat pack. So now you kind of get the idea of how I'm trying to build agents on top of ActiveGraph. And each of these packs have object types and behaviors. Again, feel probably feels very different from the way you're building agents, but I I I feel like it's actually pretty elegant. You can just take a memory pack and replace it with another another memory pack, right? Um uh uh and and it's not like skills, but like the objects and the rules and all of them are uh bundled as a pack. And you and you attach these packs together to create your agent. Um and then I jumped into the the what I really wanted to do was start playing with self-improvement loops. Um I did a project called Regimes. I did uh uh that was in paper number two. Um This one you was a really controlled self-modification. Uh Claude Code called it Regime de Scene, but basically we classified the type of failure and then based on the failure it classified it was a it was allowed to edit a specific part of the agent. Again, I did this on long mem eval. The loop was I think doing about 20 questions, looking at the answering questions, seeing where it failed, trying to self-modify, trying that on 50 different questions, see if the accuracy actually went up, and only if it went up, it would accept it. So, that was kind of the proposal patch that started happening, where it would do after it proposed change to itself, right? This is essentially the agent forking itself, proposing a change, doing a static gate check, a sandbox gate check, and then making sure it actually impacted the result, and only then accepted a change. And for these loops, it would loop like eight or 13 times, but only accept four or five of those patches. And it actually did have, you know, modest, but like statistically significant improvement on long mem eval scores. Uh and and not only did it know what worked, but it also knew what didn't work. Um at this point I was like, "Okay, can I can I get active graph to just research active graph for me?" So, I built lab uh active graph lab, which is reading all the blog posts. So, everything I've shared has a blog, a GitHub repo, and this lab is reading all of that to come up with new ideas, asking me if it can run it. And if I say yes, it'll run the experiment, um and then it'll write a blog post about it. It actually did find an error in its own code, asked me if it could it could if it could fix it. It wrote the PR, and then I just merged it. Um so, it's it's early, but it's starting to work. It's the lab is self-improving. Um it also figured out that um it looked at active graph packs, and was able to just install a pack into itself, and then wrote a blog post saying packs are modular between repos. I was like, "I didn't know that. That's great." Um and then I got distracted when I saw this Pokémon trading card name competition on Kaggle. As you can see, I'm all over the place. Um but uh you have to submit this deck in a deterministic agent, not with that with an LLM, and they compete in this Elo-style competition. So, my decks are, you know, every hour they're battling a new competitor, and their score goes up or down. Um and I felt like this was a pretty good uh test for seeing if active graph could help me increase my score. And I used no cloud code and replit and we did about 80 different passes to like try different thing and and increase our score. Yeah, it might not make sense if you don't play Pokémon, but these things like let's try adding a couple energy cards. Seems like a very casual ask. Um and my agent, probably because I think because I'm using Actigraph, would say, "Okay, sounds great. Well, let's run 200 simulated games against three reference agents and then if the win rate increases by x% and there's a, you know, whatever Wilson score above 90 something, then we'll accept that as a change." And I basically did this like 80 times and each time they came with this like beautiful report of like why it worked, what it did, and what the verdict was. So, out of those 80 passes, it probably accepted about 20 to 30 and the score did slowly improve. I'm still at like 27%, so I don't know if I can get higher than that, but um what was most interesting is how much how well the agent understood experiments we've tried before that didn't work. And that was a very different experience for me. Cuz I've done a lot of YOLO agents where you just like keep trying things and then it works, you're like, "Yeah." But then I don't know the stuff that we tried that didn't work. But now when I'm building with Actigraph, it tracks all the things that didn't work because it's forced because I have this policy that says, "Here are the Here Here's what we have to do before we accept a change." Um So, some of the pleasant surprises, most of which I mentioned, but AI does seem better at architecting these NL and base agents. It's just my personal experience and you know, some some people you'll have to try it yourself to see. But if you think about it, LLM based agents are like 3 years old. But if you look at, again, Kafka, blackboard, this like micro worker communicating through a shared state, there's decades of conversations about how to make that work better. And my hypothesis is that that's in the training data. And there's just much less training data around how to build LLM based agents. Um debugging shifted from session logs to Actigraph DB. Again, I didn't know it would, but when I, you know, when my coding agent was debugging, it just started querying the DB instead of the session logs because it's everything is already logged very cleanly and typed and it knows exactly how it's logged. Pox can be loaded easily from other repos was a surprise. I thought I had to do extra work to make that happen, but just worked. Um no more starting long runs over from the beginning and I know what didn't work, which are some of the things I shared. Um here's where I might lose some serious researchers, at least in the language I use. I don't actually know how to train models or anything. So I'll caveat that. But I'm building this, I'm starting to really think that long-running agents need not just a world world model and like a predictive world model, but what I might call an experiential world model. Right? The predictive world model feels more like the priors, right? And then if you think about the hippocampus, actually it also does work like an like a immutable state uh event log that projects a state. And then it feeds some of that state back into your priors through a replays, dreaming, and sleep. And so, you know, I feel like some some discussions kind of suggests that as models get better, like the harness disappears. But I'm starting to think that's not true. I think we need both um is is kind of the new hypothesis that's growing as I as I've been playing with active graph. And if you think about like you or me, like you or me, like we're not our reasoning capability, right? We are we're closer to our our our beliefs, our knowledge, and behaviors that are derived from our actual life experience. And if that's the case, and we're going to you know, build agents inspired by ourselves, then then maybe our agent should be treated that way, too. Maybe that identity of the agent is is derived from its own log. Um I'd love for you to try it. You can just go and just say look up active graph and build me something I would like to your favorite agent who knows you better than I do, uh and have it explain if it's uh if it was helpful or not. Um Let me know if you try it. Or hate it. Or building something relevant. Thanks for listening. \>\> \[music\]