Ryan Nystrom, a software engineer on Notion AI, shows Every CEO Dan Shipper how he turns a spoken brief into a sourced Notion task, working code, and a passing pull request. Using a real model-picker migration as an example, he lets Notion AI explore the codebase and write a detailed plan, then hands that plan to a coding agent. Before human review, a custom multi-agent review swarm checks the frontend and backend changes for accuracy and sustainability. The agent opens the pull request, watches the tests, and fixes failures autonomously. Topics include starting tasks by talking instead of typing, giving Notion AI a coding environment, turning research into a detailed task, why planning beats one-shot prompting, using Notion as the source of truth, building a multi-agent review swarm, and creating a custom skill to encode team standards. Key takeaway: define the goal, constraints, tools, and verification system before the agent starts work. # Video Identity Observations: "How to Build a Multi-Agent Review Swarm" ## Observable Channel Presentation ### Intended Audience - **Developers and AI-forward engineering practitioners.** The content assumes familiarity with tools like **Cursor**, **Codeex**, Notion MCP, continuous integration, and PR workflows: > "I usually used, for example, Cursor... I would connect it to my repository—either via Boxy or locally—and ask it to create a Markdown file, or use Notion MCP to create a similar task." - **Teams evaluating agentic coding workflows.** The discussion of verification, code maintenance review, and scaling patterns ("am I creating something that will just fall apart when it scales?") targets professional engineering concerns rather than hobbyists. ### Claimed Scope - A **technical demo/interview format** — the video demonstrates a working multi-agent code review pipeline ("swarm review") inside Notion AI, not a theoretical discussion. - The scope is explicitly about **AI-integrated engineering practice at a real company**: > "We're not some cutting-edge lab, but we're integrating AI into everything as much as possible." - The host frames the channel's orientation toward AI-era companies and products: > "I think you're one of the most underrated companies in the AI era... you understand how valuable it is to have a corporate 'brain' connected to all your agents." ### Self-Description Evidence (Creator/Host Claims) - The guest (a software engineer at Notion, per the host's introduction) self-describes his workflow as speech-driven and detail-rich: > "Almost all of my tasks start with simple chatter. I truly believe that when I speak, I provide far more nuance and detail than when I simply type." - He self-describes his project-management style informally: > "I have a pretty negative attitude towards project management. I can't look at kanban boards... I just work on one task at a time." - He claims the multi-agent review process is highly effective: > "For me personally, it was the best vetting process I've ever experienced." ## Demonstrated Patterns (What the Video Shows) - A **live product demo**: a new Notion AI thread, connection to "Boxy" cloud VMs, code exploration of a model-selector redesign, automatic task creation, PR generation, CI monitoring, and a swarm review with correctness and maintainability reviewers run across **GPT and Opus**, aggregated by a lead agent. - **Quantified outcome evidence**: the agent worked from 12:53 to 2:27 p.m., producing a passing PR — "Literally one and a half to two hours of work that she did for me." - A **workflow-as-skills** philosophy: custom skills ("swarm review," "Take Notes") created by prompting Codeex rather than waiting for official tooling. ## Notes for Synthesis - Evidence here reflects one video's guest and host; channel-wide positioning (series branding, recurring format) cannot be concluded from this material alone. - All quotes are from the transcript; no curated metadata was provided beyond the title. # Delivery Observation: "How to Build a Multi-Agent Review Swarm" ## Overall Communication Style The speaker delivers a **conversational, demo-driven walkthrough** rather than a lecture. The structure moves from introduction → live demonstration → workflow narration → Q&A-style exchange, with frequent direct quotes of what they type into the agent. ## Tone - **Casual and understated.** Opens by downplaying the company: *"we're not some cutting-edge lab, but we're integrating AI into everything as much as possible."* - **Enthusiastic but hedged** about results: *"I don't want to idealize this 'first time' as if I just said 'do it without mistakes.'"* - Occasional superlatives with personal framing: *"for me personally, it was the best vetting process I've ever experienced"*; *"has been a real magic for me."* ## Recurring Vocabulary and Phrases - **"Cool"** as a verbal connector, appearing in multiple quoted instructions: *"Cool, I'll just open the computer"*; *"I say, 'Cool, add this to our M2 milestone project.'"* - **"Source of truth"** — used twice: *"use Notion as a source of truth"* and *"Notion essentially becomes the source of truth for the entire context."* - **Emphasis on eliminating friction**: *"No switching between apps"*; *"eliminating the need to constantly switch between tabs has been a real magic for me."* - **Habit/certainty framing**: *"this is almost my de facto method of project management"*; *"I have developed a habit of pinning tabs."* ## Teaching Habits - **Quotes own prompts verbatim** to make the workflow concrete: *"fulfill all the requirements… I don't want to watch this."* - **Names internal systems explicitly** and defines them on first use: *"We called it Boxy—it's our virtual machines… or remote on-demand computing."* - **Anchors teaching in personal workflow rather than abstractions**: *"In fact, I'll even open a new thread here so you can understand how it works."* - **Gives timestamps and outcomes as evidence**: *"this thing started working at 12:53… I finished at 2:27 p.m."* - **Contrasts old workflow vs. new**: *"I usually used, for example, Cursor… But I had to switch between apps, and now I just hang out with Notion."* ## Communication Patterns - **Preference for voice over typing, stated as a philosophy**: *"when I speak, I provide far more nuance and detail than when I simply type."* - **Talks to the agent in casual, forgiving language**: *"the models don't care how many times I say 'em'… I think it even helps."* - **Rhetorical question-and-answer self-narration**: *"And now… what if instead of just a sandbox, we give Notion AI access to our pre-built Boxy virtual machines?"* - **Naming habits**: labels practices as skills — *"swarm review skill"*, *"Take Notes"* — treating named abstractions as part of the explanation. ## Stable Patterns vs. Isolated Moments - **Stable (repeated):** "Cool" as a connector; quoting prompts verbatim; before/after workflow contrasts; naming and defining custom systems. - **Single occurrence (uncertain as patterns):** the extended reflection on disliking project management (*"I have a pretty negative attitude towards project management"*), and the aside about the link-editing feature example — these appear once and cannot be confirmed as recurring delivery habits from this transcript alone. # Observation Record: How to Build a Multi-Agent Review Swarm ## Speakers - **Interviewer / Host** — introduces the guest, asks questions. No curated name is supplied; identified only by conversational role. - **Notion Engineer (addressed as "Ryan")** — identified via the host's closing line, *"Ryan, this is just wonderful."* Stated to be a software engineer at Notion working on Notion AI. --- ## Position: Voice-first, Notion-native coding workflow ### Claim: Speaking provides richer context than typing > "When I speak, I provide far more nuance and detail than when I simply type on a keyboard." - The engineer says almost all of his tasks begin with "simple chatter" — freeform voice input. - He claims models tolerate verbal fillers and self-corrections ("Oh, wait, no, forget it, do it better like this"), and **"In fact, I think it even helps"** — note this is framed as his opinion ("I think"), not a tested claim. ### Claim: Notion AI + Boxy VMs replace multi-app coding setups - Notion AI recently gained computer use via **Vercel sandboxes**, but these are limited. - Notion has an internal project, **Boxy** ("our virtual machines for codecs or remote on-demand computing"), and the team connected Notion AI to pre-built Boxy VMs instead of sandboxes. - Demo shown: he asks the agent to explore the codebase for a model-selector redesign, then create a task in the "M2 milestone project" with context, requirements, and self-testing instructions — producing "a fairly detailed task with all the code references." ### Claim: The workflow eliminated app-switching - Previously he used **Cursor** (which he says he still loves), connecting it to his repo via Boxy or locally, generating a Markdown file or Notion MCP task — but had to switch between apps. - Now everything is native in Notion and runs in the cloud: "I send the message, close the tab, and go to a meeting." --- ## Position on project management - The engineer expresses a **negative attitude toward formal project management**: "I can't look at kanban boards or anything like that." - He works on **one task at a time** and pins tabs/chats as a de facto management method. --- ## Claim: End-to-end agent execution with a "swarm review" skill Stated workflow for executing a task: 1. New chat thread with a link to the assignment and instruction to "fulfill all the requirements." 2. A **"swarm review" skill** verifies all generated code with his "in-depth verification tool" before he looks at it. 3. Agent iterates, creates a **PR**, and **monitors CI** — fixing type errors and test failures autonomously. 4. Timeline stated: started **12:53**, finished **2:27 p.m.**, PR created, tests passed. > "She did everything the first time, and I don't want to idealize this 'first time' as if I just said 'do it without mistakes.'" - **Qualification preserved:** he explicitly cautions against over-reading the one-shot success. - He estimates **"literally one and a half to two hours of work"** was done by the agent while he was in meetings and starting other tasks. - His framing of his own role: he spends time **planning** — "What is the environment? What changes are needed? What are the limitations?" — then supplies instructions and tools. --- ## Claim: "Take Notes" skill — Notion as living working journal - Notion serves as "source of truth" for the initial spec and requirements, but his "Take Notes" skill extends this: as the agent builds a feature, it updates the doc with everything it finds and problems it encounters. --- ## Claim: Non-obvious benefits of a coding agent living in Notion Prompted by the host asking about side effects (e.g., an agent searching Notion meeting minutes), the engineer cites: - **Reduced context management burden** — no more checking "Is my local repository updated? Is Codeex updated? … connected to the correct environment?" - **Notion as the source of truth for all context.** - **Eliminating tab-switching** — described as "a real magic for me." --- ## Position: Multi-agent review architecture His rationale: large change sets need more than bug detection — they must be **maintainable**. His swarm review design: - **Divide the change set into domains** (e.g., frontend, backend). - Within each domain, delegate to two reviewer roles: - A **correctness reviewer** ("will look for errors"). - A **code maintenance reviewer** ("will look for patterns") — checking for reusable code, existing patterns, and whether "something … will just fall apart when it scales." - Each review pass is run for **both GPT and Opus**. - A **lead agent** collects all findings into a report on what needs doing. - The change-set agent repeats iterations "until it's good." > "For me personally, it was the best vetting process I've ever experienced." - **Framing note:** this is stated as personal experience, not a benchmarked comparison. --- ## Claim: Custom skills are trivially composable - Rather than accepting off-the-shelf skills, he opened Codex and asked it to: analyze the other skills, note what he likes/dislikes, and **create a new skill** — "And he just did it. And it turned into my own thing." - He frames this as emblematic of "our time — how flexible it all is." --- ## Host positions (as expressed in transcript) - Notion is "one of our favorite companies" and "one of the most underrated companies in the AI era," praising its "corporate 'brain' connected to all your agents." - He notes his own team uses Notion for planning/corporate information and says this workflow is "something new to work through." - He expresses hope the swarm review skill will be made open: "I hope you make this skill open so we can all benefit from it." --- ## Unresolved / Open Items - No benchmark or comparative evidence is given for the swarm review's effectiveness beyond the engineer's personal assessment. - The host's request to open-source the skill is not answered in the transcript. - The "first time" PR success is explicitly caveated by the speaker himself and should not be read as a general claim of reliability. # Video Attribution & Provenance Record **Video ID:** `01a0f15a-63e3-7066-bb50-5fab3ec8a98b` **Title:** How to Build a Multi-Agent Review Swarm ## Speakers & Roles ### Guest — Software Engineer, Notion (Notion AI) - Introduced by the host as working on **Notion AI** at Notion. - Demonstrates his personal AI coding workflow, including the **"swarm review" skill** and **"Take Notes" skill**. - **Name uncertainty (flag):** the transcript is inconsistent — the host appears to address the guest as *"Dane"* at the opening ("Thank you, Dane. Glad to be here.") but closes with *"Ryan, this is just wonderful. Thank you very much for joining."* The guest's actual name is not definitively established by the source. ### Host / Interviewer — Unnamed - Asks questions, provides framing commentary, and requests the demo. No name or channel is given in the source. ## Referenced Tools, Products & Internal Projects (claimed by guest) - **Notion AI** — including a computer-connection feature using **Vercel sandboxes**. - **Boxy** — internal Notion project: "our virtual machines for codecs or remote on-demand computing" (cloud VMs exposed to Notion AI as a computer option). - **Cursor** — third-party tool the guest previously used and "still continue[s] to love." - **Notion MCP** — used previously to create tasks from Cursor. - **Codex / GPT / Opus** — models referenced in the swarm-review process; the guest mentions asking Codex to analyze existing skills and create a new one. - **Custom skills built by the guest:** "swarm review" (divides change sets into domains, delegates to correctness and maintainability reviewers, runs passes on both GPT and Opus, lead agent compiles a report) and "Take Notes" (Notion as living working journal). ## Third-Party & Unverified Claims - Host's editorial claims (not verified): Notion is "one of our favorite companies," "one of the most underrated companies in the AI era." - Guest's claims about his workflow outcomes (e.g., task started at 12:53, finished at 2:27 p.m., PR created, tests passed, "one and a half to two hours of work") are self-reported and unverified. - Transcript appears to be a translation; contains artifacts (e.g., "virtual machines for codecs," "Codeex") that may be transcription errors. ## Promotional Segments - **Flag:** The interview functions as a product demonstration for **Notion AI and the internal Boxy project**; the host's opening contains favorable editorial framing of Notion. No explicit sponsorship statement is present in the source. - Host expresses hope the guest's swarm-review skill will be made "open"; no commitment or availability detail is given. ## Comment Availability - No information about comments or availability limits is present in the source. ## Evidence Channels Unavailable - No curated metadata beyond the title was supplied; no visual demos, screen contents, timestamps, or links are verifiable from the transcript alone. OK. Hello, welcome. Thank you, Dane. Glad to be here. So, you're a software engineer at Notion, working on Notion AI. Notion is one of our favorite companies, we built all of our work on it, and I think you're one of the most underrated companies in the AI ​​era because you guys are just "on point." You understand how valuable it is to have a corporate “brain ” connected to all your agents, and we’re excited for you to show us how you work, especially how you write code in Notion. Yes. Well, we're not some cutting-edge lab , but we're integrating AI into everything as much as possible. We use AI continuously for everything. I see and feel it through your product solutions. So maybe we can move on to your demo to show you how you work? Yes. Yes. Now we see how much I'm talking here. In fact, I'll even open a new thread here so you can understand how it works. I'm starting a new thread. For me, almost all of my tasks start with simple chatter. I truly believe that when I speak, I provide far more nuance and detail than when I simply type on a keyboard. When I just talk, it's completely freeform. I add a lot more context and detail, and the models don't care how many times I say "em" or "do this." Oh, wait, no, forget it, do it better like this . Models don't care. In fact, I think it even helps. But this demonstration that we're seeing now—this is what I was actually working on today. We're working on a redesign of our model selector, and this version, the alpha version of our new AI, doesn't have this new selector yet. And that's cool. The guys who created this did a great job. I want to move it here, share the code, or do something else. So I just say: blah blah blah blah . We recently added the ability to use Notion AI and connect a computer to Notion AI. We are currently using Vercel sandboxes. And every time you ask Notion AI to do something that's too complex for the tools it has, it just says, "Cool, I'll just open the computer." "I'm going to start using Bash or something like that, creating PowerPoint presentations, you know?" "HTML files, or whatever." This is great, but every time you do this, you start with a very limited "sandbox ". And thanks to what we're developing, we have another internal project. We called it Boxy—it's our virtual machines for codecs or remote on- demand computing. So instead of running everything locally, I have these other machines in the cloud that I can connect to and write code on. We thought, “Listen, what if instead of just a sandbox, we give Notion AI access to our pre-built Boxy virtual machines? » And now our little menu of choices has Boxy as our computer. So when I ask him to figure out how the new model selector we're building works, he can run all these commands and explore our code. And finally, in this case, I ask him to create a task. I say, "Cool, add this to our M2 milestone project." "Create a task and include the context, requirements for this migration, and a description of how to test everything yourself." And as a result, it outputs a fairly detailed task with all the code references, requirements, and steps for verification—all ready to go. Previously, before we created this, I usually used, for example, Cursor. I love, and still continue to love, using Cursor. I would connect it to my repository—either via Boxy or locally —and ask it to create a Markdown file, or use Notion MCP to create a similar task. But I had to switch between apps, and now I just hang out with Notion, which has all these native page building tools, and it just builds them. No switching between apps or anything like that. Plus, everything works in the cloud. So I send the message, close the tab, and go to a meeting or do other things. I come back and I'm like: cool, here's my assignment. Like. Is all this on the board somewhere ? Yes, we have the company's main task board here. Hmm, I have a pretty negative attitude towards project management . I can't look at kanban boards or anything like that. I just work on one task at a time. I have developed a habit of pinning tabs or chats while I work. And this is almost my de facto method of project management: what tasks do I still have assigned? That's what I have to think about. This is really interesting. And does that also trigger verification by other people, or how do you track the entire lifecycle? Well, yes. It's just creating work that needs to be done. Then I start a new chat thread and write: "work on this." And I actually, you can see here, this is my actual workflow. So I'm adding a link to the assignment. I am writing: fulfill all the requirements. I have a "swarm review" skill because before I look at the code, I want this thing to check all the code it generates with this in-depth verification tool that I built. And when she finishes the iterations, I want her to create a PR. Then I want it to monitor all of our continuous integration. If there is a type error, the test fails—that's okay, just go and fix it. I don't want to watch this. And this thing started working at 12:53. And all this right in Notion. Yes. In the end, I finished at 2:27 p.m. PR created, tests passed. I used to just click and everything was perfect. She did everything the first time, and I don't want to idealize this " first time" as if I just said "do it without mistakes," but the coolest thing for me is that I spend time planning and thinking about what I want to achieve. What is the environment? What changes are needed? What are the limitations? And then I provide her with all the instructions and tools she needs to create that feature. And it completely eliminated routine work. Literally one and a half to two hours of work that she did for me. And I was at the meetings. I started other tasks. I think for me one of the main reasons I love doing this in Notion now is because it allows me to find an example. We built this link editing feature a while ago, and part of my skill, as I call it “Take Notes,” allows you to use Notion as a source of truth for not only the initial spec and requirements, but also turns that into a living working journal. When it builds a feature, it updates everything it finds, problems it encounters, and so on . This is really interesting. I'm curious because we use Notion for planning and corporate information. So for me, this is something new to work through. Oh, maybe I could code right here. What are some non-obvious side effects of having your coding agent live in Notion that I might not be aware of? For example, I'm thinking about constantly having Codeex search our meeting minutes , which are kept in Notion, right? So I assume that things like this happen. What is your experience at this point? Yes, for me personally, this turned out to be much more important than I thought, just so that I didn't have to manage a large amount of context. Is my local repository updated? Is Codeex updated? Do I have the latest version, is it connected to the correct environment, and so on . I think the simple possibility of eliminating more of these steps is very exciting. But, in my opinion, the best thing is that Notion essentially becomes the source of truth for the entire context. And eliminating the need to constantly switch between tabs has been a real magic for me. This is great. Very cool. Is there any other part of the process you want to show us? I noticed that we were making huge changes, especially with this feature we were developing, and I needed something that would not just look for bugs, which of course needed to be fixed, but also make sure I was writing maintainable features and infrastructure changes. So I created a swarm review skill to divide the change set into domains. For example, you have a change in the frontend and backend, and then within those changes, you delegate tasks to a correctness reviewer and then to a code maintenance reviewer. And then each of them will look for errors, and the other will look for patterns. Do we have code that we need to reuse? Are there existing patterns we should be using, or am I creating something that will just fall apart when it scales? And for each of these fragments, I also run all of this for both GPT and Opus. So they do each of their passes, and then the lead agent collects all the findings and turns it into a report on what needs to be done. And basically, I just make the agent repeat it until it's good. And for me personally, it was the best vetting process I've ever experienced. And I think that's another really cool thing about our time— how flexible it all is. I thought, cool, all these other skills are really great, but I have my own things that I care about. And instead of just saying: well, no one created this. I don't have time or anything. I just opened Codeex and said: analyze these other skills, but here's what I don't like about them. That's what I like. Create a new skill for me. And he just did it. And it turned into my own thing. This is simply incredible. I hope you make this skill open so we can all benefit from it. Ryan, this is just wonderful. Thank you very much for joining.