Talk by Jeffrey Lee-Chan (Snapchat) on building production multi-agent AI systems, arguing the main gap is the agent harness rather than the model. Covers persistent memory and contextual guardrails for task loops, splitting stacks into "Agent Orchestrator Manager" and specialized worker roles, real-time terminal routing via CMUX, token-cost tradeoffs across leading models, and examples from consumer applications like WorldAI and Consensus ML. Includes an architectural checklist for agent infrastructure and a workflow—natural language to code to automated iteration to evidence to human review—that minimizes human intervention to mid-loop bottlenecks, enabling one engineer to direct many parallel coding agents. # Channel Identity Observations — "Develop at Idea Velocity - Jeffrey Lee-Chan, Snapchat" ## Video Format & Presentation Evidence - **Format is a live, hands-on workshop**, not a polished produced video. Evidence: opening lines — *"there are some people who kind of have like never installed open claw, and I want to kind of get them set up"* — and the speaker structuring the session: *"10 minutes Q&A, 10 minutes some more advanced topics, and then just more Q&A... I'll just set some timers to make sure I don't go off track."* - **The channel hosts practitioner-led technical talks from named industry speakers.** The title explicitly attributes the session: *"Jeffrey Lee-Chan, Snapchat."* - **Audience interaction is captured in the recording**, including questions and an impromptu guest contribution (a tmux creator: *"one of the creators of tmux... Jeff as one of our power users"*). ## Intended Audience - **Mixed-skill developers interested in AI coding agents**, explicitly addressed as two tiers: - Beginners: *"some people who kind of have like never installed open claw"* - Advanced users: *"the other people like want to try an experimental thing... for more advanced people that'll be cool"* - **Self-identified non-technical attendees are welcomed**, with the speaker reassuring: *"they're not... bad questions. All good."* ## Claimed / Demonstrated Scope - **Topic scope: agentic AI development workflows.** Demonstrated through extended walkthroughs of Open Claw, Claude Code, tmux parallelization, agent orchestration, sandboxing, and model/token tradeoffs (e.g., *"I use this until like this is getting low and then I just switch to MiniMax, which is not as good, but it kind of gets the job done"*). - **Philosophy framing from the speaker:** *"I actually think an agent could replace me. So, that's kind of what I'm working on right now to like get things even more autonomous"* — a stated personal claim about his work direction, not a channel-level claim. - **Includes show-and-tell of side projects**: an *"AI RPG"* with a D&D dice system and a *"multi-AI analysis website"* — suggesting the channel surfaces builders' personal experiments alongside tooling workflows. ## Self-Description vs. Demonstrated Patterns | Type | Evidence | |---|---| | **Explicit self-description** | Speaker identifies as a Snapchat engineer (via title); describes his own setup: *"I use multiple agents with work trees, that's very key for parallelization."* | | **Demonstrated pattern** | Patient, interactive teaching style — repeatedly pausing for questions (*"I'll pause here to see if anyone has any questions or thoughts"*), honestly flagging unfinished work (*"an experimental thing, which I haven't even fully gotten working"*) | ## Notes for Channel-Level Synthesis - No channel metadata was supplied beyond title; no explicit channel self-description, tagline, or subscriber-facing claims appear in this material. - The tmux creator's mention of *"Cloud Code Teams integration"* and *"Cmux SSH"* is guest content, not a channel claim — should not be attributed to the channel's own scope. # Delivery Observation: "Develop at Idea Velocity" — Jeffrey Lee-Chan ## Overall Format - **Workshop-style, interactive session** rather than a one-way talk. The speaker announces a structure upfront: > "I think uh 10 minutes Q&A, 10 minutes some more advanced topics, and then just more Q&A." - He explicitly manages his own pacing: > "So, I'll just set some timers to make sure I don't go off track." - Frequent **pauses for audience questions**: > "I'll pause here to see if anyone has any questions or thoughts." > "Anyone have any other questions related to some of that stuff?" ## Tone - **Casual and conversational**, marked by heavy use of "like," "you know," "um," and "uh" throughout (e.g., > "So, you know, I've got this cool super diagram"). - **Self-deprecating honesty** about incomplete work: > "want to try an experimental thing, which I haven't even gotten gotten working." - **Encouraging toward beginners**: > "No no no bad questions. All good." ## Recurring Verbal Habits - **"So" as a sentence opener**, extremely frequent: > "So, this kind of describes like my setup." > "So, what's cool about this is..." > "So, here's a two websites that I've built." - **"All right" as a transition marker**: > "All right. Let me start." > "All right, so this is compacting whatever." > "All right. If not, I will go here." - **Hedging qualifiers**: "kind of," "pretty," "a bit," "whatever" appear constantly — > "you can kind of see you can talk to it and be pretty brief" and > "which usually is JSON or whatever." - **Rhetorical "right?"** to check shared understanding: > "I'm like, give it a longer time out, right?" > "we got to merge it, right?" ## Teaching Habits - **Live demos interleaved with slides**: > "All right. Can everyone see this presentation?" followed by switching to show two websites and terminals. - **Concrete personal examples** to illustrate abstractions — e.g., the "skeptic agent": > "I'm working on this thing I'm calling a skeptic agent, and it's a custom code review thing." - **Reconstructed dialogue** to narrate agent interactions: > "you're working with them and they'll be like, 'Hey, I've got this thing ready.' And I'm like, 'Okay, we'll run the test, right?'" - **Direct answering with structure** — when asked why Open Claw over Claude, he gives a labeled reason: > "The reason I use Open Claw as specialization," then elaborates with a concrete scenario. - **Checks whether his answer landed**: > "Um that was a little long, but does that kind of address the theme of your question?" - **Qualifies confidence with experience-based caveats**: > "these things didn't work that well with agents like... 6 months ago. But now like agents are pretty good at..." ## Audience Management - **Delegates to experts in the room**: > "I think is Austin or someone from tmux here?" then hands the floor over: > "I don't know, you can spend a minute if you want to talk about anything cool about tmux." - **Tiers the audience by skill level** repeatedly: > "there are some people who kind of have like never installed open claw" vs. "for more advanced people"; later, > "Or if you're not as advanced, if you want it, you know, you can try it." ## Vocabulary / Domain Register - Heavy developer jargon used without glossing: **worktrees, tmux, MCP, CI, PR, sandbox, staging, token usage, Docker**. - Occasionally defines terms briefly: > "tmux is like a terminal program, but it's pretty good for parallelization." - Casual cost-benefit framing: > "It wouldn't double your usage, but it would definitely increase it. But then the benefit is you might have some more reliability." ## Uncertainty Notes - The transcript appears to begin mid-session and includes brief garbled segments (e.g., around [357]–[368] and [752]–[763]), so some observations rely on partial context. - Filler-word and "so"-initial frequency is clearly high, but exact counts can't be verified from this transcript formatting. # Observation Document: "Develop at Idea Velocity" — Jeffrey Lee-Chan, Snapchat ## Session Setup (Jeffrey Lee-Chan) - Workshop splits attendees into beginners (installing **Open Claw**) and advanced users (experimental **staging environment with two Open Claw instances** — which Lee-Chan admits is *not fully working yet*). - Format: timers, ~10 min Q&A, ~10 min advanced topics, discussion breaks every ~5 minutes. ## Core Claims: Workflow Description ### Setup Overview (Lee-Chan) - Claims his setup works "maybe like 70% of the time" — qualifications matter here. - **Frictionless communication**: Open Claw retains **context and memory** across requests, so brief Slack-style messages ("fix this thing") suffice without re-explanation. Example: his custom code-review **"skeptic agent"**. - Uses **multiple agents with git work trees** for parallelization, plus CI/code integration. - Debugging still required (e.g., timeout fixes). ### Notable Position: Self-Replaceability > "The types of responses I give, they're not that like, special. As in I actually think an agent could replace me." - States he's working toward greater autonomy, but framed as aspiration, not settled fact. ## Concepts: What Is Open Claw? (Lee-Chan) - Frames Open Claw as a **concept**, not a repo: frictionless communication enables "higher level of velocity." - **Orchestration hierarchy**: Open Claw's agent orchestrator managers manage workers; sometimes he prefers **tmux terminals** for personal control. Beyond the Claude Code layer ("managed agents" with sub-agents), he notes control is less his — and that layer of his stack changes frequently. ### Q&A: Why Open Claw vs. Direct Claude? - **Lee-Chan's stated reason: specialization.** Open Claw decisions should weight the *spec, goals, and task history* rather than code mechanics. Claude consumes context on Claude MDs, skills, MCPs — "imagine like 25% of your context already taken up by implementation." - **Temporal claim**: agents improved significantly over the last 6–12 months; browser tests that failed before (finding pop-ups, entering passwords) "now no problem." - Recommends auditing which problems *truly needed a human*. ## Demo Projects (Lee-Chan) - **AI RPG website**: custom reactive world; differentiates itself from raw ChatGPT/Gemini chats via a **D&D dice-roll system** so players don't "just win too much." - **Multi-AI analysis website**: automates his habit of querying multiple models and synthesizing answers — "usually I like these answers better than asking one model." ## Manager-vs-Worker Bias Claim (Lee-Chan) - Uses terminals "more like a manager rather than a coder." - **Claimed benefit**: less bias — direct coding sessions show agents "want to say things are really working," whereas a manager with different context would close PR 294 in favor of a superseding PR. - Reports efficiency gains with tmux; presents this as personal feeling ("I feel"). ## tmux Endorsement (Austin, self-identified tmux creator) - Happy to have Jeff as a "power user"; endorses free of charge (Lee-Chan likewise states **"I'm not paid to say this"**). - Announces: **Cloud Code Teams integration** (auto-spawning terminals) and **Cmux SSH** (tailscale, running Open Claw on Mac minis). Invites bug reports and feedback. ## Sandbox / Token Usage Discussion - Lee-Chan clarifies **two meanings of "sandbox"**: Docker isolation vs. his dual-instance staging setup. - Dual instances: won't double token usage but will increase it; his proposed pattern is **local dev → integration tests on staging → merge/deploy to production**, trading cost for reliability. - Admits the staging setup is unfinished ("competing with the other one, so I got to debug it"). ## Model Choice (Lee-Chan) - Uses **Codex 5.3**; found **GPT 5.4 "just use more tokens"** and he gets "destroyed all the time" on cost. - Switches to **MiniMax** when budget runs low — "not as good, but it kind of gets the job done." Explicitly framed as **money-driven, not preference**. ## Preserved Qualifications & Tensions - Workflow success rate is ~70%, not full autonomy; debugging remains routine. - Staging/two-Open-Claw setup is experimental and incomplete. - Efficiency and bias-reduction claims are subjective ("I feel," "I think"). - Model satisfaction is constrained by cost rather than capability preference. # Speaker Attribution Record — "Develop at Idea Velocity - Jeffrey Lee-Chan, Snapchat" ## Identified Speakers & Roles | Speaker | Role | Evidence | |---|---|---| | **Jeffrey Lee-Chan** | **Presenter / workshop lead** (identified via video title and transcript; presents slides, demos, answers questions) | Dominant speaker throughout; title attributes the talk | | **Audience member (Q&A, "why use Open Claw vs Claude")** | Workshop attendee / questioner | Asks about Open Claw vs. Claude directly | | **Aaron** (addressed by name by presenter) | Workshop attendee / questioner (self-describes as non-technical) | Sandbox/token usage question; presenter addresses "Aaron" | | **Austin** | **Guest contributor; self-identified "one of the creators of tmux"** | Invited by presenter to speak about tmux | | Unidentified attendees | Audience (confirmation "Yes," thanks, brief interjections) | Minimal utterances | ## Promotional / Endorsement Flags - **tmux segment**: Presenter states *"I'm not paid to say this or anything, but big fan of tmux"* — an unpaid endorsement. - **Austin's tmux pitch**: Promotes tmux features (Cloud Code Teams integration, Cmux SSH, app) and solicits feedback/emails — **flagged as promotional segment** from a product creator, even though unpaid. - No other third-party promotion detected. ## Referenced Sources & Third-Party Claims - **Open Claw** — subject of the workshop; presenter's claims about its memory/context behavior (first-hand, unverified). - **Claude / Claude Code / Claude MDs / MCPs** — referenced in comparisons; presenter's characterization only. - **Claude co-work** — mentioned as comparable tool. - **Gemini chat, ChatGPT** — mentioned in comparison to presenter's AI RPG demo. - **Codex 53, GPT 54, MiniMax** — model names cited by presenter re: token usage and cost preferences (first-hand anecdote, no benchmark data). - Presenter's demos: **skeptic agent** (custom code review), **AI RPG website**, **multi-AI analysis website**, **tmux-based multi-agent setup**, **agent orchestrator framework (open source, forked)**. ## Claims of Note (Provenance) - "Open Claw" context/memory and manager-vs-worker bias claims are **presenter's personal experience**, not cited research. - tmux claims ("shipping," integrations) come from **Austin, a tmux creator** — vendor claim. - Presenter notes staging/dual-instance setup is **experimental and not fully working** by his own admission. ## Availability & Uncertainty - **No curated metadata supplied** — no channel info, comment availability, or publication date available from the source. - Audience speakers are largely **unidentified by name** except Aaron (named by presenter) and Austin (self-identified). - Transcript ends mid-sentence; final speaker turn (presenter on model choice) is incomplete. workshop setup, so um there are some people who kind of have like never installed open claw, and I want to kind of get them set up. And then the other people like want to try an experimental thing, which I haven't even fully gotten working, but I think for more advanced people that'll be cool um to get like a sort of staging environment and run two open claws. So, I think if I'm not, you know, too busy helping new people set up, then I'll try to help people like finish debugging the last part, or maybe someone else will get it. And I think uh 10 minutes Q&A, 10 minutes some more advanced topics, and then just more Q&A. And uh probably after every 5 minutes or so, we can do a bit of a few minutes of discussion. So, I'll just set some timers to make sure I don't go off track. All right. Let me start. All right. Can everyone see this presentation? \>\> Yes. \>\> All right, cool. \>\> Yeah. \>\> So, this kind of describes like my setup. Um I'll just keep it really high level for this part where when everything works, you know, maybe like 70% of the time, you describe your task. And then what's nice is open claw will have like context and memory about what you're talking about. So, for example, like let's say I'm like, "All right, fix this thing." It's a very brief Slack message, but open claw can remember like what I've asked it to do before. So, for example, I'm working on this thing I'm calling a skeptic agent, and it's a custom code review thing. But if I'm like, "All right, fix the skeptic agent." or whatever, right? It kind of knows what that is, which is nice. You don't have to re-explain a lot. Um I use multiple agents with work trees, that's very key for parallelization. Uh, CI code good integration. But you can kind of see you can talk to it and be pretty brief and then, you know, usually get reasonable outcomes, but sometimes you'll have to debug it. So, then it's like, all right, there's some time out thing. I'm like, give it a longer time out, right? And then I asked that here like like uh, you know, what are the priorities and what we're working on. And I'm reading it here. And then I might have just said like, uh, I just made code for it, right? So, another interesting thing about this is you can kind of see from like, um, \>\> \[clears throat\] \>\> the types of responses I give, they're not that like, um, special. As in I I actually think an agent could replace me. So, that's kind of what I'm working on right now to like get things even more autonomous. You know, and you and you'll see this with LLMs like you're working with them and they'll be like, "Hey, I've got this thing ready." And I'm like, "Okay, we'll run the test, right?" Uh, all right. So, I'll share the next slide now. Oops. Okay. We're going to some concepts right now. Like, what is Open Claw? So, I think what's cool about Open Claw is it's not just about a particular repo or the code, but more of the concept. So, you know, I've got this cool super diagram, but first concept is frictionless communication. So, I know we have other things like Claude co-work or whatever, but I think they still have a similar concept where it's like, can you like easily talk to your AI versus like you've got a, um, remote desktop into your computer or you have to go sit down. So, that kind of enables certain really higher level of velocity. Um then I think the central axis is very important. So, you know, there's a spectrum like I kind of go like a little more cowboy and I give it act. Open Claw has these agent orchestrator managers that manage your workers for you. Uh or sometimes when I want more personal control, I use tmux terminals. So tmux is like a terminal program, but it's pretty good for parallelization and AI development. Then I use the open source framework that I forked agent orchestrator to do the workers. And then once it gets to here, this part is like not really controlled by me as much anymore. Um so, I'm calling them managed agents, but basically like you have a worker that runs Claude code and Claude code itself can run agents and those can have sub agents. So once you get to this Claude code part, right? This part is not exactly my stack, just stuff I use, but this part of the stack I'm changing a lot more. Um I'll pause here to see if anyone has any questions or thoughts. \>\> Um can you hear me? So I I have a question. So the the real question is, you know, why why use Open Claw versus directly use Claude? \>\> So yeah, that's that's definitely a good question. Um \>\> \[clears throat\] \>\> The reason I use Open Claw as specialization. So when Open Claw makes a decision, I want that context to be more about like the spec or the goals or like the history of what I want in the task rather than the code. And you can just imagine this, right? As soon as you open up Claude, it reads Claude MDs, it reads skills, it reads um MCPs. A lot of those things are sort of independent of like the actual task. It's more about how to do the task. Um so imagine like 25% of your context already taken up by implementation. Versus like when you think about open claw, you're like, okay. I want to think about exactly what I want to do and how it relates to all the other slacks Jeffrey sent me in the last 2 weeks and put it all together and give me like a reasonable spec. Does every single thing a player does, but it does it without a browser, right? So you could use MCP, which usually is JSON or whatever or HTTP. Doesn't really matter actually. And then the final thing is the browser test. So some things like are visual. I mean, maybe you could even have CSS tests or JS tests, right? And that kind of goes here. But then the final thing is visual. So I would recommend you like try um similar approaches to this and be like, okay, like which problems truly needed that human or not. Um and I think what's kind of cool is like every quarter or you maybe even every month like it improves. So before like I had to really manually test a lot because like you know, this worked fine, right? But these things didn't work that well with agents like last year but like a year ago even 6 months ago. But now like um agents are pretty good at like nailing down a lot of browser tests for me. Versus like when I started my agent would have a lot of problems, you know, finding a pop-up and entering a password. Now no problem. Um that was a little long, but does that kind of address the theme of your question? \>\> Yeah, I think this is good. Good. Thank you. \>\> Okay. \>\> Thank you very \>\> Just just remind me for all these slash commands. I'll put um links here instead. All right, cool. Anyone have any other questions related to some of that stuff? All right. If not, I will go here. All right. So, here's a two websites that I've built. \>\> \[clears throat\] \>\> Yeah, while that loads, I'll show this one. So, this is like a AI RPG. Um what's cool about this is you can I've got a default campaign, for example. Um \>\> \[clears throat\] \>\> Yeah, pick some options. You've got a avatar. And uh what's cool about this is it builds a custom world for you that reacts to you. So, you can like kind of say whatever you want, do whatever you want, you'll get a story back. Um the main difference between playing this versus like um you know, you could just always go into the Gemini chat or or chat GPT or whatever is that um I have like a D&D system. So, if you play your own like games or novels, like you kind of just win too much. Versus like with this, you'll actually do like dice rolls and be like, you know, did the person actually um did you actually succeed in your action or not? All right. So, while this is going, I'll also um show this thing. Yeah, so this is funny. Um this website is like a multi-AI analysis website. Um so, what I found was like whenever I was doing research or or whatever, I would go to multiple models and I'd be like, what's the answer? Then I copy and paste them all. I put them into one model. Um so, pretty simple concept, but this does it for you. So, usually I like these answers better than like asking one model. And I'm like, okay, how do I do this workshop? So, I can be like, okay, what's the status? Uh what's the latest PR now? And um what I like about this is it has that sort of vertical tab type of thing. Cuz when you go horizontal, it's really easy to lose track of your tabs. Um but we've got notifications, too. So, when this thing is finished or this is finished and I need to look, it'll give me a notification. And I can just focus on clearing the notifications. So, here um the way of using these terminals, usually it's more like a manager rather than a coder. Um and it's kind of interesting, but it gives you a certain benefit where I feel I feel like the work is not biased anymore. So, when I when I code with these directly, um I usually feel like there's a bias, where it wants to say things are really working or whatever, right? Versus like, you know, um I'll show this. Let's look. If this had been working on PR 294 by itself, I think it would have been like, "This PR is amazing. Like, we got to merge it, right?" But then this one was like, "No, like, there's another PR that should supersede it, and probably we should just close this PR." Right? So, that's kind of the benefit you get, where the manager has a different context um than the workers. \>\> \[snorts\] \>\> Uh let's see. All right, so this is compacting whatever. So, it will take some time to complete. Don't want to wait for it, but when it completes, you see these notifications, and I think my efficiency has improved a lot with tmux. Um I think is Austin or someone from tmux here? \>\> Yeah, here. What's up? Go. \>\> Yeah, so Austin is uh \>\> \[clears throat\] \>\> one of the guys behind tmux. Um you know, big fan of it, and I'm not I'm not paid to say this or anything, but big fan of tmux. Uh been using it a lot. I don't know, you can spend a minute if you want to talk about anything cool about tmux. \>\> Yeah, so yeah, one of the creators of tmux. Really happy to have Jeff as one of our power users. And also like kind of endorsing it for free. But yeah, lots of uh yeah, lots of things that we're shipping. We shipped a Cloud Code Teams integration. So, if anyone is using Cloud Code Teams and wants to actually see what the salvations are doing, we'll automatically spawn terminals for that. We also shipped Cmux SSH. So, if you do any SSH work, you can use our native Cmux SSH to you know, do your tailscale and etc. To other computers. You can even use it to run your own open claw in your Mac minis. But yeah, just one of the creators. Please feel free to email me if you have anything that you think is a bug or any feedback, feature requests. I'm always available and yeah, just love seeing people using Cmux and always want to take as much feedback as I can to like get them as get them \>\> Okay, so just download that. \>\> Yeah. They actually have an app too. So, the app might be nice. \>\> Yeah. Okay. And then use that to help me get back in and set set up. \>\> Exactly. Yeah. \>\> Okay. Okay, cool. Thanks. \>\> All right. Is using the sandbox, it should not. Where would you Have you had Have you run into any troubles Aaron about sandbox token usage? \>\> Uh no. I'm just I'm not technical at all. I have open claw installed, so I'm \>\> Okay. \>\> very newbie questions. \>\> No no no bad questions. All good. Yeah. Uh they're not the sandbox wouldn't do that. I think if if everything's working well for you, you know, keep using the sandbox. For me, I just you know, don't use a sandbox and so so far nothing bad has happened. But I think if I had an external bot, then I would use a sandbox too. \>\> You changing the model testing a new model like it'll go down then I have to figure out how to get it back up again. So I think I want to set up this sandbox because \>\> Oh, sorry. Sorry. Um okay, wait. There's two kinds of things. Okay. The sandbox that most people talk about means like they run it in Docker or they like have it running in isolated part of their system. But the other one was the one where I made the staging instructions where I was like oh \>\> Yeah. \>\> Okay. So that could double your token usage if you send the same work to both of them. If you have two of them, right? Um but I think the way I would use it is not like that. Like I think I would do local development and then I would run integration tests on the um sandbox or staging one, right? So I have two of them. And then once like everything's good, then I would merge the code and deploy to the production one. \>\> I see. \>\> It wouldn't double your usage, but it would definitely increase it. But then the benefit is you might have some more reliability. So \>\> Yeah. \>\> I I made some instructions and I was kind of in the middle of setting it up, but then, you know, it's like competing with the other one, so I got to debug it. \>\> Yeah. \>\> But I think, you know, it's it's worth trying out though. So I had that for more advanced people to give it a give it a try. \>\> Cool. Thanks. What \>\> Or if you're not as advanced, if you want it, you know, you can try it. \>\> Yeah, I'm definitely going to play around. But what a what model do you use? Like do you have a something that like a default orchestrator model and then you say for this you use \>\> use Codex 53. Um I found I found uh GPT 54 to just use more tokens. Um and even 53 like like I I just get destroyed all the time, right? So I might have to adjust that. Um so basically I use this until like this is getting low and then I just switch to MiniMax, which is not as good, but it kind of gets the job done. And then this is more about money than like preference, but some some work I can still kind of just do um um here. So, I don't always have to