The video explores integrating OpenAI's live voice into compound engineering polish sessions, experimenting with ways to provide live voice feedback during those sessions. # Video Identity Observations **Video:** *Experiments with Kieran, OpenAI Live Voice inside Compound Engineering* ## Observable Presentation Evidence ### Format and Style - **Live, informal streaming format** rather than polished production. The creator tests audio at the start ("check, check, check"), mentions ambient YouTube music, and addresses live viewers ("for those who just joined," "if anyone doesn't understand what I'm doing or has any questions, please post them. I see them here, so welcome"). - **Work-in-progress demonstrations**: the stream shows brainstorming, planning documents, and agent runs in Cursor, not finished content. The creator states, "I'm just working, doing my usual thing on a regular September day, relaxing in the backyard." - **Multi-tasking and break-taking on camera**: "I'll take a five-minute break. I'll just sit here, drink some water." ### Intended Audience - **Builders and AI-tool practitioners**, not general audiences. The content assumes familiarity with plugins, repositories, CLI tools, MCP, React, and agent frameworks. - **Live chat is treated as a collaborator**: the creator answers a viewer question about product-market fit testing at length and thanks another viewer ("Brandon, I agree..."). - The creator frames the audience implicitly as users of the tools being built: "we released a new version of the plugin, the component engineering plugin." ### Claimed Scope and Self-Description - **Explicit affiliation claim**: "I work at Every, this is the only subscription you need to stay on the cutting edge of AI... We're exploring what the future of work is, and my attempts to do that are an attempt to figure out whether it's the future of work or not." - **Named projects presented as the channel's subject matter**: *Compound Engineering* plugin, a new "Compound Work" concept ("Compound engineering, but without engineering. Intellectual labor"), the *Riffraff/Refract* feedback-capture library, *Monolog*, and Cursor projects. - **Positioning as an experimenter, not a teacher**: "I have interesting ideas, and sharing them is fun," and "I'm just figuring out if it makes sense." ### Demonstrated Patterns (vs. claims) - The material *demonstrates* a real-time ideation loop: voice notes → Monolog → Cursor brainstorm → agent planning → parallel agent execution across local and cloud machines ("You can see... this is running on a computer, my Mac Mini. This one is in the cloud"). - The creator states a preference for **transparent, opinionated building in public**: "It will probably be fine there, in the river repository," and promises follow-up: "I'll probably do another broadcast tomorrow where we'll see what it actually does." ## Summary This video presents the channel as a **live working session of an Every-affiliated builder** sharing experimental AI-tooling development (Compound Engineering, live voice agents) with a technically fluent audience, positioned under a stated theme of "exploring the future of work." # Delivery Observation Document **Video:** *Experiments with Kieran, OpenAI Live Voice inside Compound Engineering* > Note: The transcript appears to be an automated capture (possibly machine-transcribed/translated), so verbatim wording should be treated with some caution. Quotations below are exact from the supplied transcript. --- ## 1. Tone ### High-energy, informal, and enthusiastic The delivery is conversational and casual, with frequent audible fillers and self-corrections that remain in the stream rather than being edited out. - Fillers appear constantly: "um," "ahem," "okay," "yeah": > "So, um, actually, we released a new version of the plugin..." > "Ahem. Let's see. So..." - Repeated enthusiasm markers attached to feature descriptions: > "New prototype functions with annotations have appeared. If you add an annotation, it automatically does cool things. This is really cool." > "I love Cursor, especially the projects now." > "So the questions here are like, 'Hey, who's the person talking to the agent...'" ### Transparent and unpolished by design The speaker narrates confusion and uncertainty live rather than hiding it: > "Right now I'm just figuring out if it makes sense. I think it's an interesting idea, and it's going to work. The only question is where do you put all of that?" > "I don't have a definitive answer, but I'm working on things..." > "I'm still figuring it out today, but it's not something I want to work on right now." --- ## 2. Vocabulary and Recurring Phrases ### "Cool" as a dominant evaluative word "Cool" appears as both approval and transition marker throughout: > "Okay, cool. Steeply." | "That's really cool about projects." | "I think that would be cool." ### "Let's see" as a pacing device Used repeatedly to move between segments, often several times per minute: > "Let's see. And I'm also trying out this streaming format." | "Okay, let's see. Um, come on, come on, let's see." ### Hedged, exploratory language ("maybe," "I think," "like") Ideas are consistently framed as experiments, not conclusions: > "I think maybe I could experiment with that... there's a mode in Polish called live mode or something like that" > "Maybe it would be some kind of mode... or maybe it's a mode using polish." > "So the concept says Compound Work is not on its own, but it could be, but it has to work together with Compound Engineering." ### Direct, spoken-register address ("Hey, ...") Instruction and thought simulations frequently begin with "Hey": > "you say, 'Hey, this should be a different color.'" | "If you say, 'Hey, look over there.'" | "when you say, 'Hey, I actually want to use this in a different way.'" --- ## 3. Teaching and Explanation Habits ### Restating the project for newcomers multiple times The speaker re-explains the core idea each time new viewers may have joined: > "And for those who just joined, I'm working on improving the polish team at Compound Engineering." > "Greetings. For all the new users, I'm creating a way to provide real-time feedback to a website..." ### Explaining by narrating a user scenario Abstract features are grounded in concrete "you talk / you click" walkthroughs: > "a website opens, you talk to it like a human, and while you're doing that, it fixes everything in the background" > "imagine you're talking and say, 'Hey, this should be a different color. Should it start fixing these things immediately or analyzing them...'" ### Enumerating options explicitly, then picking one A repeated decision structure — list options aloud, state a preference: > "So maybe... instant mode, smart mode, or do nothing mode. One of those three. Okay, I said it." > "Should it be Rivet or just Compound Engineering? Maybe. That's a good question." ### Generalizing from the current task to transferable advice Mid-work tangents turn into methodology tips: > "I think AI is a little bit conservative, so you need to think outside the box... And it's helpful to talk about what you would do if this was a long-term project for a year. And then from that big picture, you zoom in" > "The point is this: if everything is very clear, detailed, you can just hand it over to the agent and he will do everything in one go, and it saves a lot of time." ### Explicit quality/philosophy statements Short aphoristic claims about product and tooling values recur: > "you want the AI to be as close to the user as possible, to be fast, optimistic, and feel instantaneous" > "what I like about libraries is that they are quite difficult to update, which means that the AI just uses them as they are." --- ## 4. Communication Patterns ### Live-stream housekeeping The stream opens and continues with audio checks, ambient music mentions, and personal context: > "check, check, check. Ahem. I have ambient music from YouTube playing in the background." > "I have water, which is very important for hydration." ### Reading and answering chat questions aloud Viewer questions are restated in full before answering: > "I'm curious about how you choose who to test ideas with and how you get them to test the product-market fit. Yes, that's a very good question. Um, of course, by ourselves." ### Thinking out loud while reviewing agent output Long passages consist of reacting to on-screen text item by item: > "Okay, so no overlays. KW, knowledge work. I think because CW is already taken... Hmm, intellectual work. I like it. KW, intellectual labor. Okay, cool." ### Announcing the schedule forward Repeated forward references to next steps and future streams: > "I'll probably do another broadcast tomorrow where we'll see what it actually does." > "I think we should take a five-minute break... I'll just sit here, drink some water, and probably be back soon." --- ## 5. Stable Patterns vs. Isolated Moments **Likely stable (repeated multiple times):** - "really cool" / "cool" as approval vocabulary - "Let's see" as transition/pacing - Hedged idea-framing ("maybe," "I think") - Scenario-based explanation ("imagine you're talking...") - Re-explaining the project for newcomers - Reading and answering viewer questions verbatim **Possibly isolated (appears once or twice; insufficient evidence for a pattern):** - The extended Q&A on product-market fit and "taste" (~[2042]–[2318]) — a single substantive viewer discussion - The self-corrected name segment ("My name is Geller...") at [1482] — likely a transcription artifact; not interpretable as a delivery choice - "Steeply" ([73], [2516]) — appears twice, but given transcription noise, this may be a mis-transcription rather than a real verbal tic - The vocalized decision ritual ("Okay, I said it. I think that would be cool.") — seen once; plausible habit but unverified # Observation Record: "Experiments with Kieran, OpenAI Live Voice inside Compound Engineering" **Source basis:** Video transcript only. The speaker identifies himself as an employee of **Every**, working on the **Compound Engineering** plugin; no other speakers' ideas appear (viewer questions are paraphrased by the host). --- ## 1. Project Idea: Real-Time Voice Feedback for Websites ### Core concept - The host is building a way to give **real-time feedback on a website** by talking, tapping, annotating, or drawing, with an AI agent acting on that feedback immediately. - Motivation stated: the hardest part of working with AI is **"getting a quality signal"** — capturing moments when a user says something important. - He credits his library **Riffraff** (also called "Prefect"/"Refract" at points in the transcript) with this thesis; **Refract** is described as a React package that records screen, clicks, and DOM elements — but *not* in real time. ### Integration target: Compound Engineering's "Polish" command - A new version of the **Compound Engineering plugin** was released, featuring "prototype functions with annotations" that "automatically does cool things." - He wants to add a **live mode to the `polish` command**: "a website opens, you talk to it like a human, and while you're doing that, it fixes everything in the background." - His framing of polish: "it's what you do when you take AI-generated content and improve it... watching and using things for me is the best way to give feedback because you experience it firsthand." - He states he wants to use **OpenAI's newly released live audio API**, which he previously used in a "breathing exercise trainer" app. --- ## 2. Brainstorming Decisions and Positions (recorded during the session) - **Who is the user of v1?** Not end-users of an app (as in Refract), but **Compound Engineering users themselves** — "we need to change that direction a little bit." - **When does the coding agent wake up?** He likes *both* instant action and bundling: "I want to see a living board... and then let the coding agents [act] when the right boundaries or scope" are reached — possibly via a tool call where the front-end agent decides it's time to send. - **What does the coding agent do?** Proposes **three interface modes**: *instant mode*, *smart mode*, or *do-nothing mode*, manually switchable. Rationale: small design tweaks can run freely, but larger refactorings "can conflict." - **Evidence capture:** "As much as possible" — periodic **screenshots** rather than video ("too big"), plus structured/DOM event data, so "the agent is completely clear." - **Drawing layer:** **TL Draw's SDK was rejected** (licensing/watermark, not recommended for internal overlays); he favors a **thin layer on top of Perfect Freehand** (by the TL Draw author). - **Scope discipline:** When the agent proposed building just "one fragment" to be usable next week, he declined: "No, I don't want to do one fragment. I always want a full cycle." - **Time economics of brainstorming:** Brainstorming took "maybe an hour, an hour and a half," but "if everything is very clear, detailed, you can just hand it over to the agent and he will do everything in one go, and it saves a lot of time." - Open questions he leaves unresolved: whether the feature lives in **Rivet** or Compound Engineering ("Should we add this to Rivet as a mode?"), and tunnel/remote-machine support ("I think it's supported there too" — tentative). --- ## 3. Claims About AI Development Practice - **AI is conservative:** "I think AI is a little bit conservative, so you need to think outside the box when you say, 'Hey, I actually want to use this in a different way.'" - **Zoom-out prompting tactic:** It helps to describe what you'd do "if this was a long-term project for a year," then zoom in — "it knows the direction... maybe abstractions where you think, 'Hey, maybe this should be a library, not a script.'" - **Preference for libraries:** Libraries are "quite difficult to update, which means that the AI just uses them as they are," providing "a clear division of responsibilities." He likes libraries that "develop organically" and are reused across projects. - **On AI agents' scope-pushing:** "This plan would take 6 weeks, and then he does it in 2 hours... I like that he insists on scope" — but with clear specs, agents can execute in one pass. --- ## 4. Tool Endorsements - **Monolog (by Naveen):** Used to dump notes during a hike; "I love it. I have it on my Apple Watch, on my iPhone." - **Cursor (especially "Projects"):** "They really took some of the Grokbot philosophy and added it to Cursor." He praises its combination of "always-on cloud but also local work." Notable self-observation: "I spend a lot more tokens than I did before projects, which is good or bad, who knows." Projects give access to all runners/workers across his Mac Mini and the cloud, and it is **multi-repository**. - **Speed/feel as a design goal:** AI should be "as close to the user as possible, to be fast, optimistic, and feel instantaneous, like Cursor." --- ## 5. Product/Testing Philosophy (answering a viewer question) - **Test with people of taste, not everyone:** "Of course, by ourselves. You have the best taste... if everyone gives product reviews, it probably isn't very valuable, because not everyone has good design or product taste." - Suggested tester pools: trusted internal employees, experienced users, or top-tier plan customers — "use such people as a kind of leverage." - **Finding people with taste is "the hardest thing":** "honestly, if you don't have taste, then... it's just hard." - **A "new version of open source":** users who know the product can "create and do things themselves" rather than contributing code; "I think this is increasingly becoming the future." - He explicitly qualifies: "I don't have a definitive answer." --- ## 6. New Framework Idea: "Compound Work" (Work for Non-Engineers) - Working title **"Component Work"** (later "Compound Work" / "KW, intellectual work" after discovering "CW" conflicts with Katie's Compound Writing): "Compound engineering, but without engineering. Intellectual labor." - Motivation: colleagues at Every (Katie, Jack, Mike, Natesh, Dan) each have "a bit of their own style," inspiring a framework for personal intellectual-work workflows. - **Position evolves during the video:** he was "hesitant" about one unified tool, but concludes he is "really leaning towards making it completely separate from Compound Engine," while acknowledging the tension: "Oh, but brainstorming is very useful in Compound Engine... well, then install Compound Engine too." He settles on: "Compound Work is not on its own, but it could be, but it has to work together with Compound Engineering." - Skill ideas noted: **Capture** (processing raw transcripts/notes), **KW experiment**, review/source, and a perceived gap: "apparently the most important thing is missing — KW compounds," i.e., accumulating knowledge from dreams/reviews ("What have you learned? And did we miss anything?"). - He proposes brainstorming against Compound Engineering "without copying verbatim... because you can use both." --- ## 7. Explicit Qualifications and Unresolved Tensions - **Tentative status throughout:** "I'm just figuring out if it makes sense. I think it's an interesting idea, and it's going to work" — asserted, but the location ("where do you put all of that?") remains open. - **Conflicting naming:** transcript alternates between "Riffraff," "Refract," "Rivet," "Prefect," and "Reframe" for the library/tooling; positions are tied to this evolving naming. - **Token spending ambiguity:** Cursor Projects' cost increase is left explicitly unresolved ("good or bad, who knows"). - **Session outcome claimed but unverified:** "I think it works... I'll probably do another broadcast tomorrow where we'll see what it actually does." # Video Attribution & Provenance Record **Video:** *Experiments with Kieran, OpenAI Live Voice inside Compound Engineering* **ID:** `01a0f15a-45bd-7409-9c12-74c8b051b0a8` --- ## Speakers & Roles | Speaker | Role | Evidence | |---|---|---| | Single unnamed streamer | **Creator / host** — live-stream coding session; identifies as an employee of **Every** ("I work at Every") | Self-identification in transcript; title references "Kieran," though the transcript never confirms whether the speaker is Kieran — **uncertain** | - No guests, narrators, or additional speakers are identifiable in the transcript. - The transcript is garbled/auto-transcribed: the speaker appears to say "My name is Geller" but this is inconsistent with the title; speaker name is **uncertain**. - Viewers on stream are addressed ("if anyone doesn't understand... please post them," "Brandon, I agree"), implying live chat comments existed; viewer questions are paraphrased by the host (e.g., a question about choosing testers / product-market fit) but their text is not directly available. --- ## Referenced Sources & Third-Party Claims Claims below are the speaker's own statements, not verified facts. - **OpenAI live voice / real-time voice API** — reportedly "just released"; core technology used. - **Compound Engineering plugin** — speaker's own product; new version released with prototype functions and annotations. - **Riffraff / "Refract"** (transcript varies: Refract, Reflekt, River Wreck, Rivet) — speaker's library for capturing quality signal; transcript spelling is inconsistent, **uncertain naming**. - **Monolog** — note-capture app by "Naveen"; speaker endorses it (used on hike, Apple Watch/iPhone). - **Cursor** — speaker praises "projects" feature, cloud/local runners; claims Grokbot-style philosophy incorporated. - **TL Draw** — SDK considered for drawing layer; speaker reports TL Draw "does not recommend" its use for internal overlays (SDK/license key, watermark). - **Perfect Freehand** — by the TL Draw author; chosen as an alternative drawing layer. - **Codex** — mentioned as having a "live mode" and web MCP. - **Colleagues at Every** — named in passing: Katie, Jack, Mike, Natesh, Dan; "Compound Writing" attributed to Katie. --- ## Promotional Segments - **Every subscription pitch** (flagged): the host states Every is "the only subscription you need to stay on the cutting edge of AI," displayed via a "beautiful scrolling line." Promotional for the speaker's employer. --- ## Self-Promotion / Product Context - Session documents building a **live-voice polish mode** inside the Compound Engineering plugin (talk to a website, annotate/draw, agent fixes issues). - Announces a new framework in progress: **"Compound Work"** — intellectual/knowledge work for non-engineers, possibly standalone or bundled with Compound Engineering; naming unresolved ("Compound Writing" taken by CW). --- ## Availability & Comment Access - Comment channel: live stream chat implied; host reads/responds to viewer questions in real time. Persistent comment availability (e.g., a comments section) is **not confirmed**. - Follow-up promised: "another broadcast tomorrow" to review agent results — availability of that recording **not confirmed**. - Speaker notes low battery, stating the broadcast will stop when it runs out; recording end point **uncertain**. --- ## Provenance Summary - All technical claims originate from the **single host**, working live in **Cursor** with AI agents (locally on a Mac Mini and in the cloud). - No external citations, documents, or data sources are formally referenced; all product opinions are first-person. - Transcript quality is degraded (mis-transcribed names: "Rivet/River wreck," "Geller," "composite engineering" for Compound Engineering), so proper nouns should be treated as **low-confidence**. Hello everyone. Hello everyone. It seems to be working. Okay, check, check, check. Ahem. I have ambient music from YouTube playing in the background . Let me know if you can't hear me. I think you should hear me. Ahem. I just wanted to work on something a little and share it. Because I have interesting ideas , and sharing them is fun. So I'd like to, um, discuss a few things here. Let's see. And I'm also trying out this streaming format. This is probably better for readability. So, that's probably a good thing. Okay, cool. Steeply. So, um, actually, we released a new version of the plugin, the component engineering plugin, which is actually really cool. There are some interesting new features here. New prototype functions with annotations have appeared. If you add an annotation, it automatically does cool things. This is really cool. The prototype is what we started with, I did a few iterations. More flexible, compact, and very cool packages, which we should talk about at some point , but this is new. I am actively working on this. I'm also working on trying to adapt component engineering for non-engineers. I do engineering work, of course, but I also do intellectual work or experiments, something like that . And yes, I have my own approach, and I thought maybe it's time to separate it out. I also asked Every how others work, and everyone has their own style, which is very interesting. So I thought, "Okay, it would be cool to create a framework for this." The working title is " component work". This is separate from component engineering, and it is a stand-alone product. He can work alone. You can also use them together. This will be experimental, I'm still figuring it out today, but it's not something I want to work on right now. I just want to tell you about an idea I have. So, I have a library called riffraff. The idea behind riffraff is that the hardest part of working with AI is getting a quality signal, finding valuable data when a user says something important or you have an experience you want to capture. And you can capture this in many ways, for example, Refract is a React package that you can install, and it will record your screen, and of course all the other things, like where you click, what DOM elements are there, and all that, and then it will package it, but it's not in real time. And I, uh, experimented with real- time audio. So, I have another app that I built, something like a breathing exercise trainer, and I used the live audio feature that OpenAI just released, and it's really cool. So I'm thinking, "Hey, how do I make it so that maybe I can look at the site, react to it, click on elements, maybe draw in real time, so that the agent can start working with it right away ." And yeah, that could be cool. And I think we could maybe do that in Codex, because there's a live mode, and there's a web MCP and things like that. What's interesting is, I think maybe I could experiment with that, like in compound engineering, if you're doing Polish or something like Polish, which is a more interactive thing , there's a mode in Polish called live mode or something like that , and it starts the server, goes into live mode, and just works as an extension of Polish. I think that would be interesting to experiment with. So, yesterday I went hiking, and yeah, I uh, wrote down all my notes using Monolog. I was actually hiking. Also, look here. Monolog, I was actually on a hike. Also yeah yeah yeah, let's see. Yeah, okay. Cool. Uh, yeah, I was on a hike and I just dumped everything that was in my head into Monolog. Uh, use Monolog. It's from, uh, Naveen. I love it. I have it on my Apple Watch, on my iPhone. It's everywhere, and I just brought it all into, uh, Cursor. I love Cursor, especially the projects now. Uh, the projects in Cursor are just amazing. They really took some of the Grokbot philosophy and added it to Cursor, and also everything from all the, uh, IDEs and agent-based programming tools, and I think Cursor really captured the idea of ​​always- on cloud but also local work, because you want to have all of that together, and it's all interconnected, which is really cool. So, uh , that's why I'm here. I love projects. I spend a lot more tokens than I did before projects, which is good or bad , who knows. Okay, let's see. Let's go back to this. So I brainstormed and I just ran CD brainstorm here. Here's what it does. It checks the repositories that I talked about and the brainstorming happened here. And let's see where the questions are. So the questions here are like, " Hey, who's the person talking to the agent in the first version?" And the options here are like, " If you're doing this like Reflekt, maybe it's for the users of your app. So you're in the app and you're giving feedback." But I think the version that we're building right now is really just for component engineering. So we need to change that direction a little bit. I thought maybe everyone, and yeah, the questions were, "Hey, I said, the next question is, this defines the feel. Is a linear C agent single-threaded? When the voice agent pulls a block, when does the encoding agent wake up? So imagine you're talking and say, "Hey, this should be a different color. Should it start fixing these things immediately or analyzing them, or do you click a button and hit send? It's like an annotation in Cursor, and then you say, " Send to agent." Or you bundle them together. I think I like both, because I want to see a living board, some kind of collection of what we need, optimistically or quickly, because there's an agent in the browser so we can act on it. And then let the coding agents, when the right boundaries or scope or something like that, get a boost, and we can do that with a tool call, where the front-end agent decides it's time to send. So the next question was, what does the coding agent do when it wakes up? With a package of accepted blocks. Accepted means the application is changing before your eyes while you're asking. So yeah, so yeah , I think just sorting, so it's just looking at things, implementing little things, implements everything in parallel. By solving for each block. Actually, maybe you could set a mode in the interface itself. So instant mode, smart mode, or do nothing mode. One of those three. Okay, I said it. I think that would be cool. For example, if you're in the polishing phase and you're doing design stuff, I don't care. You can do all sorts of things. But if it's something larger or a refactoring, they can conflict . You can just manually switch to a different mode. I think that's interesting. Okay. So switching modes. That's good. Also, I want to change the plan a little bit. I want to add this to the Compound Engineering plugin in the polish command. I think that's a perfect feature for polish. Maybe it would be some kind of mode, like a "live" mode for polish or, um , maybe it's a mode using polish. It's worth asking, " Do you want to switch to live mode or traditional mode?" " And I want to lower the level a little bit so I can work faster. Maybe. Okay. That's good. And for those who just joined, I'm working on improving the polish team at Compound Engineering. Actually, it's going to add a live mode. For example, if you have a design, let me open up what polish does. Polish. So it's when you've already done something, the AI ​​has done all the work, and you're refining it. It's really convenient to see the result in front of you and just say, "Hey, move this. Do it better. Can you suggest a few options? "That's the polish team. But I like OpenAI's live mode, where you can just talk to the machine and it does the work. So I thought, why not add that here? I want to try to improve polish so that there's a live mode: a website opens, you talk to it like a human, and while you're doing that, it fixes everything in the background in Component Engineering. That's what we're working on. I'm in Cursor right now, in the brainstorming phase . I want to use my reframe library for that. I want to implement a reframe installation in your repository, and we can work on it . Let's see. Okay, a traditional development server. Okay, everything works. There 's already a function to start the development server and so on. Um, so we need to run some kind of CLI that listens or waits for interaction from the browser. Okay, the list of boards in the overlay panel, it's there. Okay, yeah, the dashboard is what I started, but let's focus. Yeah, it's good, live or traditional environment. Runtime. It obviously needs an OpenAI API key and a microphone. Um. Yeah, okay, let's see. Overlay delivery, extensions. Drag extensions to the desktop ribbon . Character. Okay, yeah, that's the question. Rivet used to do it differently because it was putting everything in a zip file. And now it's streaming, so um. So the question is , should it be Rivet or just Compound Engineering? Maybe. That's a good question. Should we add this to Rivet as a mode? Because it's a different type of mode, right? I like not having to keep everything in Compound Engineering, and I'm also looking forward to updating Rivet to work in non-code sessions. It's streaming delivery instead of a zip file. So that's how I would phrase it. And also for the sake of improvement, can we see how we do this um how we do this in the Compound Engineering prototype , because we have a similar pattern for the server and the listener. So you can see in the latest version Compound Engineering, that we have a prototype. And we have a lightweight web server. Um, and this one kind of, yeah, runs a web server and also binds and listens to it, so it's like streaming. So you can start the server, then open your browser, and then we can get information from any website into the agent, and the agent can take actions based on that. So that's already in the prototype, which is really cool in the final release. Um . Okay, so So right now I'm just figuring out if it makes sense. I think it's an interesting idea, and it's going to work. The only question is where do you put all of that? And usually AI is -- I think AI is a little bit conservative , so you need to think outside the box when you say, "Hey, I actually want to use this in a different way." And it's helpful to talk about what you would do if this was a long-term project for a year. And then from that big picture, you zoom in, because it knows the direction, it knows maybe abstractions where you think, "Hey, maybe this should be a library, not a script." Uh, something like that. And what I like about libraries is that they are quite difficult to update, which means that the AI ​​just uses them as they are. But it is also a clear division of responsibilities. So I like to create libraries that I use in many projects. And it's good if they develop organically. And I think that's why Cursor projects are so good, because the projects have access to all the runners and workers that I have on my machine, on my Mac Mini, in the cloud. So they actually have access to all the source code of all the projects. So if you say, " Hey, look over there." as a project, it can just look anywhere. So, yeah, that's really cool about projects. So I'll actually get to that now . There is also this overview of the project, which is interesting. So he kind of keeps track of the things he does. Ahem. Yes, that's cool. Okay, so let's move on from ...so, what's here? So, okay, new form in place, rift live mode, accessibility, stream instead of zip, drawing layer, drawing layer. Oh, yes. Oh yeah, can we use TL Draw for that drawing? I think they have an SDK. Can we use this here ? Um, you can also multitask here, which is great. Um, let me actually just open this. Okay, let's review this. So I love looking through all of this. Ahem. Okay, so let's move this along as intended. Okay, the stream contract becomes the new public API. Like the ZIP scheme today. So ZIP is now an API. Ahem. yes , a flow contract. Interesting. how does he do it I wonder what this stream contract is. Is this a survey ? This...How do we do this now with a prototype? Just keep it as simple as possible, yet elegant. Um, so no LM calls in Prefect, no API, and everything is rewritten. Yes, so there are API keys because we need OpenAI in real time there. Um, that's totally normal. I think this is a good change. It's just something I said at the very beginning. I said Prefect is just a fascination and intelligence, but the whole point is that you want the AI ​​to be as close to the user as possible, to be fast, optimistic, and feel instantaneous, like Cursor—it's very fast, even though it's a remote machine. Yes , it just has to feel great, and I think working locally makes that feel great. Um, Proxy is gone. React target hosts. We simply created a framework- agnostic tool. Okay, so a regular embed script is cheap. Goals—yes, okay. So if it's not React—no problem. But for now, most of my apps are on React, and most of other people's apps are too. So that's good. Also, if anyone doesn't understand what I'm doing or has any questions, please post them. I see them here, so welcome. I'm just working, doing my usual thing on a regular September day, relaxing in the backyard. The weather is wonderful. I have water, which is very important for hydration. And you're watching a stream from me, I work at Every, this is the only subscription you need to stay on the cutting edge of AI, as you can see in this beautiful scrolling line. We check the vibes. We're exploring what the future of work is, and my attempts to do that are an attempt to figure out whether it's the future of work or not. So, let's get back to this topic with real time . It would be cool if we could run this through to the end to maybe release a version. Let's see. Ahem. Yes, the screen. Yes, we definitely want the screen to be recorded all the time, and we probably need to initiate the screenshot capture. It probably doesn't have to be a video because it's too big, but if you say it should change color, we probably want to see that, and if you're drawing in a window, we probably want to see that too. Or in structured data , event data, DOM data, or something like that. The main thing is that the agent is completely clear about what it is. Um, and briefly about the essence . Greetings. For all the new users, I'm creating a way to provide real- time feedback for any... sorry if my music... My name is Geller. These are public sounds. First. Geller was there. So, um. This is new. So , yes, yes. So, I'm creating a way to provide real- time feedback to a website by simply tapping, annotating, maybe drawing, and then the agent corrects it. Since there are all these annotation tools, but with OpenAI's real-time voice API, I think, "Well, this is great ." Let's see if we can do something. And I create this inside the Compound Engineering plugin. I'm adding this to the polishing commands because polishing is what you do when you take AI-generated content and improve it, make it really high-quality and your own. And so , just watching and using things for me is the best way to give feedback because you experience it firsthand. So , that's what we do. I'm brainstorming ideas right now. Let's see how far this goes. Ahem. Let's see. So, okay. The next question is: how much evidence should be recorded with each block? Transcripts. Yes, Rich Lin. Yes, for evidence — do as much as possible. Apparently, there's no live video, but we should definitely take screenshots, anything that can make the situation as clear as possible for the agent. We can also do optimization to see if something gets better or more confusing. Maybe we'll do it later as an experiment. We should work on this too. Yes, this sometimes happens for a reason. For some reason my keyboard shortcuts no longer work. So, how much evidence should be recorded? I think, as much as possible, exactly screenshots. We should take screenshots periodically and... Mhm. This song is actually called... conducting experiments to see if adding or removing certain things is better or worse. I started this song at one of my last public events, so I like to play it here. Okay, we're going to give... Okay. Stop talking. Yes. OK. Ahem. So, Tldraw does not recommend this for internal overlay, SK4, license key, free key with watermark. Got it. Okay, maybe it's better to put a thin layer of your own on top of Perfect Freehand. Of course, from the author Tldraw. Okay, of course. Yes, that sounds like a good option for Perfect Freehand. Let's go back to the flow contract, not a survey in the literal sense, namely prototype three. Okay, page. So, that's how we get information from...the website we're testing, for the coding agents. Can you make sure we can run this through the tunnel as well? Because we can run this on a remote machine and want to open it on another machine. Just make sure it works the same as in the prototype. I think it's supported there too . So, as you can see, I like to browse and add things already. I know I'm not exactly answering the question below, but it's a little faster this way. Next question— I'll do the first option. Good. So, if the first option was to be worth using next week, take just one fragment. No, I don't want to do one fragment. I always want a full cycle. This is one of the features of an AI agent. It's good that they deny it, but usually they can just do anything. This is a modern version of the plan. This plan would take 6 weeks, and then he does it in 2 hours, like this. I like that he insists on scope, but if everything is clear and you know exactly what you want to build and everything is well set up. That's good. Okay, so shortcuts. Okay, that's what I did. Ideally. Blah-blah-blah-blah. Okay, let's see. Or in brainstorming, document review. Good. So, I think it's brainstorming, and that takes the most time. As you can see, it probably took me, with initial thoughts, maybe an hour, an hour and a half total or so. But the point is this: if everything is very clear, detailed , you can just hand it over to the agent and he will do everything in one go, and it saves a lot of time. Cycles, iterations, immersion in unnecessary details, and so on. Therefore. So, now he keeps it . You see, it will probably be fine there, in the river repository. What I also like about projects is that it's a multi-repository thing. Initially, there were four repositories. I narrowed it down to two: compound engineering and river wreck. But I could definitely say, "Oh, I actually want to implement this in another repository for users at some point." Hmm . Therefore. Ahem. So, what do you do while you wait for it? I was thinking. Sometimes I go for a walk. Sometimes I go downstairs and play a little piano. Ahem. or you switch to another project. Well, like here. Um, come on, come on, let's see. So, I've been working on something else here. Uh, we also have a question. Let's take a look. If people have questions, please ask them. Uh , let's see. Eh, let's think about it. So, what you're doing is great for integrating into your presentation, but I'm curious about how you choose who to test ideas with and how you get them to test the product-market fit. Yes, that's a very good question. Um, of course, by ourselves . You have the best taste, and I think that applies to the products as well. And, of course, if everyone gives product reviews, it probably isn't very valuable, because not everyone has good design or product taste. Ahem. I think it's still worth giving access to people you trust, either internal employees, or experienced users, or those with the most expensive plan. Use such people as a kind of leverage. Hmm, at least it's interesting to see what comes of it. And I think it's also really important to have a way in your process to evaluate whether it's a good idea and whether it aligns with what you want to do. For example, does this really move the product in the right direction ? Is this grinding, or is it, on the contrary, destruction? And so, I think now making it as simple as possible for advanced users or for yourself is the first step. But I want to bring this to real feedback on the product so that it solves the problem for you, it's like a new version of open source where the code is open, but it requires a lot of engineering work. Here it's more like: hey, if you're a user and you know how to use the product, this is a kind of version where you can create and do things yourself. I. Yes, I think that this is increasingly becoming the future, and. Yes, I will definitely share something else . Yes, definitely. So, yes. This is my answer to that. I don't have a definitive answer, but I'm working on things that will help make sure what you do aligns with your goals and your company's goals. So yes. Okay, so the plan is being written. Hmm, let's see. Let's move here. They could write a brainstorm where everything is already taken into account. He updates the document in the sync room for us to review. I usually like to jump around a bit from one to the other. Hmm, that's draining my brain a bit . So I think taking breaks is also a good thing. So yeah. I think we should take a five-minute break. OK. I'll take a five-minute break. I'll just sit here, drink some water, and probably be back soon. And yes, Brandon, I agree, I don't have a ready answer, but if you create restrictions... in general, the main goal right now is to find people with taste, because that's the hardest thing. And honestly, if you don't have taste, then...yeah, it's just hard. Users are usually people with taste or experienced users because they use your software and have a deep understanding of how it works because they use it themselves. So I would say that anyone who has been using your software for a long time would probably be a great candidate for collaboration. Okay, I'll be back in 5 minutes to see the results of the brainstorming. Yes. Steeply. So one of the nice things is you see agents and cloud launches here. You can run it on my computer, you can run it anywhere. And you can click on that. You can even manage agents right here. I like this pattern. Ahem. So, he writes a plan. OK. Multitasking is also very good. Multitasking, you can just say: do this, do that. It can manage agents that are already running. He can restart them, all that. Ahem. Good. Let's see. We want to see if the results of the document review come in , ask questions, or automatically resolve them if there's a good idea, but overall, it's good and... Can you launch the planning of the complex design when this is complete? Once all this is done, run the planning so that we have all the nodes, as well as partitioning by postgres, etc. And then we have a plan here . And after that, can you run the LFG thread for each node separately and, yes, do as much parallelization as possible ? We'll see here on the right, he'll start doing more manipulation. You can see This is interesting because here you can see: this is running on a computer, my Mac Mini. This one is in the cloud. You can see what he's doing. Ahem. I think it just works. Ahem. Normally I would switch to something else. I think it's set up to actually complete everything. Hmm, I think I'll just do the next post to see how it goes. Um, maybe even tomorrow in streaming format. Um, let's dig a little deeper into Compound Work, which is also very interesting. This is, um, yes. Compound engineering, but without engineering. Intellectual labor , and I have a version that works locally, but having it at, let's say, scale as well. And that's cool because different people use different things. Here, for example, Katie, Jack, Mike, Natesh, um, Dan, all the guys, they have a bit of their own style, which is really cool. So you can be inspired by what they do. Um, or create something completely your own. And I'm really leaning towards making it completely separate from Compound Engine, because I was hesitant . I thought I wanted to have one thing that did all the work, but if you're not an engineer, the form of work is completely different. And you can install both, but then the question arises: "Oh, but brainstorming is very useful in Compound Engine." " Hmm, well, then install Compound Engine too." Let's use them together. So, um, I think I want to make the Compound Work plugin, um, separate. So maybe we should call this the Compound Work plugin for GitHub, like Compound Engineering, or maybe like Compound Writing? Because we also have Compound Writing by Katie. Hmm, I still love looking at those plans. If it's a big feature—okay, but if it's just "add a feature, make it work, " then that's YOLO. But these are really important decisions, and I want to understand exactly what I want. So it's also so that I can get inspired. Let's see. The form is defined. So , the repository, the separate connection to Compound Engineering—that's um, yeah. So, a complementary ecosystem for useful work. Work is not autonomous. Um, brainstorm, plan, review documentation. Ahem. Yes, you install both. It could be standalone, but I think the concept is...So the concept says Compound Work is not on its own, but it could be, but it has to work together with Compound Engineering. I think that's the concept. Okay, so no overlays. KW, knowledge work. I think because CW is already taken by the title Compound Writing. Hmm, intellectual work . I like it. KW, intellectual labor. Okay, cool. Um, Compound Work. Yes. Mm. Okay, so what are the skills here? Let's see, let's see if we need to add this here, actually. LSC The setup is good. Capture Capture So if you have raw transcripts, meeting notes, monologue material, for example, Capture will process it and put it in the right place. Uh, Knowledge Work Dream, um dreams, um, sums it all up. We need to I need to figure out what they do and how they actually work. Um KW experiment M-m. Review and source. OK. This is not good. Okay, let's take a look at the skills presented here. I think you can look at my example of time for a plan, a day, a week as well. I think we should also add something like composite engineering or strategy. So, we can also have a strategy for intellectual work. Intellectual work is experiments. Um, not sure if this is necessary, but uh, apparently the most important thing is missing—KW compounds. For example, if you dream or do a review, this can also accumulate knowledge. Because, what have you learned? And did we miss anything? It might be worth brainstorming on composite engineering to see if we're missing any skills in this area. The thinking about this also took place completely independently of composite engineering. But also without copying verbatim from composite engineering, because you can use both. Okay, the main point is that uh, I wrote a piece about the folder being an agent, and oh, yeah. Oh, yes, I'll go. Uh, someone says my battery is very low. Yes, I'll stop the uh broadcast when the battery runs out. Uh I Yeah , so basically we ran the whole uh live voice within the composite engineering here. It will work, execute LFG, and build everything. I'll probably do another broadcast tomorrow where we'll see what it actually does. Uh, I think uh, it works. Um, I'm working at M-um right now.