Conference talk by Onur Solmaz (OpenClaw) on scaling AI coding agents on Kubernetes. Covers acpx, a headless CLI for the Agent Client Protocol (ACP) that replaces PTY scraping with structured agent-to-client communication and drives sessions through a node-based workflow graph (bug reproduction, implementation judgment, conflict checking, review loop, structured JSON output). Describes running parallel Codex sessions via Discord channels and a Go operator that provisions disposable per-task agent pods on Kubernetes, wires them into Slack, and tears them down when complete. Framed by the problem of handling hundreds of largely unmergeable AI-generated pull requests per day at OpenClaw. # Channel Presentation Observations ## Intended Audience - **Developer/technical audience familiar with AI agents and Kubernetes**: talk assumes knowledge of Codex, Claude Code, MCP vs ACP, Kubernetes pods, Helm charts ("It's an A10-like workflow engine... driving a Codex session"). - **Enterprise-oriented developers and open-source maintainers**: "we're used to selling the enterprise for a few years now"; "One of my goals is accelerate enterprise adoption of OpenClaw and adjacent software." - **Open-source project maintainers** receiving high PR volume: "If you have a system... of hundreds of issues per day, I can help you process it." ## Claimed Scope / Self-Description - Speaker identifies as a **founding engineer at a startup (TextCortex)** and an **OpenClaw maintainer**: "I became a maintainer along the way. I was there when it was renamed two times." - Focus stated: "My focus is on agent interoperability and orchestration." - Claims longevity in the space: "I've been building harnesses since a few months before ChatGPT came out." - Explicitly separates affiliations: "I just want to make clear... the OpenClaw side and TextCortex side... the last part was TextCortex." ## Observable Presentation Patterns - **Conference-style technical talk with live demos**: screen-shares PR review workflows, Discord-driven agent usage, and a Kubernetes-orchestrator UI demo. - **Project showcase**: introduces ACPX ("slowly turning into a Swiss Army knife for ACP") and a TextCortex open-source orchestrator ("The repo is at textcortex"). - **Practitioner credibility through personal workflow**: "I'm running a full IDE on Discord... I'm working with one to five agents." - **Community engagement framing**: cites Zed, "shoutout to Zed," references Peter's earlier talk, credits Harold as "another maintainer," Vincent's extension, and shout-outs to Cognition/Devin. - **Honest, informal delivery**: acknowledges unpolished tooling ("we are developers and our tools we don't have time to polish them very well") and learning in public ("I'm also learning along the way"). - **Offer of direct help to audience**: "If you have an open-source project, first of all, I can help you set this up." ## Explicit Claims vs. Demonstrated Content - **Claims**: ACPX exists as a workflow engine; product "running on Kubernetes"; open-source repo available. - **Demonstrated**: recorded workflow replays ("It's reproducing the bug. It's judging refactor"), demo of Slack error-reporting dispatch, and description of a web UI hosted in the cluster — though the demo is narrated rather than fully verified in the transcript. # Delivery Observations — "Scaling Agents on Kubernetes with acpx and ACP" ## Overall Tone - **Casual, conversational, improvised** — filled with filler words ("um," "uh," "you know," "like") throughout, e.g., *"So, the talk is building an ICP on uh at Open Clo. It's uh also about other things."* - **Self-aware and modest** — admits limitations directly: *"we are developers and our tools we don't have time to polish them very well"* and *"I'm also learning along the way."* - **Enthusiastic, anecdote-driven** — personal stories dominate the opening: *"my mind was blown... everybody else's minds were blown."* ## Recurring Vocabulary & Phrases - **"Ship" / "shipping"** as a positive framing: *"you just get it ready. You ship it"*, *"you build it once and then you you ship it."* - **"Slop"** (and "AI slop") as recurring criticism: *"you can't merge it... they just send some slop your way"*, *"without creating AI slop."* - **Repeated rhetorical build-up phrase "So,"** introducing nearly every new topic — appears dozens of times ("So, that's how it started," "So, uh it started by..."). - **"You know"** as a constant connector, appearing in most paragraphs. ## Teaching Habits - **Contrast-based explanation**: clarifies concepts by distinguishing them — *"MCP is for giving tools to the model. ACP's for... standardizing agent to client interaction"*; *"agent to agent protocol, that's for agent talking to agent. ACP... is for human talking to agent."* - **Metaphors and analogies**, repeated at least twice: - *"a ship of Theseus"* (describing his harness evolution) - *"a Swiss Army knife for ACP"* - *"it's like an ointment that you apply generously on any problem that can be solved with agents"* - *"I was playing the telephone game. I was telling Opus to tell Codex to do something."* - **Demo-driven**: pivots from slides to live playback — *"Let me show you in action... Let's speed it up a little bit."* - **Repetition of key sentences for emphasis**: *"That's what I'm working on"* / *"that's what I built into ACP-X"*; *"What is this? ... You ask, what is this."* ## Communication Patterns - **Stream-of-consciousness transitions** — topics flow loosely with "So," "Um," and "And then," rather than signposted structure. - **Frequent mid-sentence corrections and restarts**: *"I I I hope uh to I will today share in a nice way"*, *"It's not exactly It's like so much wasted work."* - **Time management awareness near the end**: *"I'm running low on time. So, the repo is at text cortex"* — then rapid acceleration through closing material. - **Disclaimer habit**: separates affiliations explicitly at the close — *"I just want to make clear like of my... OpenClaw side and TextCortex side... just to give a disclaimer."* - **Direct audience offers**: *"If you have an open-source project, first of all, I can help you set this up."* ## Uncertainty Notes - Filler-word and "So," frequency are **clearly stable patterns** given transcript-wide repetition. - Some apparent mis-transcriptions ("Open Clo/Cloak/clone" for OpenClaw, "A10-like" possibly "n8n-like") limit certainty about exact wording in those spots. - A single talk cannot show whether the demo-heavy, anecdotal style is habitual across talks; these observations describe **this delivery only**. # Position Record: Scaling Agents on Kubernetes with ACPX and ACP — Onur Solmaz (OpenClaw) ## Speaker - **Onur Solmaz**: founding engineer at a startup (TextCortex), OpenClaw maintainer, focused on **agent interoperability and orchestration**. ## Background / Credibility Claims - Building agent harnesses since before ChatGPT; built a Jupyter Lab extension over the original Codex (DaVinci Code 2). - OpenClaw user since "Clo Bot" first dropped in Discord; became a maintainer; added MS Teams integration; witnessed two renames. ## Position on ACP vs. MCP - **MCP**: for giving tools to the model. - **ACP** (Agent Client Protocol): for standardizing **agent-to-client interaction** — "human talking to agent." - Distinction from A2A: agent-to-agent protocols are for agents talking to agents; "agent can use the human one to talk to other agents as well." - Rationale for choosing ACP: practical, not ideological — "when I needed the most, I needed adapters for Codex and Claude Code and only that had built them and Google didn't have them back at the time." - Claim: per-editor plugins (VS Code Codex, Claude Code) are "so much wasted work"; standardization means build once, ship once. Shoutout to Zed. ## Position on PR Triage / "Automating the Automator" - Context: OpenClaw has **60K+ PRs total, 300–500 opened per day**; biggest challenge is absorbing and balancing tens of thousands of stakeholders' needs without "creating AI slop." - AI-generated PRs can't be merged but "you can't fully discard it. You need to take this data point" — they indicate broken code. - Result: built abstract PR workflows (find intent, judge implementation, resolve conflicts, make CI pass, review/refactor loops) so mechanical work is resolved before a human reviews. - **Qualified defense of agent loops**: running an agent in a loop "doesn't necessarily have to be something that will create slop" — fine for "shallow bugs" and superficial refactors; fundamental refactors should be escalated to humans. Describes "standard operating procedures for agents — a fancy word for workflow." ## Position on Enterprise vs. Personal Agents - Enterprise agents will consume far more inference → more money to be made there; this motivates his enterprise focus. - Believes in **on-demand disposable agents**. - **Unresolved limitation he flags**: chat platforms (Slack/Teams/Discord) lack standards for **multi-agent provisioning** (separate cosmetic agents per app manifest); "this must be supported for this vision to work." Until then he uses a separate UI. ## Architecture Claims - Key components for one-agent-per-task: **Kubernetes**, an agent harness (OpenClaw, Codex, Claude Code), read/write access, and state synchronization (rsync-like / Dropbox-like algorithms). - Using a full Kubernetes pod per agent is "wasteful, but I think it's the better abstraction" — echoing the claim that giving an agent a full computer is more powerful. - Acknowledged gaps: "not so well versed on all the different virtualization frameworks" (notes OpenHands uses Firecracker); "I'm also learning along the way." ## Products Discussed - **ACPX**: an "n8n-like workflow engine" driving Codex sessions; started as a CLI for calling agents over ACP; "slowly turning into a Swiss Army knife for ACP." - **TextCortex orchestrator** (day job, open source, repo: textcortex "sprits" as transcribed): goal operator handling Kubernetes wiring, Slack concierge UX, agent dispatch for error reporting; built with ACP so "you're not locked in any agent." ## Explicit Disclaimers & Qualifications Preserved - "It'll be secure... Peter talked a lot about that earlier... It's work in progress" (on OpenClaw security). - Claude Opus "was not so reliable" for complex tasks back then; "It's now better." - Clarified the last portion of the talk was TextCortex work, separate from OpenClaw, "just to give a disclaimer." - Skipped shout-outs to Cognition/Devin ("they invented the category") due to time. # Speaker Attribution Record — Scaling Agents on Kubernetes with acpx and ACP ## Video Metadata - **Title:** Scaling Agents on Kubernetes with acpx and ACP — Onur Solmaz, OpenClaw - **Video ID:** `01a15ca9-4468-78fd-8a22-b6fb5e613d12` - **No curated metadata supplied** beyond title; no comment availability information present. ## Speakers ### Primary Speaker - **Onur Solmaz** — **Creator / presenter / sole talk speaker** - Self-identified roles stated in the talk: - **Founding engineer** at a startup (identified later as **TextCortex**) - **Maintainer** of OpenClaw (became maintainer "along the way"; present through two renames) - Focus: **agent interoperability and orchestration** ### Referenced Third Parties (mentioned, not speaking) - **Peter** — referenced as having "talked a lot about that earlier" (security) and as author of "Clo code is like computer" - **Harold** — "another maintainer," developed the **Codex app server protocol** - **Vincent** — referenced as having "talked about that a little bit" and shown via a code extension demo - **Audience/hosts** — implied listeners; no speaking lines beyond the closing "Thank you." ## Promotional Segments (Flagged) - **TextCortex product segment** — the final portion describing the open-source orchestrator ("the repo is at text cortex sprits"), Kubernetes demo, and offer to help set it up. The speaker himself gives an explicit disclaimer: > "the last part was TextCortex about the work I do at TextCortex. Uh just to give a disclaimer." - **Social links** — "Some social links in case you want to get in contact with me" at the close. ## Referenced Sources & Third-Party Claims - **ACP (Agent Client Protocol)** — speaker credits **Zed** ("shoutout to Zed"), building a Rust editor; contrasts ACP with **MCP** and with agent-to-agent protocols ("competing standards"). - **OpenClaw / Clo Bot** — Discord deployment history; "over 60 K PRs total," "300 to 500 [PRs] per day on average" (speaker's figures, unverified). - **Cognition / Devin** — shout-outs explicitly skipped due to time ("they invented the category"). - **Open Hands** — speaker notes it "uses firecracker" while disclaiming his own limited knowledge of virtualization frameworks. - **Platforms mentioned:** Slack, Microsoft Teams, Discord, GitHub, Codex, Claude Code/VS Code plugins, Kubernetes, Helm charts, rsync/Dropbox-style sync. ## Provenance Notes & Uncertainty - Speaker names **Harold**, **Vincent**, and **Peter** are given by first name only; no full identities supplied. - Startup name ("TextCortex") and repo ("text cortex sprits") are phonetic transcriptions; exact spelling unverified. - Statistics (PR counts) are **speaker-reported claims**, not independently sourced. - **Comment availability:** not stated in the supplied material. Welcome. Um So, the talk is building an ICP on uh at Open Clo. It's uh also about other things like how to uh put open source agents on open source agent frameworks on Kubernetes and stuff. So, I'm I'm I hope uh to I will today share in a nice way what I've been working on in the last 2 months. Um A little bit about me, very brief. My uh A little bit about me. I've been building harnesses since a few months before ChatGPT came out. I built a Jupiter Lab extension over OG Codex model like DaVinci Code 2. Um back in the days that eventually So, I'm currently working for a startup um that that initial coding harness turned into its current harness over time like a ship of Theseus. It got ripped apart and put back together so many times. And uh yeah, I'm a founding engineer there. I've been uh in the industry since 3 and 1/2 years. I've using Open Clo since Clo Bot uh first dropped in Discord. I went in there. I've been following Peter since he wrote a Clo code is like computer. I was like w- this guy is crazy. Uh because uh Clo forced on it like I wouldn't give my machine to it, but he he went he ventured for it and he's paved the way for us. And when I saw uh Clo Bot in Discord, my mind was blown. The next day I installed it uh at the company uh at at in our like uh cluster. Uh it was basically talking to people on Discord and also everybody else's minds were blown. And we were like we have the we're used to selling the enterprise for a few years now. So, that's why I started by adding in MS Teams integration in case it might be useful at some point. And I became a maintainer along the way. I was there when it was renamed uh two times. Um and today I I my focus is on agent interoperability and orchestration. Um like one of my goals is accelerate enterprise adoption of Open Claw and adjacent software and also address, you know, Open Claw's not secure. Well, it'll be secure or Peter talked a lot about that earlier, so I don't have anything else to add on top of it. Like it's work in progress. Sorry. I'll use this. So, uh it started by I started on develop workflows right away, so I created a PR. Uh and then call it Discord driven development. Well, Telegram driven development suits better because it's TDD, so uh and then uh right after I set it personally and I realized, you know, Opus is not so reliable for complex it wasn't back in the days. It's now better. It's agents are improving, but Codex was main my main harness and I wanted to use Codex in Discord. But it was basic I was telling I was playing the telephone game. I was telling Opus to tell Codex to do something. God knows what it's saying, you know, because wording matters when prompting, but it was working somehow and I would go uh and then look at the Codex session and then it paraphrased kind of what I was saying, but eventually, you know, uh I got some stuff done, but I knew this could be done much easier. Um and today I'm uh running a full IDE on Discord like uh you know we have parallel workloads. You have like one to five like five channels at any point I'm working with one to five agents. Um so you see Codex one to five and then close it for testing the open close ACP feature. And then the basically that's how my that's how my channels look like. And it's very good for coding on the go because I have I'm a guy who's addicted to side projects and uh I I like to just you know uh AI is making thing making things a lot easier to do these sort of things like in parallel you have an inspiration you execute on a weekend and you just get it ready. You ship it. ACPX which I'm going to talk about is is like similar. I built it through uh Discord. And what you do is uh you bind the channel uh Discord channel to Codex through ACP. You can also use Codex app server protocol. Harold another maintainer uh developed it. And here it is me using it before I flying to London to create an PDF about like convert the docs of ACP into PDF. And then I have to say put it in temp because open the uh Codex doesn't know about the harness and it cannot send me in Discord. So I then I go to another channel and tell it to send it to me on that channel. So we are developers and our tools we don't have time to polish them very well but we know the like advantages and disadvantages. What is ACP? So it's not uh most people say is it like MCP? MCP for is for giving tools to the model. ACP's for uh like standardizing agent to client interaction. It's uh shoutout to Zed. I actually forgot to put Zed logo. Um Zed is building a new editor in Rust more efficient like lower memory usage not Electron. So uh I'm using Zed also since like uh last fall and uh you know, if you use Codex on VS Code or Cloud Code, they are all building different plugins. It's not exactly It's like so much wasted work. If only you could just standardize them under one interface and you just build it once and then you you ship it. And that's what the I the idea That's the idea they had. It's also more It's It's much less duplicate duplicated work. Um There are competing standards. Uh so agent like agent to agent protocol, that's for agent talking to agent. ACP, agent client protocol, is for human talking to agent, but agent can use the human one to talk to other agents as well. In the long run, as as these protocols get adopted, we will support all of them. We will weigh the you know, the advantages and then we will use them somehow. But when I need So, I chose ACP because when I needed the most, I needed adapters for Codex and Cloud Code and only that had built them and Google didn't have them back back back at the time, so that's why I chose it. Uh that So, I when you're adding in functionality to Open Clo, you do uh you use do it through a CLI. Um So, I said, "Okay, let's create a CLI for ACP. Let's Let's agent call any other agent over the command line." So, that's that's how it started. Uh And it's slowly turning into a Swiss Army knife for ACP as I will show in a bit. So, in at Open Clo, you have fire hoses. Uh we have 60 KPRs uh over 60 KPRs total. 300 to 500 per day on average are open. And basically, overnight, people woke up and decided they want they like open claw. So, we have tens of thousands of stakeholders who want to add features to open claw. And the biggest challenge of the project currently is how do you absorb all the needs and wants of these people? And how do you balance them? How do you Like you can't please everyone. And how do you create an elegant system without creating AI slop? Uh that can cater everyone's needs. And Peter's workflow gave me an idea. So, this is also something I do. You go and you ask your clanker, what is this This is where PR comes your way. And like most of the time it's AI generated description. Like what is this? Uh if the human put thought into it, great. But like yeah, you need to ask what it's doing. You ask, what is this the best possible fix? Most of the time it's no. And then you either continue this like he he wrote it. Like You you do some back and forth with the agent. The reason is people just run into an issue with their open claw. And it's it's going to use GitHub. And then they just say, "Please fix." And then they just send some slop your way. You know, this is You can't merge it, but you can also fully discard it. You need to take this data point. You It's like crucial feedback from the user. Uh so, you need to put it categorize it put it in a bin, you know. It tells you when some part of the code is broken. And Vincent also talked about that a little bit. Uh and here is uh his his code extension doing that. On one side there's the one that says it's it's a good fix. Uh and on this other side it's just saying it's not bad. And it's just so mechanical and you you once you do this over and over you realize you're repeating something. If only you could program something to automate it. So, you're automating the automator. Um so, I created I started uh to create like workflows uh in an abstract way. So, uh item comes PR and then you find the intent. You're judging implementation. You're like uh looking in if it if it has conflicts. Uh if reviews gives you like some issues that need to be addressed. So, you need to make the CI pass if it's not passing already. Most people just don't care about that. All this mechanical work by the time the main the PR ends up in front of you, should be resolved ideally and can be resolved and that's what I'm working on. Ah, in the workflow we have the shameful uh Ralph review refactor loops. Uh I I'm a believer that, you know, when people say like give the just just run a running a agent in a loop doesn't not necessarily have to be a uh something that will create slope. As long as you're not making it design something, but you're making it uncover shallow bugs that can be easily fixed should be fine. So, in the abstract workflow that I created uh which is actually called like for like turning to a program, you can uh tell it to uh do superficial refactors and then you can tell if it's it needs a fundamental refactor relate to the human. Um resolving conflicts also doesn't need like No, it was hard back in the days. I don't think anyone is like resolving conflicts by hand by now. Um So, we are basically creating a standard operating procedures for agents. That's a fancy word for workflow. Um so, that's what I built into ACP-X. Uh it's a an A10-like workflow engine. Uh but it's driving a Codex session. You you see on the right. Let me show you in action. So, this was one PR. It's loading. Let's speed it up a little bit. So, there is some programmatic parts. So, this is just replaying what it's doing. It's reproducing the bug. It's judging refactor. It's reviewing. I'm doing a review loop. And then review didn't bring anything. It It's what I do, but I make it output like JSON structured data so I can put it in like an an it's unlike workflow. And I will talk, you know, this is a general workflow engine. Engine. Um You can use it on other things as well. Um Like you need to apply agents generously on problems. So, I see it's like an ointment that you apply to generously on any problem that can be solved with agents. You need to take yourself out of the loop and solve it with agents. Um and personal agents, I think are on a like there there's spectrum enterprise and personal agent. And normally you see the work you use at the computer and then the PC the PC use at work. And the PC use at at home used to be relatively similar, but that will not be the case with agents because at at work you will be using consuming a lot more inference. And that means in enterprise there will be a lot more money to be made. So, that's why I'm a bit also excited about enterprise agents and open close a potential. That's why I I believe in on-demand disposable agents. If you use open clone slack or teams or discord, this is one instance. You create an app. The problem is you can't really talk to multiple instances of this um like you create an a connection, you create a Slack app to connect another agent and another name and another uh profile picture, you need to create another app. You have to create an app manifest. And it's it's it's something that shouldn't be managed manually by clicking. And the platforms like chat apps don't have the standard yet where you can uh mult- do multi-agent provisioning, like cosmetically create different agents. This is what This is on on the screen you see I asked ChatGPT to generate the idea. You have an agent uh and then the name can be generated by an an underlying app, and you can talk to them separately. This is not supported, and this must be supported for this uh vision to work. Until it's supported, I'm using it on another UI. Because like we are all going to start one uh agent per task, and there will be uh they will work on these tasks, and they will be creating files, editing files, and it will all be synchronized. It will be it will be uh tied with different than what you're used to with your uh personal agent. And to do to have that, you need to have co- like a few key components. You need to have Kubernetes. You need to uh have an agent harness. Open Cloak could be one of them. Codex Cloud Code could be one of them. Could be a CP Could be uh GitHub. Uh you give read write access, and you do like state data synchronization. Like you maybe you do something that's use rsync. Some something like what uh whatever algorithm Dropbox is using. And there are some projects that are uh taking this on. I've been working on this uh this is outside of Open Cloak. This is my day job. And uh it's an open source uh orchestrator. Uh it's the goal operator that's uh, basically handles the complicated parts. You know, there's a user experience. You want to create a concierge on Slack. Like concierge agent and you're talking to it, but you get bottleneck because you have 100 employees on Slack. So, you need to do for for some other task, you may you may need to create a new one and then it creates and gives you like a website link. Um, I'm going to skip the like shout-outs to Cognition and Devin um, because uh, yeah, they invented the category, but I'm running low on time. So, the repo is at text cortex sprits. I'm just going to uh, demo it for our use case. We use it on error reporting uh, currently. So, if you're on Slack, you know, uh, you can ask it to dispatch an agent to debug it, you know, you're like, "I'm uh, asking any new bugs after prod release." And it's got saying something. And then you ask it uh, to create an agent. Uh, if I could put it that agent into Slack, I would, but I can't do that. So, I have to put it in another UI. And this is an open source project. You can take this uh, UIs also uh, like a react app hosted in the cluster that you're deploying this these helm charts to. And uh, yeah, you you it starts the conversation there. It starts working on the uh, problem. This is like codex web or Devin or anything, but it's actually using like a full Kubernetes pod. Wasteful, but I think it's the better abstraction because open core showed the power, you know, when you give a full computer to an agent, it's a lot more powerful. Um, and I I believe that as well. Um, I uh, I I think uh, open hands uses firecracker. So, I'm also uh, not so well versed on all the different virtualization frameworks. Uh So, it's I'm also learning along the way, but I have a working product that's running on Kubernetes. Uh and you can use this uh product uh you can if you're interested in deploying internally and using Codex on the web. Uh and then you just spin things off if you like, I don't know, a back-end uh If if you have an open-source project, first of all, I can help you set this up. If you have a system like inlets of just 100 of hundreds of issues per day, I can help you process it. Um this is uh that does all the wiring around Slack and like keeping those agents like on and the user experience and then the the interoperability. You're not locked in any in any agent. You can switch. It's all abstract the way build with ACP. Yeah. That was my talk. Uh thank you for listening. Some social links in case you want to uh get in contact with me. I just want to make clear like of my uh the OpenFlow uh side and TextCortex side. Uh so, this the last part was TextCortex about the work I do at TextCortex. Uh just to give a disclaimer. Um thank you for listening. I guess Yeah. Well. Thank you.