Interview on the AI & I podcast, recorded at Microsoft Build, in which guest host Mike Taylor speaks with Kyle Daigle, GitHub's COO and Microsoft's chief marketing officer for developer products. Daigle explains how AI agents are expanding GitHub's user base: annual commits are projected to grow from 1 billion to over 14 billion as humans and their agents issue pull requests, including 17 million agent-created PRs in March alone. Topics include GitHub's infrastructure for an agent-native world (agentic code review, model routers that automatically select the right model per task), supporting open-source maintainers through the PR surge, implications for code quality, the shift from per-seat licensing to usage-based pricing, non-developers (legal, sales, marketing) building apps with the GitHub Copilot app, and developer choice as a durable competitive advantage. # Channel Identity Observations — "GitHub's COO Explains Why AI Hasn't Replaced Developers" ## Explicit Self-Descriptions - **Host role / affiliation**: The interviewer states, *"Hi, I'm Mike Taylor. I'm the head of tech consulting at Every"* — tying the video to the **Every** media brand. - **Channel subscription pitch**: *"Every is the only subscription you need to stay at the edge of AI… Go to every.io/subscribe today"* — explicit positioning as an **AI-focused subscription publication** aimed at people who want to track new models and tools and "separate out the signal from the noise." - **Outro (apparently AI-generated or scripted parody)**: References *"the show… the epitome of awesomeness"* and names **Dan Shipper** as *"captain of the spaceship,"* plus a call to *"smash that like button and subscribe to AI and I."* This suggests either the channel/show is called **"AI and I"** hosted by Dan Shipper, or it's a self-aware joke segment. ## Observable Presentation - **Format**: Interview content — the host sits down with **Kyle Daigle, COO of GitHub** (transcript spells "Kyle Dagel"), recorded at a conference (*"thanks for spending some time with me at the conference"*). - **Audience**: Developers and AI-power users concerned with practical costs and workflows, evidenced by framing questions like *"How do I stop my $200 a month coding agent subscription from ballooning into a $2,000 a month usage limit?"* and references to **DSPy research** and agent experimentation by the host. - **Scope claimed**: Front-line coverage of the AI/agent economy — the host frames GitHub as *"at the forefront of what's happening in this new economy"* with coverage of users, open-source maintainers, and costs. - **Editorial gimmick / demonstrated novelty**: The host discloses *"I told Kyle I had made an AI clone of him to practice the interview"* — a demonstrable pattern of using AI clones of interviewees as prep, fitting the "AI and I" framing. - **Topics demonstrated**: GitHub agent-driven PR statistics (17M agent PRs/month), open-source maintainer burden, pricing/business models for agents, hill-climbing evals, model routing, personal AI agents for self-improvement. ## Claim vs. Demonstration Notes - The **"only subscription you need"** claim is a creator claim, not demonstrated by the content itself. - The conversational, interview-driven format and technical depth (agent economics, evals, token costs) are **demonstrated patterns** in the transcript. - No channel-wide conclusions beyond this single video are drawn here; the outro's tone shift is unexplained in the source and left as observed text. # Delivery Observation: Kyle Daigle (COO, GitHub) ## Overall Style Conversational, reflective executive-interview delivery. Frequent hedging and thinking-out-loud markers: *"like"*, *"you know"*, *"I mean"* appear constantly, e.g. *"Yeah. I mean I think uh uh for GitHub in particular, we've always really had this really expansive view of what a developer is."* ## Recurring Vocabulary & Phrases - **"Developer choice"** — repeated as a core framing: *"we care a lot about like developer choice. That's always been true."* and *"That choice is core and that's something that I don't think we'll uh ever, you know, like back down on."* - **"Building for builders / for the developers"** — *"we're not building for the buyers, we're building for the developers in 100%."* - **"Hill climbing"** — reused heavily near the end: *"It is just climb climb improve new eval improve new data improve and just keep going."* - **"Mouse trap"** — used twice as a metaphor for vendor lock-in: *"Sometimes I'll say it's like a little bit of a mouse trap"* ... *"They'll just be stuck in another kind of mouse trap."* - **"On-ramp"** — *"make sure there's always an on-ramp into you know writing some software."* ## Teaching / Explanatory Habits - **Concrete first-person anecdote as evidence**: personal history *"I wrote code to pay for art school"*, weekend multi-machine coding routine *"I have my Mac, I have my PC and I have my Omar Linux box."* - **Historical analogy to explain current decisions**: free private repos as precedent for free agent usage: *"way way way back, right? You'd have free public repos, but you didn't have free private repos."* - **Name-dropping practitioners and products as credibility anchors**: Mitchell Hashimoto's *"vouch system"*, *"Peter Steinberger says, you know, 150 agents are doing everything"*, Hubot/chat-ops history. - **Metaphor-heavy style, which he self-reports**: *"like I want to use a lot of metaphors"* — consistent with observed metaphors (mouse trap, climbing the hill, "bites of the apple", "slop"). ## Structural Patterns - Opens nearly every answer with an agreement/backchannel: *"Yeah. I mean..."* or *"Yeah, I mean, like I don't think we know like yet"*. - Uses **balanced/concessive framing** — states a claim, then adds the counterweight: *"we're definitely not at the peak, but we're climbing that hill"*; *"sometimes the hard measures in Evals... show that we've made an improvement but like user sentiment will crash."* - Admits uncertainty directly rather than overclaiming: *"like I don't think we know like yet ultimately"*, *"this is an incredibly hard problem."* - Self-deprecation appears occasionally: *"which was a very silly decision as an adult now I guess"*; admitting early skepticism: *"this is like a magic parlor trick."* ## Isolated Moments (low confidence as stable patterns) - Humor/warmth in the AI-clone exchange: *"Now I want to see what AI Kyle said"* — only one instance, so not clearly a habit. - The closing outro segment (from *"smash that like button"* onward) is a stylized promotional voice-over, stylistically unrelated to the interview delivery and not attributable to the interviewee. ## Uncertainty Notes - Transcription is rough (filler words, garbled names like "cold code", "open claw", "claw"), so exact phrasing of proper nouns should be treated cautiously. - Fillers ("uh", "like") are so frequent they function as a baseline trait rather than individual emphasis markers. # Position Record: "GitHub's COO Explains Why AI Hasn't Replaced Developers" ## Speakers - **Mike Taylor** — head of tech consulting at Every, interviewer - **Kyle Daigle (transcribed as "Kyle Dagel")** — COO of GitHub, also CMO of developer at Microsoft (per his own statements) --- ## Key Claims and Positions ### Changing Developer Demographics - **Daigle**: GitHub has "always really had this really expansive view of what a developer is." He personally wrote code before calling himself a developer, writing code to pay for art school. - Non-traditional developers — legal teams, finance teams, "knowledge workers" — are building apps with the GitHub Copilot app. - Position: GitHub focuses on developers but wants "an on-ramp" for anyone who chooses to write code. ### Open-Source Maintainer Burden - **Taylor's framing**: maintainers are "drowning" in the flood of PRs. - **Daigle's response**: tools like **agentic Copilot code review** and **agentic merge** (set permissions, wait for CI/policies) reduce PR processing work. - On standards: GitHub deliberately won't impose one. Cited **Mitchell Hashimoto's "vouch" system** — many communities don't want it. GitHub will provide "building blocks of controls" and only "cement" a standard **if one emerges**. ### Exploding Agent Activity - **Daigle's statistics**: - Last October (GitHub Universe): ~1 billion commits for the full year. - This year: "on track to be 14 billion **if the growth is linear**, which it will not be." - March: **17 million pull requests created by agents** in one month. - Rejects the "slop" characterization of AI-generated code; claims the industry is "climbing the hill" past early adoption, not yet at peak. ### Business Model Under Agents - **Taylor's question**: does freemium break when agents work 24/7? Usage-based future? - **Daigle**: "I don't think we know yet." Existing mechanisms: licenses, API rate limits (where "agent back pressure" appears). He invokes **Peter Steinberger's "150 agents all at once"** as a goal to enable. Analogy: GitHub once added free private repos as fairness evolved. Enterprise needs differ from individual dev needs. ### Microsoft Dual Role - **Daigle**: 13 years at GitHub; as COO focuses on developers ("not building for the buyers"); as Microsoft CMO of developer, ensures "holistic, authentic" solutions across Microsoft. Notes Build this year intentionally featured community speakers (Peter, Swix) — "software development is a team sport." ### Competition and Differentiation - **Daigle** (concurring with Taylor that it's "the most competitive market probably"): differentiators are **developer choice** and avoiding "unintentional walled gardens" / "mouse traps." Will partner with Anthropic, OpenAI, Google, anyone bringing a model or agent. Commitment: "if we [back down], developers will still choose" — just stuck in another mouse trap. ### Internal Tooling / Dogfooding - **Daigle**: uses Mac, Windows, and an Omar Linux box on weekends; uses Copilot app only on Windows to validate that platform's experience. Laser focus alone is "myopic"; wants to know *why* developers pick other tools. ### Filtering Short-Lived Ideas vs. Long-Term Bets - **Daigle's reasoning**: - Short-term: cementing support for **many simultaneous agent sessions** ("everyone's doing it"). - Long-term assumptions: models improve; **token economics** matters more; **small language models on local devices** are near. - Persistent truth since ChatGPT/Copilot launch: **personalization / memory / context** — industry has "experiments but not a long-term vision." Ideal: an agent that "completes a thought" without the user codifying it ("Always remember I insert things 100%"). ### "Hill Climbing" - **Daigle**: Satya, Mustafa, and Jacob (Copilot group lead) use the term. Weekly review of hill-climbing results, using thumbs up/down, acceptance rates. Key tension: **hard evals can improve while user sentiment crashes — overfitting** ("100%" agreeing with Taylor's interjection). Goal: give customers the same machinery; cites **Frontier Tuning on MAI thinking one** base model with M365 data. - **Daigle's admission of prior skepticism**: initially thought frontier tuning was "a magic parlor trick"; now argues "sometimes that's where the alpha is — where it feels too simple to work." ### $200 → $2,000 Subscription Problem - **Daigle's answer**: two parts — (1) frontier-tuned models that know you; (2) **auto model routing based on task intent** (also Microsoft Foundry's API-level router). His example: after an agent does enormous work, the last step may be a "find and replace" — a user won't manually switch to a cheaper model for it, "but the tools could." ### AI Clones and Personal Agents - **Taylor**: made a terminal-based AI clone of Kyle to practice the interview; found it "immensely useful" and "spot-on." - **Daigle's reciprocal reveal**: he does something similar — an isolated Claude ("can't talk to work stuff") plus an OpenClaw he named **"Baxter"**. Daily "comms report" of patterns ("Kyle, you keep saying this"). It reads his last seven days of emails/Slack and checks whether he did what the agent previously told him. - **Daigle's claim**: humans take critical feedback better from robots than humans (traced back to Hubot/chat-ops era at GitHub); "a lot of my agent loop is really about me." --- ## Qualifications and Tensions Preserved - Growth figure (14B commits) explicitly conditional: "if the growth is linear, which it will not be." - Business model: explicitly unresolved — "I don't think we know yet." - Standard-setting: deliberately deferred; no GitHub-imposed norms. - Evals vs. sentiment: hard metrics can improve while sentiment crashes (overfitting risk) — acknowledged by both speakers. - Daigle's initial doubt about frontier tuning vs. current endorsement, stated in his own words. - Notably, despite the video title, the question "why AI hasn't replaced developers" is not directly posed in the transcript; positions on that theme are indirect (agent growth, developer focus, human-agent loops). # Speaker Attribution & Provenance Record **Video:** GitHub's COO Explains Why AI Hasn't Replaced Developers (`01a0f15a-8265-7cd0-8a26-22c42d5041d8`) ## Speakers & Roles ### Mike Taylor — **Interviewer / Host** - Self-identified: "head of tech consulting at Every." - Conducts the interview; frames topics, asks all questions. - States he built an AI clone of the guest to practice the interview. ### Kyle Daigler (transcribed "Kyle Dagel") — **Guest / Primary Subject** - Introduced as **COO of GitHub**; also states he holds the role of **chief marketing officer of developer for Microsoft**. - Attributed with 13 years at GitHub; personal background: former developer, attended art school, paid for it by writing code. - Primary source of all substantive claims about GitHub, Microsoft, and product direction. ### Unattributed Promotional Voice — **Third-party narration (non-substantive)** - Delivers the closing "smash that like button" promotional segment referencing **"AI and I"** show and **Dan Shipper**; not part of the interview. ## Promotional Segments (Flagged) 1. **Pre-interview ad:** Mike Taylor promotes Every subscription ("Go to every.io/subscribe today"). 2. **Post-interview outro:** Unattributed hype voice urging likes/subscribes for the "AI and I" show, naming Dan Shipper as host. Neither is interview content. ## Claims & Third-Party References ### Attributed to Kyle Daigler (GitHub/Microsoft) - 17 million agent-created pull requests in March (per month); 1 billion commits shared for last full year at GitHub Universe (October), projected 14 billion this year. - References to **Mitchell Hashimoto** (vouch system for open source PRs) and **Peter Steinberger** ("150 agents doing everything all at once"). - Names Microsoft leaders Satya, Mustafa, and Jacob (Copilot group) re: "hill climbing." - Mentions partnerships with Anthropic, OpenAI, Google; Microsoft AI models; Microsoft Foundry model router; "MAI thinking one" model; frontier tuning. - Personal anecdote: personal open-claw agent named "Baxter." ### Attributed to Mike Taylor - References experimenting with DSPy ("DSP auto") for research; the $200→$2,000 subscription framing. ### Mentioned but not present - Open-source maintainers (paraphrased as "drowning" in PRs — secondhand via Taylor); a customer anecdote relayed by Daigler. ## Provenance & Availability Notes - No cited external documents, links, or written sources. - **Comment availability: not specified** in transcript or metadata. - Third-party claims (Hashimoto, Steinberger quotes) are relayed secondhand by the speaker — no direct quotes verified. Hi, I'm Mike Taylor. I'm the head of tech consulting at Every and I sat down with Kyle Dagel, the COO of GitHub and talked to him about what is happening on the front lines of coding agents. We have 17 million poll requests coming in every month to GitHub now. It's growing exponentially and that puts them at the forefront of what's happening in this new economy. We talked about how that affects their users as well as how this affects open- source maintainers. And we covered a topic which is dear to everyone's hearts. How do I stop my $200 a month coding agent subscription from ballooning into a $2,000 a month usage limit? In this interview, we did something a little bit different, which is I told Kyle I had made an AI clone of him to practice the interview, and he revealed something surprising in return. Here's the conversation. Every is the only subscription you need to stay at the edge of AI. If you care about being on top of the latest models and using latest tools, you have to subscribe to Every to separate out the signal from the noise. Go to every.io/subscribe today. \>\> Hey Kyle, thanks for spending some time with me at the conference. Yeah, \>\> of course. Yeah. Uh it was good to meet you as well in the the day before and uh I feel like we already kind of covered a few of these questions but I think it'll be good to you know for the wider audience uh so they can understand what's going on here. \>\> Absolutely. \>\> Yeah. Um so first thing I think is uh we were talking about really interesting is that uh the the demographics of the uh customer are changing right like a lot of people uh who previously maybe never used GitHub or never used developer products before are now using them. So how has that changed the way that you decide the product road map? \>\> Yeah. Yeah. I mean, you know, I think uh uh for GitHub in particular, we've always really had this really expansive view of what a developer is. Uh I started as a developer before I would have ever called myself a dev, you know, uh where I was just like writing code, but it was just for me. And I went personally like a completely different career path. I didn't go to school for computer science. I was going to art school. I wrote code to pay for art school which was a very silly decision as an adult now I guess but like at then you know that sort of journey of just like I can create tools with the team and deliver them to people who can have that same experience of like I just want to build an app that's for me or for my family maybe as a startup maybe as a business uh we very much have like serious developer tools you know like uh uh all the largest businesses are using GitHub But when I look at something like the GitHub copilot app, I see just as many, you know, developers that are using AI every day, running multiple projects, like all all kinds of agent sessions at the same time. And I see, you know, our legal team at GitHub using the GitHub copilot app or the finance team or I was meeting with a customer today and they were saying the same thing. a lot of the folks that you know the industry would call knowledge workers or just you know non bytrade developers are using these tools to build little apps or assets for them and so while our focus is very much on developers I think we want to make it easier for people to choose to try to you know write some code uh and make sure there's always an on-ramp into you know writing some software now with you know things like the GitHub copilot app. \>\> Yeah. And then how do you deal with the or how do you help developers deal with the burden of all of that extra you know there's a flood of PRs now like open source maintainers I talked to are like drowning. \>\> Yeah. How do you how do you like what what needs to happen to help them? \>\> Yeah. I mean I think for all developers were you know building tools like the copilot code review. Uh it's now agentic so it finds a lot more novel vulnerabilities and you can just like comment and the you know the agent will take that on and go implement the change if you want to. So I think that code review step is uh in some ways like overlooked as a really great way to get PRs to a place that are much more easily reviewed. Um I think that uh the agentic merge in the app is another place where we see a lot of times internally uh and in the community you know you may comment on something you that might have a code review and you might go through and get it almost all the way there but then there's all those like manual steps just to finish processing the PR and instead I can go in and set exactly what I want to allow you know GitHub copilot to do and say okay now go merge this PR and wait for CI and wait for policies and all of that. I think that's a big part on the open source side. It's a unique set of needs because you don't control who's, you know, sending everything in. Uh, or you haven't really historically. And that's been really where we've been focusing is giving maintainers more tools to decide, well, do you want to accept all of these PRs? Like, who do you want to accept them from? Uh, you know, how much work do you need to do to kind of prove that you're going to contribute something that is going to be meaningful to this project? And that's something that we want to provide tools to open source maintainers but really leave them in control. Every community is choosing a slightly different way to approach the problem. And for GitHub, we've always wanted to leave that in their hands. Like give them tools and enable them, but if a standard comes out of that or most are using a certain practice, we'll lock that in. But we don't really ever want to be the first to create a standard or an approach. Like I think you know Mitchell Hashimoto uh shared like the vouch system that they use and I was getting questions like well why aren't you roll this out to everybody? But there's just as many communities that don't want to use that system because they have their own ideas of how it should work. Um and so for now we're focusing on the the building blocks of controls for maintainers and then as we all are kind of learning together and as maintainers send feedback in we'll uh uh you know we'll cement an entire system if one emerges. \>\> Yeah. And um I feel like you have a front row seat to this like new agent economy where um you know I think you said publicly on Twitter that uh you've had more uh poll requests submitted in like per month than you did all last year. like how how are those stats exploding? \>\> Yeah, I mean I mean we're seeing way more activity on GitHub. Uh you know, we've always been talking about our users, you know, uh for for many many years and that growth, but this year we're seeing obviously the growth of developers having agents building with them. And so last year uh in October at GitHub Universe, we shared, you know, there's a billion commits on GitHub for the full year. uh we're on track to be 14 billion if the growth is linear uh this year which it will not be you know uh in March you know there were 17 million poll requests that were created by agents that's just the agent pull requests and so there's so much more code being created and I think at times everyone goes like oh this is all just like slop this is all just code that's being pushed up but and no one cares that's not like really true we're all just actually getting to the point where we're no longer in the super early adoption. We're definitely not at the peak, but we're climbing that hill, you know, to see what can we build when it's not just Kyle building, but it's Kyle in one to to N, you know, agents um that are using my skills, using uh my resources, using my context, and so on and so forth. And so like we're, you know, investing heavily in preparing for the next wave of growth because it doesn't seem to be kind of like growing and plateauing. It's just going to continue to grow because no matter where you're, you know, building or what tools you're using to build, \>\> all of that code ends up on GitHub, you know, and or that's where you're sharing it with the world or that's where you're collaborating in a PR. Um uh and so we need to be able to support everyone's you know sort of agent moment uh and not just you know GitHub copilot. \>\> Yeah. Yeah. And um how does the business model change because I think fremium makes sense uh you know in a human- centered world where we go to bed uh but the agents are still working while we're asleep now. So does that change like to usage based like is that you see that kind of where things are going? Um, \>\> yeah. I mean, like I don't think we know like yet ultimately like I think you know we very much right now like Kyle's going to have a license or Kyle's using github.com for free and we've always had you know API rate limits uh you know and like things like that. And that's usually where folks are seeing this, you know, the agent back pressure. I think uh I think the goal is that if you want to be able to do way more, if you want to be able to like Peter Steinberger says, you know, 150 agents are doing everything all at once, you know, um uh that's great. We want to be able to enable that to you, but at the same time, I want you to have a great core GitHub experience, you know, and at the very least, there's some amount of agent uh usage as part of that that uh is necessary. you know, I need similar to how we uh way way way back, right? You'd have free public repos, but you didn't have free private repos, you know, and then we said, "Okay, well, actually, like it's fair for an individual to have some code that they don't want to put out into the world, and we'll give you free private repos, you know, to allow you to do that." So, GitHub's always evolving as the industry and community does. Uh, but we're always sort of focused on I need to make sure you, the dev, have what you need to be successful, you know? uh uh and then work with enterprises, you know, to make sure they have what they need to do at scale uh which is usually a little bit different than what you know an individual dev's doing. \>\> Yeah. Yeah. And um I guess the business model pricing kind of that all leads back into like the wider Microsoft orbit. Uh and because you have a dual role now, right? Um partial responsibility for the wider kind of marketing or uh so do you want to talk me through how that's changed and how you prioritize between those two? Yeah, I mean so uh I've been at GitHub for a very long time uh 13 years and uh you know as a a developer myself and leading engineering teams for a lot of that time and I think what's always been unique about GitHub is we really really focus on the dev. Like we're building tools for the developers and the fact that people uh uh like enterprises are buying them is awesome and that's definitely great. uh but we're not building for the buyers, we're building for the developers in 100%. And so uh in this uh uh and that's like been my focus as the COO of GitHub, which I continue to do. And then now as the um uh chief marketing officer of developer for Microsoft, my goal is to look across all of Microsoft's tooling, their developer tools, their technology that they're bringing to developers and making sure that we're bringing like holistic solutions that you can use that are like authentic, you know, to developer experiences. And at events like this where you know we've taken a very different approach to build this year. Uh we're in San Francisco first off like the vibe is a bit different than you know the conference hall uh setup you know really focused on can I go to a session can I use the thing I don't want to be pitched on a thing I have to be able to use it expo hall kind of so on and so forth. uh it's really bringing that expertise and you know love and focus on the developer that GitHub's always had to have even broader impact uh throughout uh all of Microsoft. \>\> Yeah. And um did I hear you say that uh this is the first build that you've had external contributors and \>\> it's the first build that I think by intention we focused on having you know speakers uh you know from the community like in these primary sessions you know uh that includes you know in the keynote we had a bunch of folks like Peter uh you know there's sessions uh from you know Swixs and others as well um I think that it's important like software development is a team sport. You know, it seems silly to think that there's any one company, one group inclusive of GitHub and Microsoft and everyone that can just answer every single question. That's not how software gets made. You know, we're all at least using open source and we're building on the backs of these giant open source projects. Let's invite people in that can help tell their part of the story um uh together. Uh cuz I deeply believe that that's what developers want. I know that's what I want. You know, I know that's what my friends that are developers want. And when we look at the events and we \>\> hear the feedback, they're excited to see people from Microsoft, from GitHub, and then, oh, I get to see this outside perspective at this event. Uh, it's really it's really meaningful. \>\> Yeah, that makes sense. And um it's a very competitive market, right? \>\> Like, yeah, mo the most competitive market probably. Um maybe the last competitive market. I'm not sure but uh but uh like how how do you differentiate in all of that given that the pace of change is so quick? \>\> Yeah, I mean I think we continue to focus uh like on our roots which is you know we care a lot about like developer choice. That's always been true. We care about building for builders and enabling builders. And so I think we're in a moment that's like really interesting because we've uh went from an era of, you know, having a ton of APIs, all this access to a little bit of like an unintentional walled garden setup, you know, where like you get a kind of affinity. Sometimes I'll say it's like a little bit of a mouse trap, you know, and then you realize, oh, this thing's really interesting over here, and then I have to, oh, I have to go learn a new thing or a new tool or a new account. And I think for us, we always want to enable developers that are building with GitHub to go use these other tools and we'll partner with everyone to make that as simple as is possible. And I while I think that, you know, there's uh uh there's other folks that are doing, you know, similar things, I think the the ability to do that across the entirety of building software and not just the codegen side or not just, you know, the collaboration review side, but across everything is a real, you know, uh uh superpower uh of ours. And so, you know, I think you'll see us invest in our own tech. Like we talked about the new uh Microsoft AI models, you know, that we'll continue to bring to developers. We're also continuing to partner with Enthropic and OpenAI and Google and kind of anyone who's bringing a model to market or a coding agent to market. We'll partner with you and we'll both let you bring that to us or we'll emit it through GitHub and GitHub Copilot. \>\> That choice is core and that's something that I don't think we'll uh ever, you know, like back down on. Um because if we do, developers will still choose. They'll just be stuck in another kind of mouse trap. Um uh uh and we don't you know we don't want the world of software to be like that. \>\> Yeah. Yeah. Yeah. And um how do you like when you're making the decisions internally like there was in a news cycle recently about how cold code cancel licenses are being cancelled and um like how do you make the trade-off between dog fooding your own products like using the new models you made or using the GitHub copilot app desktop app um versus like you know using like letting developers kind of experiment with other tools. \>\> Yeah. I mean we all are using a variety of tools because otherwise you lose track you know or you kind of uh uh you're too interested in your own uh own work uh so like for me you know uh I've been a daily driver of a MacBook for for many many years. I use Windows PCs on the weekends when I play video games and I got this roll and I have my Mac, I have my PC and I have my Omar Linux box so I can make sure that every weekend I code most Saturdays I do my kids sports activities in the morning and then in the afternoon I'm coding and I'm swapping between the boxes because I want to understand like okay what's that experience you know uh the GitHub copilot app I only use on Windows because I want to make sure that developers who are on Windows also deserve great apps you know it's not just the audience that's on a Mac for example and that's true across our teams uh especially like when we're looking at okay what about the coding agents what about the harnesses what about the desktop apps what about memory management what about you know everything we have this really great culture of just you know experimentation everyone is building and using these tools while obviously we're putting most of our energy into our own tools it's such a blind spot that I think uh uh it's happened to GitHub in the past where like when you're doing something and you're doing it well, you really laser focus. And that's what every piece of startup energy says, right? It's like look down and just keep moving and keep moving fast. And I think that's um myopic. Uh you know, I I think that while I can't spend every day using every tool, uh when something comes out, I want to know why this is really great. Why are people having a great experience with this? not uh uh not only so I can understand but so I can figure out okay well for our goals for our goal of developer choice I don't need this like I want to focus over here but I want to know why a dev would pick these tools uh and the same thing goes you know the same thing goes for our teams \>\> yeah and how do you how do you filter because obviously a lot of these ideas are relatively shortlived enterprise product development cycles are longer lived uh how do you decide \>\> yeah I think that like right now we're in a moment where we're really looking at the like uh uh the short term in capturing the ability to have a multitude of you know agent sessions. This idea of just you know because that that that seems quite clear you know uh everyone's doing it. How can we cement it? But it seems clear on the longer term path models are going to continue to get better. \>\> Uh the prices of tokens like token economics is going to be a bigger bigger factor in like what models everyone is using. And I do strongly believe that we're not very far off from having serious uh the serious ability to use, you know, something above a small language model on a local device, you know, uh to be able to do some of our work. And so if I assume that I have all this optionality when it comes to tokens, you know, effectively, um the thing that I think uh uh seems to be true from the beginning to open claw to now is this idea of like personalization or mine or context or fine-tuning with context or like you know memory. All of these ideas seem to be a truth that's been there since, you know, chat GPT came out or GitHub copilot came out. And there's experiments, but not a long-term like vision, I think, for this like across the industry. So, I think it's a good example of like where I need to get you to use agents incredibly well, a lot of them, because like if you're if you're into using agents, you're not just going to be staring at a single agent working. But that's not going to give you a long-term great experience. Using an agent that you feel like is completing a thought for you uh will give you that great experience, especially if you did not have to personally codify that thought \>\> to your agent. Always remember that I insert things 100%. like and it it should be able to intuitit uh uh that or potentially again like you know uh post trainer fine-tune or frontier tune like a model uh that deeply understands me in how I'm using the work. Uh uh that is that is kind of how we're looking at it is like sometimes it's shortterm and sometimes we got to take a bunch of bites of the apple or a bunch of attempts at the long term uh to get to something really tangible to help us move forward. \>\> Cool. And I I heard the term hill climbing like a hundred times yesterday and uh and uh I'm a big proponent of that because you know experiment experimented with DSP auto research a few others. Um can you talk a little bit about how that's become a big focus? \>\> Yeah, I mean you know uh uh I think you know Satia and Mustafa talk about it a fair bit and Jacob too leading the co-pilot group. Uh the biggest thing that we've kind of learned is we need to use the you know the use of the tools as a core way to improve the underlying use of the models our own models etc in just the evals that are necessary to ensure that we're actually improving from things like you know uh using the thumbs up thumbs down data that comes in to using like whether you're accepting it and how much you're accepting the you know all of that data is enormous to create a that magical type of experience that's not you just for you but for everyone. And so every week we're talking about the hill climbing results. You know, we're looking at the data. We're looking at the improvement. We're looking at both the hard measures and the soft measures because sometimes the hard uh measures in Evals and Rubik will show that we've made an improvement but like user sentiment will crash uh you know uh even with the same latency and performance overfitting basically \>\> 100%. And so like being able to being able to really um do that loop incredibly quickly and then I think the main goal uh is uh giving everyone one of these hill climbing machines and not have you have to do it the kind of hard way that we've all been doing it but particularly if you're you know in an enterprise and you are using M365 we know so much about that data or we could know so much about that data because of all the assets all the documents the chat ads and so uh being able to you know turn on something like Frontier Tuning and using on the you know using a MAI thinking one uh uh as the the base model it shows real results without having to do all that extra work and it's been interesting because when I first heard about this I'll be honest like I was like \>\> this is like a magic parlor trick you know that is not going to be real you know and I think the reality is that's sometimes um where the alpha is is like where it feels like this is too simple to work, you know, um you know, we all have all this data and what are we going to do with it? We have to do all this effort to make it work. But I think so much has come down the pipe to allow us to just use the data and improve. Look at the workflow and improve and just keep doing the hill climbing. That's why I think we say it so much is that it's not these moonshots or like a hill climb. It is just climb climb improve new eval improve new data improve and just keep going to get to the point where you know we're able to launch these models seven models for ourselves and then you know allow customers to use the same uh or similar you know tooling to do it. \>\> Is that the answer to um stopping the $200 subscription becoming a $2,000 subscription? I mean, like I think that uh the, you know, $200 subscription to $2,000 is really going to be uh uh not only, you know, making these models or, you know, frontier tuning these models so they way they know you better, but I also think it's really really going to be about how can we particularly for developers, you know, help you automatically choose the models and potentially either have like a model in that step \>\> like the model router and Exactly like the you know uh auto model router with task intent and GitHub. Uh Microsoft Foundry has a model router as well that can do this sort of at an API level. the more and more that we can help you um uh you know tell us a bit of like where your bars are like this is an incredibly hard problem and I'm willing to go all the way to the top or I don't I just kind of want to sit you know here uh and let us help choose the models because there's a lot of times where a lot of the reasons why tokens are expensive is because we're all going and choosing our model of the day or week or hour you know and those models are incredibly expensive but \>\> my train of thought is slipping in and out of a hard problem to like a simple problem. I I personally I feel like I'll, you know, I'll get an agent to do like an enormous amount of work and then there's always that last \>\> step that is like a smalish thing, you know, like, oh, I don't actually like change all the naming of this to this. \>\> And that's like a find and replace, you know what I mean? But but am I going to actually go and like, oh, I want to save tokens right now, so I'm going to go off a 48 or 55 and down to ha coup or something, you know? Probably not. uh \>\> but the tools could \>\> and I think that will really help us uh uh uh particularly in the enterprise but even for you know individual developers and you know folks that are building like automations and using their co-pilot uh you know SDK uh to power that it'll it'll help them too. \>\> I did something a little bit weird. I hope you don't find it creepy but I I made a AI version of you to practice this interview. \>\> No way. \>\> Yeah. And it's actually been pretty spot-on so far and hopefully you think the questions have been good. \>\> They've been great. Now I want to see what AI Kyle said. \>\> And um yeah, it's just in the terminal. I didn't have like I didn't go the full whack and make a video thing, but I'm sorry. But um but I found it immensely useful. Like I I just wanted to ask like what other like weird things are you seeing people do internally or externally? \>\> Oh man. So, like it's so funny that you say that because I uh I do a very similar thing where like I I uh I have both like um via the app and then I have a claw that can't talk to work stuff, you know, just so I have like separation of a state. Uh where I spend a lot of time having it read everything I write and say like this interview will get fed into it ultimately. Um, and every day I get a like uh like a comm's report that's not like what Kyle said, but like Kyle, you keep saying this. \>\> Yeah. Okay. \>\> This isn't super clear. \>\> Like, you know, based on how you speak, cuz I I find that I write and speak in like a very particular way that like I want to use a lot of metaphors and so it'll just give me examples of metaphors that are clearer. Yeah. I find that the like self-improvement loop as a human from these agents to be incredibly powerful. We used to talk about it way back with Hubot at GitHub like chat ops and we used to say like humans are way more willing to take critical feedback from robots than other humans. \>\> Yeah, it's less threatening \>\> 100%. And so when my uh open claw that I affectionately named Baxter, you know, tells me how terrible I did in something like \>\> I feel way better going tell me why. and then ensure that when I'm writing emails, when I'm writing a script or I'm reviewing details that you're giving me that feedback. So, a lot of my agent loop is really about me and less about like the software side. I still have all those tools too, you know, uh but it's always looking backwards. It's going, okay, the last seven days I'm going to read all Kyle's emails, Slack messages, you know, and then give me feedback and then look back at what the agent told me to do. Did Kyle do it? And go back seven days. that loop is uh like super super super powerful and I think honestly uh uh like the type of personal consumer experience that I want out of AI, you know, uh uh to be able to tune these tools that way. \>\> Yeah, we need to recursively self-improve as well. \>\> 100%. \>\> Yeah. Thanks so much, man. I appreciate it. Appreciate it. Thank you. Thank you. \>\> Enjoyed that. \>\> Oh my gosh, folks. You absolutely positively have to smash that like button and subscribe to AI and I. Why? Because this show is the epitome of awesomeness. It's like finding a treasure chest in your backyard, but instead of gold, it's filled with pure unadulterated knowledge bombs about chat GPT. Every episode is a roller coaster of emotions, insights, and laughter that will leave you on the edge of your seat, craving for more. It's not just a show. It's a journey into the future with Dan Shipper as the captain of the spaceship. So, do yourself a favor, hit like, smash subscribe, and strap in for the ride of your life. And now, without any further ado, let me just say, Dan, I'm absolutely hopelessly in love with you.