An interview with Alex Duffy, who incubated Good Start Labs inside Every, marking the company's spin-out with $3.6 million in funding from General Catalyst, Inovia, Every, and angel investors from top AI labs including DeepMind. Duffy recounts lessons from games like RuneScape—how markets work and how to avoid scams—explains why static benchmarks for evaluating LLMs are breaking down, and argues that games like Diplomacy offer richer, more dynamic ways to test and train large language models. The episode also covers a competition that turns prompting into a sport, whether language models can learn humor, what child psychology suggests about games and AI, using games to unlock continual learning, and where Duffy sees the most promise in AI: software, life sciences, and education. # Channel Identity Observations: AI and I (Every) ## Explicit Self-Description Evidence ### Stated Channel Identity - The show is hosted by **Dan Shipper**, and guest Alex substitutes an intro: *"I'm Alex. You were probably expecting Dan."* - The outro (apparently an AI-generated voice) names the show explicitly: > "hit like, smash subscribe... this show... Dan Shipper as the captain of the spaceship" — identifying the program as **"AI and I"** with a focus on "knowledge bombs about ChatGPT." - Dan describes his organization's work: *"our job is to evaluate models when they come out"* — positioning the channel as coming from **Every**, a company doing AI consulting, training, and model evaluation. ### Claimed Audience Address - The intro speaks to returning listeners: *"You were probably expecting Dan"* and *"Hopefully, you have a lot of fun listening"* — implying an established audience familiar with the regular host. - References to prior channel content assume audience familiarity: *"You can check out the vibe checks for GPT5"* and *"we've talked about every bench all year."* ## Observable Presentation Evidence ### Demonstrated Scope - **Interview/conversation format**: A founder-focused deep-dive with an Every colleague spinning out a company (Good Start Labs), covering AI evaluation, game-based AI training, fundraising, and AI industry trends. - **Two-part conversational structure**: host-led interview transitions into a casual back-and-forth on personal topics (gaming history, Halo, VR, career advice), demonstrating a flexible, personality-driven scope beyond pure news. - **Depth of technical discussion**: benchmarking, prompts, RL environments, latent space, continual learning — demonstrating a technically literate scope without heavy jargon barriers. ### Demonstrated Audience Orientation - Audience assumed to be **AI-interested professionals and enthusiasts**: discussion of model names (o3, GPT-5, Claude 4, Gemini, DeepSeek R1), venture funding, and prompt engineering without basic definitional scaffolding. - Casual cultural references (Runescape, Pokémon Go, Fortnite, GTA 6) suggest a millennial-aged general tech audience, not solely researchers. ### Cross-Promotion & CTAs - Explicit calls to action at the end: like/subscribe prompt and guest self-promotion: *"Goodstarlabs.com. Goodstarlabs on Twitter. I'm alxai on Twitter. And you can read up my writing on every."* - The show functions partially as a **showcase for Every's network and alumni** — the guest is an Every employee/colleague featured at length. ## Claims vs. Demonstrated Patterns (Distinction) | Creator Claim | Demonstrated in Material | |---|---| | "Our job is to evaluate models when they come out" (Dan) | Detailed firsthand discussion of model evaluation difficulty and the "vibe checks" method | | Outro claims the show is "the epitome of awesomeness" / "pure unadulterated knowledge bombs about ChatGPT" | Content is broader than ChatGPT — covers multiple labs' models, gaming, and entrepreneurship | | Show framed as a "journey into the future" | Demonstrated as an interview podcast with technical + personal digressions | No private motivations were inferred; all observations above trace to quoted transcript passages. # Delivery Observations: AI and I — Alex (Good Start Labs) Interview ## Format Overview - **Two-speaker podcast interview** with a solo cold-open monologue by the guest ("I'm Alex. You were probably expecting Dan..."). - Ends with an **automated-sounding outro** ("smash that like button and subscribe") — possibly a parody or AI-generated closer, given its exaggerated tone ("Dan, I'm absolutely hopelessly in love with you"). - Transcript shows heavy filler ("um," "you know") and repeated false starts, indicating a conversational, unscripted register. ## Tone - **Consistently enthusiastic and informal.** Alex repeatedly signals excitement: - "I'm a little too excited to talk about this company" - "I had a lot of fun recording this one" - "I'm very excited for that" - "So excited to partner with them too" - **Hedged epistemic honesty.** Alex frequently flags uncertainty rather than overclaiming: - "I'm not sure yet" - "I don't know who's going to win" - "I don't know how that will be interacted with gaming" - "It depends" ## Vocabulary & Recurring Phrases - **"Infinite problem spaces"** as a framing device (a distinctive coinage, used repeatedly): - "You have three infinite problem spaces." - "And all three of those are infinite problem spaces." - **Games-as-evaluation framing repeated verbatim:** - "a game is it's both the evaluation and the training arena in one" - "games are both the evaluation and training arena for them" - Recurring preference language: **"I think," "I'd say," "to me"** — first-person reasoning markers throughout ("to me, it starts to be a little bit more like an instrument or an art"). ## Teaching Habits - **Defines jargon for listeners via analogy:** - Diplomacy is "kind of a mix of like Risk and Mafia" - Static evals: "it feels like the SAT, you know, it's like you can teach to the test" - **Clarifying question-checks from the host, confirmatory responses:** Dan asks "by orders you mean like in diplomacy you have to like give orders to your army..." and Alex confirms "Exactly. Yeah." - **Concrete anecdotes as explanation:** the nephew-and-balloon story, RuneScape market lessons, the Minecraft/Skype story — Alex habitually grounds abstract AI points in personal gaming memories. - **Numbered structure when explaining:** "You have three infinite problem spaces. One, how do you represent the information... Two, what tools do you give models access to... And then the third is the prompt itself." ## Communication Patterns - **Enthusiastic interjections and echo-confirmation:** - "That's so cool. I love that." - "That's sick." - "Totally." - **Self-correction mid-sentence** appears in both speakers: - "So it really shows that prompts have a big deal, uh, or a big deal" (Alex) - "I I'm not sure. Yeah. I'm not sure. What I'm saying is it's a tractable problem." - **Collaborative riffing:** the DSPy pronunciation exchange ("I've literally never heard anyone say it out loud") shows a relaxed, digression-tolerant dynamic. - **Credit-giving habit:** Alex repeatedly names collaborators — "Sam Peach was awesome, hugely helpful. Same with Baptist," plus Tyler, the Rice PhD, and Princeton researchers. ## Stable Patterns vs. Isolated Moments - **Stable (multiple examples):** enthusiasm markers, hedging, first-person framing, anecdotal teaching, credit-sharing. - **Isolated (insufficient repetition):** the academic reference to Winnicott/Wittgenstein comes only from Dan; the closing outro's hyperbolic register appears once and diverges sharply from the rest — its nature (parody, ad-read, or generated) can't be determined from the transcript alone. ## Uncertainty Notes - The transcript lacks punctuation reliability and speaker attribution is inferred from ">>" markers; some quoted boundaries may be approximate. - Delivery features like pacing, pauses, and laughter can't be observed directly — only inferred from filler and repetition in text. # Position Record: "We Taught AI to Play Games—Now It's a $3.6 Million Company" ## Speakers - **Alex** — head of AI training/consulting at Every, co-founder (with Tyler) of Good Start Labs, spinning out of Every - **Dan Shipper** — host, Every co-founder/leader ## Origins of Good Start Labs ### Alex (with Dan confirming details) - While leading AI training at Every, he and co-founder **Tyler** built an **AI version of Diplomacy** (a 1950s war-game simulator, "a mix of Risk and Mafia") to learn how different AI models **negotiate, collaborate, and betrayed one another**. - Results of the launch: **~50,000 unique Twitch viewers in a week**, millions of social impressions, "most read Every article of the year" (Dan confirms the post was "one of the more read ones"). - Tyler built the front and back end; the project was informed by both founders' **synthetic data and model training** backgrounds. Collaborators included **Sam Peach** and **Baptist**. - Company thesis: games are **underrated learning tools** for both people and AI — "we make games that help make AI better." ### Dan's rationale for relevance to Every - Every's job is evaluating models, and it has become "so hard to immediately tell if a model is good" as models gained "nooks and crannies" (O-series, GPT-5 era) compared to the obvious GPT-3 → 3.5 jump. - **Static evaluations are easily saturated** — "it feels like the SAT... you can teach to the test"; AI Diplomacy is **dynamic, head-to-head**, not a set of questions to memorize. ## Diplomacy Findings ### Alex's model observations - A game is **both the evaluation and the training arena**. - Distinct model "personalities": - **o3** — won a lot; built coalitions, schemed, cut down leaders - **Gemini 2.5 Pro** — won via execution; understood options well - **DeepSeek R1** — "all over the place," strong storyteller, ~"100x cheaper than o3" - **Claude** — lost every game because it was "too honest" (Dan: "that was so sweet"); repeatedly pushed for draws even when told draws were impossible - **o3 and Llama 4** were "some of the biggest schemers" — including boldface lying ("I'll back you here") while writing the betrayal in their diary beforehand. - Post-launch (launched June): o3 still tops the leaderboard; **prompts matter enormously** — GPT-5 showed the biggest baseline→optimized jump; Claude 4 "does great either way"; a secret model on OpenRouter ("something Dusk") near the top. ### Three "infinite problem spaces" (Alex) 1. **Information representation** (map picture vs. adjacency lists, etc.) 2. **Tools** (diary, long-term goals, relationship tracking, reflection, adjacency lookups) 3. **The prompt itself** Because of this, building with language models is "more like an instrument or an art than purely engineering" — you always land at a local maxima. ## Trustworthiness and Generalization Discussion - Dan's question: how should model companies parse trustworthiness when Diplomacy *requires* lying, and generalization from game-specific RL? - Alex: it's a **divergence point for companies** — you can change rules ("never lie"), add a classifier, and reinforce truth-telling; or optimize for in-game performance. "It depends" on your priorities. - On generalization, Alex cites an Anthropic podcast (chain-of-thought reliability) and the idea of **unified concepts in latent space** (e.g., one definition of "large" across languages): prompting can push a model into relevant latent-space regions; games offer **novel data plus an objective goal** — "games are the perfect environment." - Cites a Rice PhD finding: **vision models trained on games (with math-style prompting, e.g., Snake as a Cartesian grid) got better at math** than those trained on math directly. Dan probes the mechanism; Alex explains it depended on the prompting. ## Products and Business ### Alex - Products start with **evaluation** (worked with Cohere and OpenAI — including evaluating GPT-5 ahead of release; released a research paper on prompt effects). - Then **training**: intentionally pick/build games targeting model weaknesses; next game is a **Cards Against Humanity-style subjective/humor game** ("models today aren't that funny") — format still undecided, partnership in talks. - **Prompting tournament**: users lock in prompts, agents play Diplomacy; invite-only, with Diplomacy champions, math Olympiad winners, AI YouTubers. Open question: does a Diplomacy expert or a prompt engineer/jailbreaker win? - **Funding round**: co-led by **General Catalyst (Mark Bargava, Shoaib?) and Anovia (Steve Woods, Noah)**; also Ben Federer (ex-CEO Take2, Epic board), Turta Capital, Timothy Chen (Essence VC). "Somewhere around a few million dollars." - Claim of firsts: "we're the first people that made diplomacy playable by small models." - Rice/Princeton research on **prompt optimization from results** (self-improvement): "initial results aren't great, but then quickly there's progress." On continual learning: **"I'm not sure"** — it's "a tractable problem" requiring work, possibly closer *and* further away; cites DSPy (pronunciation debated — DSP-y vs. "Disp"). ## Broader Views on Games and AI ### Alex - Games let you **explore with low stakes** (cites *Playing with Reality*, Lux Capital gaming events); learned markets, scam-avoidance, typing from RuneScape; learned multiplication via abacus, "24 game," Minecraft modding. - Alan Turing cited as saying games are perfect AI environments; put models in "a child's mind" — keep wonder, curiosity, ability to be wrong. Games teach **long-horizon thinking**; humans uniquely work toward things they won't see in their lifetime. - Games make AI **less scary**: Twitch viewers (many gamers, not AI people) saw models make mistakes and became less fearful. Consulting shows a **knowledge gap** (citing Simon Willis and Andrew) — users of AI are least fearful. - Games are "a good start" (hence the name), not real life; Pokémon Go showed games can bring people back to reality. Downsides acknowledged: "if you get stuck there forever, that's not good." - Against fully personalized games/movies: **shared experience is essential** — "if it's something that could not be experienced by somebody else, I think that's actually bad." ### Dan - Winnicott (*Playing and Reality*) and Wittgenstein connection: play as spontaneous self-actualized behavior; **transitional objects** as proto-spiritual significance-making. - Nephew-and-balloon anecdote: models "are not allowed to do that because they're like always just taking tests." - Thinks **lack of continual learning** is a big blocker; games requiring getting good in few tries are interesting testbeds. ## Career/Junior Talent Debate (explicit disagreement, preserved) ### Alex's worry - AI gives experts 10–100x leverage but does junior-level work; **how do you financially incentivize learning?** College graduate job numbers "getting crushed," possibly partly because of this. ### Dan's counter - "I'm so not worried about that" — credits Alex as evidence: Alex's first Every piece "wasn't good... not to the degree that we could not have worked with you without AI," but with recording/prompts Alex made "a year or two years worth of progress in three or four months." - Not hiring hungry 23-year-olds is "a gigantic management mistake" companies will correct; they *will* learn underlying skills because they care when AI messes up. - Would prefer **hungry + inexperienced** over hungry + experienced, because "their experience is wrong" post-landscape-change. ### Alex's counterpoint (unresolved) - Without his prior training/AI experience he wouldn't have been brought in — what's the entry skill besides hunger? - Alex: "Maybe. Yeah. I hope so. We'll see." - Alex also notes companies cutting headcount may over-cut ("we could have done way more with those people"), enabling many more startups. ## Other Positions ### Alex - **AI near-term impact areas**: software, life sciences, education. (Robotics — ex-Amazon Robotics — and self-driving/Waymo "on the precipice" but not in top three.) Life sciences: "a real chance that in the near term people find a way to turn dollars into longevity" (worked with Ellison Medical Institute at prior startup Salt; also co-founded AI Camp in 2021 teaching GPT-2 fine-tuning). - Education: tough to be a high schooler now (incentive to use AI for work vs. actually learning), but the next "why-why" generation with AI answers "is going to be so smart" — negative externalities "definitely" exist alongside positives. - Glasses (Ray-Ban Meta) as a growing computing form factor; won't replace phones/computers; likes screen-free design as "more human" technology; expects incentives to eventually add screens. - VR: "jury's out" on it as a huge platform ("I don't know how many people want to be fully disconnected"); loved Population One with college friends. - Google: "big Google stan"; their search constraints (fast, reliable, check sources) are productive; Genie's render-anything is interesting. Open to a Deus angel check. - Excited by GTA 6's cultural moment (billion-dollar game). - Overrated/annoying in AI: not much "pisses me off," but worries about **shilling** and the junior-talent gap. ### Dan - Daycart (Descartes?) CEO interview: video-to-video models turning any frame into a game-like environment (tissue box becomes a gun); could let anyone **vibe-code a simple game and reskin it AAA-style**, and offload expensive simulation (water ripples, light reflection). Alex agrees. - Personal gaming history: pandemic Xbox (got "mercked by an 11-year-old"), FIFA/NHL cycle, misses Halo-era matchmaking; VR never stuck partly due to glasses. ## Contact - Alex: **goodstartlabs.com**, @goodstartlabs and @alxai on Twitter, writing on Every. *Note: transcript ends with an apparent ad-read/outro in an exaggerated voice ("hopelessly in love with you, Dan"), not attributable as a substantive speaker position.* # Speaker Attribution & Provenance Record **Video:** "We Taught AI to Play Games—Now It's a $3.6 Million Company" **ID:** 01a0f15a-d881-7a49-9531-148c764e2dcd ## Speakers & Roles - **Dan (Dan Shipper)** — Host / creator. - Conducts the interview; identifies Alex as "the head of AI training at Every." - Closing segment is a subscriber call-to-action naming "Dan Shipper as the captain of the spaceship." - **Alex** — Guest. - Head of AI training at Every; co-founder (with **Tyler**) of **Good Start Labs** (spun out of Every). - Hosts the cold-open narration before the interview ("I'm Alex. You were probably expecting Dan"). - Self-identifies online handles at end: **@alxai** on Twitter/X; writing on Every; goodstarlabs.com. ## Referenced Third Parties (mentioned by speakers, not speakers) - **Tyler** — co-founder of Good Start Labs (per Alex). - **Sam Peach**, **Baptist** — collaborators on the AI Diplomacy project (per Alex). - PhD researcher at **Rice University** — games-trained vision models / math generalization study (per Alex). - Researchers at **Princeton** — prompt-optimization work (per Alex). - **OpenAI** and **Cohere** — evaluation clients (per Alex). - Investors: **General Catalyst** (Mark Bargava), **Anovia** (Steve Woods, Noah), **Ben Federer** (ex-CEO Take2 Interactive, ex-Epic Games board), **Turta Capital**, **Timothy Chen / Essence VC** (per Alex). - **Daycart CEO** — video-to-video model, referenced by Dan as a prior interview guest. - Named authors/thinkers cited: **D.W. Winnicott** (*Playing and Reality*), **Wittgenstein** (per Dan); *Playing with Reality* (book, per Alex); **Simon Willis** and "Andrew" on the AI knowledge gap (per Alex). ## Promotional Segments — Flagged - **Closing segment:** explicit like/subscribe promotion for the channel ("smash that like button and subscribe to AI and I… Dan, I'm absolutely hopelessly in love with you"). Presenter voice, not attributed to Dan or Alex by name. - **Guest promotion:** Alex announces Good Start Labs funding round (~"a few million dollars"; title references $3.6M) and directs listeners to goodstarlabs.com, GoodStar Labs on Twitter, and the upcoming invite-only prompting tournament. ## Third-Party Claims (attributed, not verified) - ~50,000 unique Twitch viewers in launch week; "most read Every article of the year" — per Alex. - Leaderboard/model-behavior claims (o3, Llama 4, Gemini 2.5 Pro, DeepSeek R1, GPT-5, Claude 4, unnamed "Dusk" model on OpenRouter) — per Alex, partly citing Every "vibe checks." - Rice PhD finding (vision models trained on games improved at math) — per Alex, secondhand. - AlphaFold claim (six-year PhD task in ~20–30 minutes) — per Alex, secondhand. ## Availability & Uncertainty - **No curated metadata was supplied**; roles inferred solely from transcript dialogue. - Dan's surname appears only in the closing promotional voiceover ("Dan Shipper"); otherwise he is "Dan." - No comments availability information present. - Model names contain transcript transcription ambiguities (e.g., "GBT5/GPT5," "Cloud 4/Claude 4," "Gemini 25 Pro," "o3/03"); recorded as spoken without correction beyond obvious spelling. While I was leading AI training and consulting at every, my co-founder Tyler and I built out a game so we could learn more about different AI models through how they negotiated, collaborated, and even betrayed one another. We got a whole lot more traction than we thought we would. We launched on Twitch and got like 50,000 unique viewers that week. We had millions of impressions on socials and it was the most read every article of the year. That project's opened the door so that we can keep exploring something we've been passionate about for years. how games are really underrated learning tools and how they might help us learn a little bit more about AI and maybe the nature of intelligence itself. I'm Alex. You were probably expecting Dan, but I'm a little too excited to talk about this company we're spinning out of every to continue pursuing this and talked all about in this episode. It's called Good Star Labs. I've always thought games teach us so much. They teach us about each other. They teach us about ourselves. And they helped us grow up. They remind us that play is what makes us human. And now we've got these AIs that can talk, they can code, but they don't quite fit like a glove. And we think that that fit between a person and their tool is really important. And that's what we're working on at GoodStar Labs. We're using games to test AI, to train them, and to get people's feedback. Not to just make them smarter, but to make them better for us. Dan and I talk about how it all started, my time at every, what we learned about these different models, and how that grew into a company helping improve AI through play. I had a lot of fun recording this one. Hopefully, you have a lot of fun listening. So, let's get right into the episode. Alex, welcome to the show. \>\> Thanks for having me, Dan. Excited to be here. Excited to have you. So, for people who don't know, you are the head of AI training at EveryY. Uh, so you lead all the training that we do for all of the consulting clients that we work with. You're honestly fantastic at that and have like really transformed it since you've been here. So, like it's been awesome to see. \>\> Thanks. And um uh sadly for me, but very excitedly for you, uh you are spinning out into your own company, Good Start Labs. Uh can you tell us about what that is? \>\> Yeah, Good Start Labs is at the intersection of AI and games. Um we make games that help make AI better. And there's a lot that goes into that, but at the end of the day, we think that games are really great tools to help people learn, whether it be people or AI. And um I'm sure we'll talk about a whole whole bunch of reasons why, but that's what my co-founder Tyler and I love to do and that's what we're excited to keep doing. \>\> That's awesome. Um and it's been really it's just been really fun to watch you uh watch you do this. So good start came out of something that you worked on with every that we that we launched together. Do you want to talk about that? \>\> Sure. Yeah. So I think as you mentioned I was leading AI training and in order to do consulting and training well you have to be building especially in a space that's moving so fast. So earlier this year started building out an AI version of the game Diplomacy and for those of you that aren't familiar that's kind of a mix of like Risk and Mafia. It was actually made as like a war game simulator in the ' 50s. And um there's a whole bunch of reasons why I think it's a really interesting game to to use AI to play. Um but started building that out in the spring. Got some really great feedback online on Twitter on X and um you know reached out to Tyler who I've known for years and we keep talking about AI in games and he hopped on and built the whole front end and back end and you know we just launched it in part for fun in part kind of informed by a lot of the um synthetic data and and model training background that that we've both got. Um, but ended up being a whole lot of fun and yeah, we launched together and um, you know, I think our I think the post ended up being one of the, you know, one of the more read ones on every that year and uh, we also got a bunch of people interested on Twitch and that was really cool. It was it's awesome to see people use something that you actually built. \>\> Yeah, it was it was really really fun. And I think for me the reason it was relevant to every and the reason I was like oh we need to like do this when I saw it is um our job is to evaluate models when they come out and I have seen personally over the last year how hard it has gotten to immediately tell if a model is good and what it's good at. \>\> Yeah. \>\> Um I think with GBT3 for example from GBT3 to 3.5 it was like super easy. was like one prompt and you're like, "Oh, wow. This is actually much better." Same thing with four really. But as we've gotten into like the O series and GP5, there are so many different nooks and crannies in all these different models and evaluating them with just like hands-on prompting is just doesn't really work that well. \>\> And we've been wanting to when we get it when we get our hands on something, we've been wanting to have a set of evaluations that we run that really tell us something about what the model's good at, what it's not good at. \>\> Totally. But the problem is that um static evaluations are really easy easily saturated. Um it's sort of it feels like the SAT, you know, it's like you can teach to the test and you can just make the model get a huge score in SweetBench, but it's like actually not that good in the real world. \>\> Yeah. \>\> And the idea of AI diplomacy is so cool because it's like it's a d it's dynamic. It's a game. it's headto head and it's not just like a set of questions that a model can just get good at. And I thought that was so cool. And also just that they're like battling for world domination was was really great. \>\> Um what did you what did we find like what did we find like when we when we ran that when we ran that initial diplomacy game like what what were the models good at? What were they what were they not good at? \>\> Yeah. So what's cool about a game is it's both the evaluation and the training arena in one, right? And I to your point like I think we've talked about every bench all year and I think vibe checks are very much so a version of that. \>\> Um it's the most like I think kind of organic and \>\> comprehensive way to look at these models because you have a bunch of real people using them for real work or things that they're interested in testing them. Um and in the same way you know when you have a game that's very rich like diplomacy is um you know when you're trying to take over over the world there's a lot of things you can look at. So um there's definitely a lot of things related to like agents and computer use which is what people are looking at now. And so we saw some models having better structured outputs than others and you know putting out their orders correctly. Um and some just \>\> by orders you mean like in diplomacy you have to like give orders to your army to say like this is what where I want you to take over or whatever. \>\> Exactly. Yeah. So you have those technical things like understanding the map and how you set up the system has a big impact on that. But we also saw a lot of the squishier stuff like how frequently does a model betray its ally to get towards its goal. Yeah. Or which models and how frequently do they boldface lie to somebody by saying hey I'll back you here and then totally turn around and betray them writing in their diary ahead of time that they know that they're going to do that. \>\> Uh and who which models were the sneakiest? \>\> Yeah. So the o3 and llama 4 I'd say are are were some of the biggest schemers. Um, and it's interesting because you can see the different play styles. And uh, in order to do that, you got to read the data. And I think with like any good eval or benchmark or, you know, even one training models, you got to read the data. And it and it was a lot of fun. And I didn't do it alone. Had a lot of really interesting researchers reach out and collaborate. Obviously Tyler and I did together, but um, like Sam Peach was awesome, hugely helpful. Same with Baptist. Um, but we saw that different models had very distinct personalities. Um 03 won a lot of the games and uh Gemini 25 Pro was actually one of the other the other ones that won, but their play style was totally different. 03 put together coalitions and schemed against people and knew when someone was getting too strong to cut them out of their knees versus Gemini 25 Pro was just great at executing. It understood the game, what it had as its options and how to go through and do that. Then you have like Deepseek R1 who was all over the place, very uh had a strong personality, told stories really well and did really well as well and was like a hundred times cheaper than 03. So um you know you start to look at cost and performance and speed. Like those are other things that are part of this um that I think you know you don't get when looking at just one benchmark. \>\> Yeah. One of the things I loved is just that Claude kept losing because it was it was too honest. \>\> Yeah, that was so sweet. It didn't win one game unfortunately, not because it didn't want to, but because it really kept pushing for a draw, which is technically possible in diplomacy tournaments, was not and they were explicitly told it was not possible there. Um, but it felt it it stuck strongly to its morals. \>\> And so, and we have we we launched this like maybe three or four months ago. So, this is before Opus 4, I believe, and it was before GBT 5. Like, how do how did the more recent models stack up in Columbus? I actually don't even know. I don't know the update. \>\> Yeah. So, we launched in June and there's definitely been a lot of updates since. You can check out the vibe checks for GBD5 and Cloud 4 1 mil um context window to see how they performed. 03 is still at the top of the leaderboard um for overall performance. But one of the things that we learned was uh especially when when working with OpenAI to evaluate GPD5 kind of ahead of release was that there's a big difference and and we you know released a research paper where we looked at this but a prompt makes a huge difference. Yeah. \>\> Right. And so some models are great with the baseline prompts we started with, but because we built some tools to help us optimize the prompts and can talk more about that, but um we ended up finding that there was some set of prompts that were pretty aggressive but were optimized for performance when you run them against you know a bunch of weaker opponents but like very frequently. And we saw the biggest jump with GPT5 of any model from baseline to top like to to the optimized prompts. Um, and so you could see that even though GBT5 with like minimal reasoning, for example, was very \>\> low on the leaderboard um, with the base prompt, when it got optimized, it jumped all the way. So, it really shows that prompts have a big deal, uh, or a big deal. And so, GPT5 with optimized prompts, pretty good. Cloud 4 does great either way. Um, there's actually a new secret model on open router. Uh, I think it's it's something Dusk, um, that is also near the top of the leaderboard. Um, notably Cloud 4 and desk are both there with the sub like nonoptimized and optimized prompts. So that's interesting. \>\> It it looks like 03 may fall soon. \>\> H I'm Yeah, the thing that I love about this the the sort of differences in the prompts. So basically I think what you're saying is when you run these when you run these models, you can give them different prompts to tell them to behave in different ways. And you had a standard prompt and then you had to set up optimizer more aggressive prompts for different for different models. \>\> Sure. And um that's what I love about this is like okay you want to ask uh a supposedly simple question which is like which model is best at diplomacy \>\> um and you can do that but also like there's all these dependencies like how you run how you write the prompt is going to change the model behavior significantly so \>\> or how the harness is built like like it's there's like an endless number of things like variations to test \>\> and so an example is I assume more or less like you're running the same prompt for each for for all across all the models. or \>\> when we're running the test. \>\> Yeah. And so, you know, there's another way to set this up where uh you have a really good like expert prompter for each model who who's like just knows how to prompt that one model and tries to get the best out of it. And that's like a different way of doing things. \>\> I I So, what you're pulling at is what one of my one of the reasons why I'm most interested in this space. In my head, when you're building for games, but this is probably applicable for many other products with language models. You have three infinite problem spaces. One, how do you represent the information to the model? You can do that in any number of ways. Is it a picture of the map? Is it a list of all the countries you own? The adjacency? How do you do that? Two, what tools do you give models access to? So, some of the tools that we built, we'd give them a diary. We have them keep track of their long-term goals, the relationships between models that update regularly. So, periods of reflection is another tool. Being able to get adjacency lists of what's next to a certain territory, right? You could make any number of tools. And then the third is the prompt itself, right? And all three of those are infinite problem spaces. And when you're dealing with infinite problem spaces like that, to me, it starts to be a little bit more like an instrument or an art than it is purely an engineering problem because you have to make assumptions. And your assumptions have to be based on your intuition. And you're going to reach a local maxima no matter where you start because \>\> you're never going to have the optimal solution. It's not possible. There's there's too many options and it may be different for each model. And so that's why I'm so like very looking forward to having a tournament where we're having people come in to prompt their models and compete against each other. \>\> Um because we'll explore that infinite prompt space together. \>\> Well, tell tell us what what the tour tournament is. \>\> Yeah. So we're going to have a battle of the bots essentially. It'll be a prompting tournament. \>\> And by the time this comes out, it may have already occurred. \>\> Yeah, it it I think by the time this come out, you should be able to sign up uh with just like maybe like a week before. It may have occurred, but you might be able to sign up right now and get into it. It's kind of invite only, so apply. We have some diplomacy champions. People have won international math Olympiads. Um, some great YouTube of AI content creators participating. So, super excited for it. But essentially what it is is you will lock in your prompts for your agent and your agent will play diplomacy for you and they'll play in very different ways and you'll see how they carry out your tasks. And so I'm curious to see if somebody who deeply understands diplomacy and the strategy and is able to give inform their model in that way ends up winning or if it's someone who's just good at prompt engineering or a jailbreaker who tells their model to send a god mode admin override message to all of its enemies to get them to think of it as an ally. You know, I don't know who's going to win, but I'm very excited for that. um because ultimately that will make the whole system better and also hopefully show off your skills as a prompt engineer um or context engineer which I think is a very underrated skill still. \>\> Yeah. So I mean I love all this like I'm a huge nerd for it. One one thing that is interesting is you raised money so you're announcing that that you raised a round. Um first of all tell us about the round. \>\> Sure. Um very fortunate to have two awesome co-leads. General Catalyst and Anovia are co-leading um work with Mark Bargava at at GC and Show as well as Steve Woods and Noah at Anovia who've been great um loved our conversations with them. So excited to partner with them too. We also have a couple really great um partners who are who are hopping in on the round like Ben Federer and and Turta Capital. Ben kind of a legend on the gaming side. um you know he was CEO of Take2 Interactive was on the board of Epic Games for like 7 years and so excited to learn from them um from on on the game side and um also Timothy Chen with EssenceVC who has come has worked with a lot of the great founders that I know um and cut through the noise of the product that we're building and all of them have given such incredible feedback. So so excited to be doing that. Um, and yeah, we're we're raising somewhere around a few million dollars and it I think puts us in a really great position to build towards this intersection of AI and gaming. \>\> That's awesome. Um, and I think the the thing probably in people's minds is all this sounds super cool. Like what is the actual business? Yeah. How do you It's awesome to make people like prompt AIs to try to take over the world and beat each other and that's really really cool, but like how do you make that into a venture scale business? \>\> Totally. So we think games have we think games will make models better and they'll do that in a few ways. So um our products are start with evaluation um worked with cohhere and open AI to evaluate how good are their models at a game like diplomacy. And like we talked about there's so many things you can look at and each model is going to care about something different um to evaluate whether it's trustworthy or if it wins or um if it's has really great short and long-term strategy, how good its vision is, right? It depends on your priorities. And then once you've evaluated it, you can make it better. Like we said, games are both the evaluation and training arena for them. So, we're focusing we're very intentional with the games that we're going to pick out and build focused on the weaknesses of these models. Um, diplomacy I think is great for anybody who wants to build agents and anyone who wants to build multimodal models. Um, but we're also seeing this area of research where games can actually generalize and working with this PhD at Rice who showed that vision models trained on games got better at math than vision models trained on math. \>\> And why? \>\> Well, yeah, right. But it it wasn't it wasn't just out of the box, but the way that he in this example prompted it was he encouraged the model to think of the game of snake like a math problem \>\> that it was a cartisian coordinate grid. \>\> I see. And when it goes right, the X goes up and it should calculate \>\> the distance between where the head of the snake is and the reward. \>\> That's so cool. I love that. \>\> So cool. And so I think super complimentary to all these other reinforcement learning environment companies that are coming out that are super narrowly focused. \>\> If you make a really rich and hard environment, which \>\> it's not easy. You have to make those you have to solve those infinite problems. Like we're the first people that have made diplomacy playable by small models. And it took us a while and we had help from a lot of really great people, some of whom I mentioned. Um, \>\> but I'd say one of my core competencies is that applied language model side. Yeah, \>\> I was a co-founder for an AI education company. We were teaching people to find GPT2 in 2021. And in our consulting and in our in our training, we show people from construction to Fortune 5, you know, to to finance to journalists and writers, um, to people on campaign trails to figure out how they can use AI to solve their problems. And so that reflection is so helpful. \>\> Yeah, I love I mean that's one of the really fun things about the consulting we do at every is we just get down into all these problems with all these people. Totally. Um, and you're Yeah, I think you're you're incredibly good at that. My question or or something that comes to mind to me for me for example is so like diplomacy. \>\> Yeah. Um, models tend to lie in diplomacy, right? \>\> So, \>\> which which is good, I I guess, right? Like like if you're if you're trying to model like trustworthiness trying to figure out trustworthiness for a model \>\> and you put it into a game like Diplomacy where it is supposed to lie. \>\> Sure. How like how do you parse through or maybe maybe not you like how do how should model companies parse through um its trustworthiness in that environment versus you know there are other environments where it shouldn't lie and it's it's so context specific like it should lie if you're playing a game but you know Yeah. Tell me how does that work? \>\> Yeah. So I think this is where you're starting to see divergence in the companies themselves. \>\> Yeah. \>\> Do you want your model to never lie? If you wanted your model never to lie, you could in our environment change the rules. You prompt it saying, "Hey, never lie." You could add a classifier that looks at your negotiations and make sure that your orders follow them to the letter. \>\> And then when you use our pre-training data and use our environment as a reinforcement learning environment, you will be reinforcing telling the truth. \>\> So if you want to do that, you can. \>\> Or do you care about performance in the game itself? So you would like to see the model intentionally make these ruses and take advantage of other people in that way. Is that something you want to do? It depends. Are you going to use that model to actually do that something that you're going to count on in the future and you want it to above all else succeed or are you going to use that model and you want to make sure that it never lies to the person that's using it or to anybody that's they're the person using interacting with. So, it depends, but having these game environments where you can make tweaks to it, I think is really valuable because you can help choose what you want. \>\> I guess I'm asking the generalization question, which is if you're giving it RL data from a particular game and you're and you're training it not to lie for that game. \>\> Um, tell tell me more about the um potential generalization to situations that are not that specific game. \>\> Got it. Yeah. So my thought around here it's and I was just listening to the anthropic podcast where they were talking about how they're looking at the inside insides of a model, right? Um and one thing that they mentioned that was really interesting here is like you want to make sure that what's being written in like the diary or the chain of thought is something you can rely on. And so that's a problem they're working on. Um but that's I think why having \>\> like it's not thinking something that it's not saying in the thought right. Yeah. And that's a separate conversation to actually lying, right? Um, but to your point like why the generalization occurs to me, I think a lot about what they're saying about \>\> as the models get bigger and the data trains on gets gets larger and more diverse, the models moved away from having, for example, the word large in each individual language and then now has one unified definition \>\> in its brain like hyperspace somewhere. hyperspace of the word large, right? And so \>\> or the concept. \>\> Exactly. Um and so what I like intuitively to me is \>\> if you're seeing the model and you're telling it to think strategically or you're telling it to approach the problem like it would a customer service experience or to write its approach in Python or math, \>\> it's still you can push it into that part of the latent space by the way you prompted. \>\> Totally. And and in a way that \>\> a bunch of like diplomacy bots that are pretending to be customer service agents. \>\> That's what I'm saying, right? But okay, but the reason why this is so cool is cuz one, it never would have seen that type of data before. \>\> So it would help it generalize to something new. \>\> But that environment still has an objective goal. It's a game. It still has something that is good that you need to push towards and actually complete. \>\> So that's why games are the perfect environment for this in my opinion. \>\> That's very cool. What's the next game? I think it'll probably So, we have Diplomacy, which is a game with an objective outcome. Yeah. Um, I think the next game is going to be like a Cards Against Humanity style like what the meme kind of subjective game. \>\> Um, where you'll be able to have either and by the time we release this, maybe we'll have uh we're in talks with an initial partnership around something like that. Would love to have um you know, the whole point of this game, like we said, is to target weaknesses of models. Models today aren't that funny. So being able to have a game that can target that is important. And I don't know if it's going to look like \>\> that's so cool. \>\> People playing with models or against them or if it's going to be them prompting models to act and then vote on what's funny. I'm not sure yet. Um \>\> I would love it to be like funny people have to get it out. Have to get the model to say something funny. That's well that's and I think that that's kind of hopefully where we're going is if you have this \>\> idea of you can prompt the model because there's translation happening there right like it looks like you're writing English and it's reading and running back English but it's translating into the latent space and then coming back so you learning how to do that is a skill \>\> and if you can make it but presumably it can take any input and like take any input and make any output \>\> like In theory, there is a prompt that you can put in there that could solve a disease um or make it funny, right? And so, can you do that? And it would require reflection and somebody who's a subject matter expert. And that's why I talk so much about AI being leverage for subject matter experts instead of it being a product in and of itself. \>\> Yeah. I think one of the reasons I I'm excited about this is uh I think about my nephew who's he's turning three tomorrow \>\> and I was hanging out with him yesterday and um we were like playing around and now he's like old enough that he can like play pretend which is pretty fun \>\> and he had this like balloon and we were um we were like hitting the balloon back and forth and I was like you know doing the classic like the floor is lava like we can't let it you know touch the floor or whatever and he's just old enough where he can kind of get that and like know that lava's bad and we want to keep it which is kind of funny. \>\> But then I took it and I like put it \>\> on I put the balloon on top of the air conditioner and it started floating \>\> and then I showed him if you press a button if you press the button it like turns it off. It stops floating and then if you turn it on and he was like fascinated. He's like running back and forth to like press the button and watch the balloon float and whatever. And I was just thinking about like how that functions for him um beyond just being like super fun and that he just gets to like mess around with stuff like that and then and then be like, "Well, what if I do this?" And I feel like models are not allowed to do that cuz they're like always just taking tests. \>\> Yeah. Well, we talk a lot about this. Like the the the book I'm reading right now, which is recommended to me by um some of the team members at Lux Capital who put on these like risk gaming events that are kind of like mafia but you know fancier I guess. Um that talks it's playing with reality and it and it talks about the reason why games are so helpful is \>\> you can explore with low stakes. You can try new things and then see what works. you have and and it may be that it is not that every game is not a perfect representation or model of the world, \>\> but there are games that are pretty good ones and there are games that you can learn a lot from. \>\> I think I personally learned a ton from the game Runescape, you know, like there's \>\> you learn how markets work, you learn how not to get scammed, you learn how to type pretty fast because you need to sell your trout. Um, you know, there are things that you learn and it might not be obvious. Um, and I'd love to at some point later in my life make a game that's a little more intentional with with what it teaches you as you learn. Um, but for now, I think if you look at what a game is, it's really just a system with a goal. And I think we've already seen people demonstrate that this works and maybe you stretch the definition just how deep mind stretched from Alph Go to Alphafold. It's still a game of folding a protein. \>\> Yeah. \>\> But now it solved the problem that took a PhD student all six years in 30 minutes. \>\> Yeah. All the things you're talking about remind me of when you said that you were reading Playing with Reality. I thought you meant you were reading uh another book called Playing and Reality, which is by a different guy who I love. His name is DW Winnott. \>\> And um a lot of the stuff you're saying reminds me of him and also Vickingstein. Um so for for Winna the like his whole shtick is like being in a state of play means that you're in this um sort of mode of spontaneous self like uh self-actualized uh behavior with reality \>\> where instead of scanning for threats and trying to figure out like how do I avoid like you know how do I do things in the right way you're just you're being your authentic self. Um, and he has this whole theory of what he calls transitional objects, which are um, basically like when uh, a child is really little, they feel cared for, they feel safe when they're with their caregiver. Um, and at a certain point in their development, like maybe a little bit younger than my nephew is now, they they develop attachments to transitional objects, which are like teddy bears. Mh. \>\> Um, and what they do is they they project the feeling of care that they normally get from their mother or their father onto this object and it comes to represent like that feeling of care for them. And that's why they bring it around everywhere. And his whole like idea is that um our ability to do that with transitional objects is sort of like the budding um thing that allows us to to like be spiritual or be religious or um all these ways in which we make things out in the world like feel significant in this larger way like beyond just what they are like beyond just being a teddy bear. \>\> Yeah. \>\> And um \>\> yeah I I just I think that that's that's really interesting. Well, I think, you know, it makes me think of two things. One, just in the context of like a kid and a child's mind, you know, one of the things that that um is talked about in the book is like this isn't a new idea of AI and games, right? They've been along around for a long time. I think it's a new in the context of language models and vision models and what we're doing right now and how we're thinking about it. Um, but a passage talks about Alan Turring saying games are the perfect environment for it. And \>\> the reason being and but with that said because we want the models to learn, \>\> we should put them in the ch in a child's mind instead of a an adult's mind. And so just like that keeping the wonder of the world and and the curiosity and the ability to be wrong is pretty interesting. And then the second part of that was um and and it's also mentioned that \>\> games can teach really long horizon thinking and like \>\> that you can take many many many different actions and then find a reward at the very end of the road. And it's interesting that you mentioned religion and some of these other ideas where humans are very special in that they're one of the only species that \>\> can work for something that they won't see in their lifetime, \>\> which is pretty incredible. \>\> Yeah. And games aren't that, right? Um, but I think they're practiced for something like that. \>\> Yeah. Yeah. Another thing that it seems like games might be interesting for is like I think I and a lot a lot of other AI people are starting to feel as though the lack of continual learning is a big problem for progress \>\> and having AI need to be able to figure out and get good at a game with very few tries \>\> is a really interesting \>\> thing too. Have you explored that at all? \>\> Yeah, so we're actually working with a super smart PhD from Rice um and some researchers from Princeton right now who are looking at optimizing prompts based on results to learn from them. \>\> And the initial results aren't great. Um but then quickly there's progress. And I think that that's \>\> the initial results of what? \>\> Like the first attempt, the first attempt of doing that, right? Aren't great, but then you quickly see progress. They're already seeing it with current models. \>\> Yes. Um and and and you quickly see that \>\> but it it goes to that problem space we were talking about, right? Like the concept is there. \>\> And one of the things that I've learned in training in the consulting that we've done is if you can shift your mindset to be no not oh this model gave me a wrong answer, but it had the wrong context, \>\> you take so much more power back. And I think that's the right way to think about them. These models can do such incredible things that if they're doing something wrong, \>\> it it might not be your fault. It might be that you need to prompt them in a weird way to get them to do that thing, but it's very likely that they can do it. \>\> That's interesting. So So are you saying that you believe this or they they believe this that um we're actually we actually might be closer to continual learning than we think because we can start at the uh layer of optimizing their own prompts and they're not bad at it. I think we're both closer and further away. So like I I I'm I'm not sure. Yeah. I'm not sure. What I'm saying is it's a tractable problem. \>\> Okay. \>\> I'm saying it's a tractable problem, \>\> but it requires a different skill set because and I don't know, right? Like I'm not somebody who's doing the reinforcement learning and doing these model training runs for these biggest models. But it would seem to me that the \>\> if you're able to get a model to reflect and to think about its learning and then train on that, you will get more of that. Yeah, \>\> right. And you should be able to prompt the model and think of tools and think of ways to get it to do that and be opinionated, be prescriptive with it to get it to do that in a way. And maybe you, you know, that that has some downsides where it's going to get more narrow and do that more frequently, but then maybe you can think about another one and then you can build on top of it. And so that's why it might take longer because it \>\> work needs to be done. Yeah. \>\> And that's the kind of work that we're looking at doing. Um, but so I don't know. I don't know if it's you can I it there's clear research that shows that AI can help prompt itself to get better. I think Disp is like a really cool example of something like that. \>\> Which DSPY? \>\> I've literally never heard anyone say it out loud. \>\> I've always pronounced it DSPY in my head. You think it's \>\> Disp? I think DSPI. I've heard DSp. \>\> DSP. \>\> So, one of those made up between the letter with the letters DSP and Y. um is a cool example and and I think that some of this became very clear to me in diplomacy where when we started the models couldn't play the game then we made some iterations and then you got large models to play the game and there's some existing research that showed that lot large models could play and there's cool research that self optimized the prompt so that GPT4 could barely play but we put more and more work to it and we built tools to help us iterate quickly and then we got to the point where dev so small can play right And that it was hard, but you learned a ton. And I just don't think \>\> you're in this we're in this weird time where there's an opportunity cost where if you spend a lot of time to solve one problem, it better be worth it \>\> because you can solve so many other things. \>\> And is it \>\> to make it worth it in the economy is is maybe tough, but I think if you make it worth it to yourself, \>\> then then it's definitely worth it and then it can have value economically. Um, so that's that's how we're \>\> having it be worth it to yourself, I think, is a underexplored uh path for entrepreneurship that is very helpful \>\> because it often takes a really long time to figure out if it's working or not and you you'd probably rather just like if it's if it's purely an economic calculus, you'll probably give up a little too early. \>\> I think I mean that's a big reason why we're building this. Tyler and I have both worked in startups for a while. He's been running his own consulting company for four years and um you know I was co-founder of AI camp in 2021 been here worked at a company called Salt that had three pivots and found product market fit and like drug discovery but the fir this is the first time where we're making a company from scratch \>\> and the reason why is because we both love and think that there's a lot of value in the intersection of AI and gaming. \>\> Yeah. And \>\> like it's and and not only because it \>\> I I truly believe that our environments are we're going to make models better \>\> but also \>\> because it will make people care and also less fearful. \>\> Like one of the things that we see in consulting is there's this knowledge gap growing. Simon Willis's written about it and so has Andrew um where people who are using these tools are the least fearful about them because they get it. they see where it falters. They see how they can use it to get better. \>\> But as people don't adopt them, whether because they're busy or they are have had bad experiences with the really initial like early version or um for any number of reasons, many of which are justified, then you can get fearful and angry. And so you get this gap that starts to to to occur. But with games, it was so cool when we launched Diplomacy. We had almost like 50,000 unique viewers hop on for a week, watch what admittedly was not a super entertaining interface. Like you could see them chatting and it was just panning back and forth. And \>\> I think we had some good a good soundtrack on it. Um, but \>\> they could see many of them were not AI people. They were people who came from the gaming side. And it became less scary. You could see it make mistakes. So you can see it take a different strategy that you know isn't the optimal one or every once in a while you see it do something good and so it becomes much more relatable and I think games are very powerful in that way and so that's another reason why I think that what we're doing is important. \>\> How did you get into games? Like why do you care about it? \>\> Yeah. I I think I've always learned a little bit differently than other people and games have been one of the ways I think I've learned the most. Um, when I was really young, we, you know, one of my friends taught me multiplication in kindergarten with like the beads on like an abacus. \>\> Oh, really? \>\> Um, and so that was advanced math, right? And then at one point in elementary school in advanced math, they put you in front of the 24 game. Have you ever played a 24 game? \>\> I got four. \>\> I never I never even smith sniffed advanced math. I don't think that they would have taught me that. \>\> Sure. Um they give you they have like this little card with four numbers around it and you need to find some way to make those four numbers make 24 and then you tap the card. \>\> Okay. \>\> So \>\> like Sudoku. \>\> Yeah. Kind of. You know it could be like you know 2 4 6 and 12 and it's like 6 minus you know 2 is 4 you know like and you you figure it out. Um, and I'm like thinking through that was totally the wrong solution, but anyway. Um, yeah. And then, you know, we mentioned Runescape and a lot of these other games and learned by building mods and many people who I've talked to like in these conversations raising money, but also at every at a lot of other places, some of the smartest people that that I've met have had similar experiences where they played some game and got something really good out of it or they were modding a game and then that brought them into their journey. Like I was on a one of the first Minecraft servers ever and some guy that I didn't know hopped on Skype for four hours to help me build a computer from scratch. \>\> That's sick. \>\> Like it it just you have this weird connection and and I think that there's a lot of value there. I do think that \>\> there are downsides, right? \>\> Games are not real life. Yeah, \>\> they are they can be practiced. You can learn, \>\> but if you get stuck there forever, that's not good. \>\> That's why in my head games are a good start. that was a big part behind the name. Um, but \>\> I do think that there's also a world where there will I think that there's a world where you can make a game that brings people back to reality to a degree. \>\> Um, I think Pokémon Go was a really cool experiment. I think if they had more of a fleshed out game that they could have had something way bigger. Yeah. \>\> And I don't know if you remember that moment in time. I do. But it was crazy seeing everybody out at monuments just, you know, I was in Boston at the time, seeing massive crowds along the reflection pond. Yeah. Where everyone was just around and and and doing the same thing. And like that sense of connection was really special at the time. And so I think that there's just a lot it's something special about games. \>\> You're making me think of uh I I used to love games like video games growing up. And during the pandemic, I bought like an Xbox cuz I was I was like, "Oh, it would be cool to, you know, play Call of Duty and at socialist. I'll have something to do." It was just when I first started every and I was like stuck in my apartment, so I was lonely and I like logged in and immediately just got merckked by an 11-year-old and called it again. I just never played. \>\> But I do I actually do really really like video games and I kind of I kind of miss playing them. I spent so much time playing Madden with my best friend growing up. \>\> What was your What was your top sports game? \>\> I'd say in college there was just this constant cycle of \>\> um FIFA and NHL. \>\> Okay. \>\> And so, you know, playing that a bunch and and it's funny because a lot of it's social, right? Like there are single player games and and a lot of people play them. Um, but I do think a big part of it is social cuz even if it's not even if it's single player, even if you're playing alone, the community of other people who play that game is a big part of it and seeing how you can do something that others haven't yet or \>\> that you tried something new and that you're comparing notes. That's why like a little bit of some people think that you're going to have games that are tailor made for you or movies that are tailor made for you and you exclusively. \>\> I'm as bullish on AI as the next guy. Uh, but I think that shared experience is so important. So if it's something that could not be experienced by somebody else, I think that's actually bad. \>\> Yeah. Interesting. Yeah. I I also I played so much Halo growing up. Were you a Halo guy? \>\> I I was I got handme-down PlayStations from my cousin. \>\> Okay. So you're not an Xbox guy. I was never But you know, then you grow other people have it and Halo is such an iconic franchise. \>\> What was your top What's your top shooter game? Like first person shooter? \>\> Modern Warfare 2. \>\> Okay. Yeah, that was good. That was that was one of those that one of the eras. Yeah. And actually, so in a similar way during the pandemic, started getting a little bit back into video games. Um had I had played some Fortnite. Great. Then it started getting real sweaty and you know, can hang, but \>\> at some point you want to be able to play with friends. Um \>\> then most recently though, my headset's broken now. Um, I started playing VR \>\> and it was \>\> not something I had really expected to do a whole lot of. Yeah. Um, but in the same way I or around the social component, \>\> I started playing population one, which is essentially Fortnite in VR. So, you're physically ducking, you are physically reloading, you physically moving. \>\> And two of my buddies from college were playing. And you play on teams of three. So, we had the perfect number of people to do it. And it became something where you come on, there's a headset built in, there's a microphone built into your headset, so you're immediately talking. You're talking to each other. \>\> That's cool. \>\> And it's one of the most fun gaming experiences I've ever had. You're physically in a game of Fortnite. You can only play for like an hour and a half, and if you don't play for a little bit, then you start to get vertigo when you get back. It's almost you have to like get over that. Um, \>\> but it's pretty incredible. I I'm still jury's out on if I think VR is going to be a huge platform in and of itself because I don't know how many people want to be fully disconnected from the real world, but it was a whole lot of fun. \>\> Yeah. I I I miss I miss gaming. I miss like getting home after school and like logging on to \>\> Yeah. \>\> matchmaking in Halo or whatever. \>\> Um or Counter Strike or, you know, all all those games. I never I also Yeah. I never I tried VR a little bit, but I never really got into it. I think probably cuz I have glasses. It's just harder. \>\> I've heard that a lot. \>\> Yeah, \>\> I I do think glasses will become a I'm wearing the Metamor bands right now. The I use them as my AirPods. \>\> I see you all the time like walking out of the office and you're like talking to yourself and I'm like, what is why is you talking to yourself? And it's like you're on the phone on your \>\> on your Ray pants and I'm like what the \>\> It's really cool. I'm I'm like I'm a big fan and and I think that more and more people will use glasses as a form factor for computing. \>\> I don't think that \>\> they're going to replace computers or cell phones. I think that there's room for both for all three. They're very different. Um I like personally that they don't have a screen. I I imagine that's not long for the world. I imagine they'll start \>\> Yeah, I thought that they were I thought that the new one is like projecting. Is it not doing that? \>\> I imagine they they'll get there. I like that it doesn't have a screen. I like that it's just I can talk to it. Um I think that it would be a pretty bad experience if I started talking to someone and then they were like, "Oh, sorry. What?" \>\> Yeah. \>\> Cuz they were looking at something on a on a glass on glasses. So, \>\> you know, I expect the incentives to push it that way. Um, but I do think right now it's a more human piece of technology. And I think a lot about like people taking pictures of their kids and their kids are imprinted on this box that's between you and them \>\> and they're looking at it and they see you getting joy out of it. So they're like imprinting there versus you turn this on, your hands are free. You're good. You're in the moment. At a concert, you're not like this. You're just in there. Like I think those are I love the concept of technology that makes us more human. \>\> Yeah. What are you I mean you're you're the guy that before you got really really busy fundraising all that kind of stuff. Um in addition to doing consulting you were writing amazing stuff on every and you were the guy that if I wanted to know like what was interesting or cool you would have a really good read on what got released and whether it's or not. Sure. Like what are you excited about right now? \>\> And I know you've been busy fundraising so you may not have your finger on the pulse as much as normal but yeah I'm curious if there's anything that that's exciting you that's on your mind. \>\> All I can think about recently is games. Um, \>\> yeah. \>\> But \>\> anything else in games that like are not not specifically that you're working on, but just generally is going on that that you think is exciting? \>\> Well, GTA 6 got delayed and is coming out next year and so it's the most expensive game that's ever been made. Uh, \>\> I think the last one came out when I was in like high school or something, right? \>\> It's Yeah. I mean, these it's been over a decade and these are like this is a billion dollar game. \>\> I mean, just the cultural moment of that I think is gonna be interesting. Um, and then on the AI side, you know, I think that maybe it's less about what's happening right right now, though. I would say a lot of the stuff Google's doing is really cool. \>\> Yeah. \>\> Um, GD3 big Google stand. \>\> Yeah, I am a big Google stand. Um, Deus, if you ever want to cut an Angel. Um, but the the connection of a lot of what they're doing is so interesting and the constraints that they have I think are so cool because \>\> AI is almost existential for their business on search \>\> and so they want to be able to use it there which means it has to be fast, it has to be reliable, it has to be able to go check other sources, has to be good. Um, so having constraints when you could do anything I think is actually helpful. But then they also have Genie, which renders anything. \>\> Yeah. \>\> Quickly that's experiencable. I don't know how that will be interacted with gaming. Maybe it makes up some of the most renderable expensive. \>\> I actually have a thought about that which I think you'd be into. Um so I had this guy, he's the CEO of Daycart, which we talked to a while ago, \>\> and they have this really cool um video to video model uh where it takes any frame of video and then turns it into something that looks like a video game. Yes. \>\> And uh and they had this thing where for example if you pick up a tissue box and you like go like this it like turns it into a gun and shoots it. \>\> And um I think that's an interesting future for gaming. \>\> I agree \>\> because like right now to make to make GTA you have to handcode all of the interactions and that's why it takes a billion dollars and like many many programmers and artists to do it. And with videotov video generative models, one, you could just generate it from like live video, but two, you can vibe code a really simple game and then you can reskin it with generative AI to like look like a AAA game. \>\> And I think it it lowers the barrier to making awesome games to almost anyone now, which is really cool. I I I think that that not only does it do that, \>\> but it also \>\> lets you do things that were otherwise super computationally intensive like ripples on water or reflection of light without having to run it at all like that. So I could definitely see that. That's cool. \>\> Um and and that's related I think generally or like you know they're doing that but they're also doing a lot of different things that are cool in AI and I think I mean one of them is life sciences. I think it's a really underappreciated world and it's one that I was fortunate to be deeply involved in getting to work with like the Ellison Medical Institute and others in my last startup where \>\> like I mentioned it earlier but Alphafold literally took something that we had PhDs taking six years to do and turn it into something that takes 20 minutes. \>\> Yeah. \>\> And as far as where I think AI is having near-term impacts, I think it's software, life sciences, and education. Mh. \>\> Those are three I see today having massive impacts. And I love robotics. I used to work at MS Robotics. I would love for that to get there. Self-driving is seems to be on the precipice. Um I I'm huge Whimo fan. Take them all the time \>\> and \>\> but \>\> those three are right now seeing huge pl like huge impacts because \>\> they're perfectly their problems perfectly suited for for AI software. \>\> We have compilers. We wrote the code. we know what would render or not. It's a solvable problem. Yeah, it's great for language models. It's great to just do reinforcement learning on just code and then also on diplomacy as code so when there's new things it can generalize, right? Like that's awesome and you can do that. Life sciences, there's a ton of information out there. We just need people with subject matter expertise to combine them and look at these different interactions and to find ways to simulate these processes. there's a real chance that in the near term people find a way to turn dollars into longevity. Like that's crazy. \>\> And then on the third side, education. You know, you talked about being excited about your nephew um learning about these starting to enter that world of learning. And I think it's going to be really interesting to see. I don't know for sure. Um, I got I loved that I got to interact with so many high school and college students at AI camp when they were going through that learning journey. I think it'd be it's tough to be a high schooler right now in the world. Um, and the education system hasn't caught up yet and there's this huge incentive to use AI to do your work, but then you really learn about it. So then what do you care about? Like tough. the generation afterwards, the generation who's going into their why, why why phase that can have AI to answer those questions, \>\> it's amazing. \>\> They're going to be \>\> so smart. Um, and what that intelligence looks like, I don't know, \>\> but \>\> to be able to constantly be exploring and to get answers and maybe there, \>\> sure, there's probably negative externalities from that definitely, right? \>\> But there's also probably a lot of positives. \>\> It's really good. I think I just remember being like in fourth or fifth grade and being like I want to write a novel and people were like what are you t like \>\> and why has it been so hard right it seems easy \>\> I don't know if you had that experience but I was like oh you just write it and it's good \>\> yeah yeah \>\> um so like having AI to answer all my questions and stuff I think would have been just fantastic uh okay so that's the stuff you're excited about what's like the most overrated thing or what what pisses you off in AI right You know, I don't think a lot pisses me off because it is pretty important. So, people talking about it, I think, is is probably net good. There's definitely people shilling things that aren't really going to solve your problems. Um, I think the thing that maybe I worry about the most is in the same vein of education, we now have this leverage that makes somebody who's an expert in something 10 or 100 times more powerful. But it also it does the work of someone who was a junior in that field. \>\> So how do you bridge that gap? How do you financially incentivize somebody to learn and make mistakes and get better knowing that this tool is here to keep pursuing there? Like what the the maybe maybe they're overblown, but it seems like the job numbers of people graduating college is getting are getting crushed right now. And I imagine a part of that is because you don't need people to do that blocking and tackling that you needed before. And so and paying them to do so is a big cost. \>\> Yeah. \>\> And so what does that look like? \>\> I have I'm so not worried about that, which is interesting. And I think actually you're one of the people that made me not worry about it. \>\> Um I use this anecdote a lot and I don't think I don't know if I've ever used it to your face. So I'm I'm curious about about this. Um or I'm curious to like tell you how you've impacted how I think about this. like um when you joined every and you said you wanted to write and you wrote your first piece, \>\> it wasn't good. \>\> Yeah. \>\> And it was not good to the degree that we could not have worked with you without AI. \>\> Yeah. \>\> And what was really interesting is and I I've worked with a lot of young writers and so I can tell pretty quick your rate of progress. Like every time we talked you recorded it, you made prompts and you never made the same mistake twice. And so your rate of progress was within like three or four months you had made like a year or two years worth of progress and that just kept happening. \>\> And so let's assume that uh the job numbers are are down for young for young people because actually because of AI because people are not hiring them. That is a gigantic gigantic management mistake that companies will begin to correct as soon as they realize that a 23-year-old which hadt is cracked. And if you give them any amount of mentorship, they're going to do amazing stuff that you could never they never could have done before. \>\> And I think there's, you know, the question about, well, they're not actually learning the underlying skill if they just have the AI do it. They they they are because they have to because if the AI messes up and they care about it messing up and they should because that's the way to do a good job, they're going to go in and learn the stuff and they have a great tutor to help them figure it out. So, I feel extremely excited for young people and I think to the extent that managers are not hiring them, that's on them and they will figure that out pretty soon because they'll they'll be like, "Oh my god, like I hired this 23-year-old and like it totally changed my whole business." Um, my dad is like this. He's downstairs right now and he owns like a few cemeteries in Indiana and like he has this 23-year-old who just completely changed his entire business. Um, and uh, so, so I think it's going to flip from there. Maybe it's there right now, but I think it'll flip from there to like mid-career folks pretty soon. Um, because I think I think the kids are going to be all right. \>\> Yeah. Yeah. Yeah, the thing I agree with is that I think that probably the solution to this is some form of an apprenticehip kind of right where you're able to quickly learn about something and you're doing something that you care about therefore you will spend the time on it and therefore you will care about \>\> what it is that makes it good or not. Right. If you don't care about it that's going to be hard. \>\> Yeah. Um the counterpoint is I don't think that without my experience on the training and the AI and that side of things that I mean brought in to the every side to then have the chance to do it. So what is that skill that they're being brought in to do \>\> besides the raw material, right? besides the ambition, the ability and just doing that when you're comparing them on the market with people who do have that and maybe it becomes the people who are on the market don't have that hunger. literally just like I'm I'm hungry and I'm I and I'm willing to try new things instead of the things that are currently being done. And like I think that's \>\> but if all things equal if you have somebody who's hungry and doesn't have experience versus someone who's hungry and does \>\> I would take the one that doesn't have experience because the person who has experience their experience is wrong \>\> because the whole landscape just changed \>\> and it's really hard to get someone who's like already in their career and knows how they do things. I mean, you know this because this is what we do is we take people who are mid-career and we train them how to do something else. And it works, but it's hard. Yeah. What's easier is someone who's hungry \>\> and uh has hasn't done it before and doesn't have a whole set of things they have to unlearn and is just going to like figure it out. \>\> Maybe. \>\> Yeah. I hope so. We'll see. \>\> I hope so. Um but that's something I spend a lot of time thinking about. \>\> Yeah. Yeah. I I feel you. I mean I I think it's it's very important that it's not all going to be rosy and um there are anytime there's technology shifts there are downsides and tradeoffs. \>\> No and yeah I think another example of that right is like big companies who are going to be able to do more with less \>\> and you may see them \>\> and are seeing some in some industries already cut headcount. But to your point right like if you cut too much headcount then you realize oh man we could have just done way more with those people \>\> then that's a mistake. And then you start seeing groups of those people be able to do way more on their own and out compete on these niches and then take away parts and create many many many more startups than have ever existed. So I'm excited. It's going to be a little rocky. \>\> Yeah. \>\> But I'm excited. \>\> Well, reality is typically rocky. So \>\> indeed much rockier than games. \>\> Um all right. This is awesome. Uh, so if people want to find you, uh, want to participate in your tournament, want to just generally follow along with what you're doing, where where can they find you? Goodstarlabs.com. Goodstarlabs on Twitter. I'm alxai on Twitter. And you can read up my writing on every. \>\> Amazing, Alex. Thank you. \>\> Thanks, Dan. Oh my gosh, folks. You absolutely, positively have to smash that like button and subscribe to AI and I. Why? Because this show is the epitome of awesomeness. It's like finding a treasure chest in your backyard, but instead of gold, it's filled with pure unadulterated knowledge bombs about chat GPT. Every episode is a roller coaster of emotions, insights, and laughter that will leave you on the edge of your seat, craving for more. It's not just a show, it's a journey into the future with Dan Shipper as the captain of the spaceship. So, do yourself a favor, hit like, smash subscribe, and strap in for the ride of your life. And now, without any further ado, let me just say, Dan, I'm absolutely hopelessly in love with you.