The Every team live-tests Anthropic's Opus 4.8 across coding, writing, and knowledge work, comparing it with OpenAI's GPT-5.5 in Codex. # Channel Identity Observations: LIVE VIBE CHECK — Opus 4.8 ## Explicit Self-Description (Creator Claims) ### Who They Are - **Dan Shipper** opens by identifying himself as **"co-founder, CEO of EveryY"**, joined by **"Kieran Classen, GM of Kora"** and **"Katie Parrot, staff writer at EveryY."** - Recurring positioning statement, repeated nearly verbatim multiple times: > "Every is the only subscription you need to stay at the edge of AI. You can think of us a little bit like a frontier AI lab focused on the future of work." ### Claimed Scope of Activities Dan explicitly enumerates three pillars: - **Testing/publishing:** "We test models and write about them... Every day we have something new about how we're using these models in our work and in our lives." - **Products:** Kora ("an AI agent for your email"), Monologue ("a speech-to-text app"), Spiral ("a writing agent"). - **Training/community:** "We do live streams like this. We do courses. We do camps." - **Bundled model:** "You pay one price and you get access to everything that we make." ### Claimed Access and Cadence - "We get it early. So we put our hands on it... And the day it drops, we have a vibe check." - "We do these on the day models come out with Anthropic models, with GPT models, with lots of other stuff." - "We've been testing it for about a week." ### Claimed Organizational Scale - "We have about 30 people" (stated twice, also referenced via the model's writing: "I run a company of about 30 people"). - References a recently published report, **"After Automation"**, about internal agent use. ## Observable Presentation (Demonstrated Patterns) ### Intended Audience - **Knowledge workers, writers, and developers who use frontier AI models daily** — evidenced by discussion of coding harnesses (Codex, Claude desktop app, Cursor), writing workflows, style guides, and benchmarks. - Chat interaction suggests a **live, engaged viewership**: "we're getting to almost almost a thousand people. We're only like 13 minutes in." Dan also invites Anthropic staff ("Sam," "Dario") on-stream by name. - Audience questions are read and answered live (e.g., Christian's prompt-literalism question, Dominic's UI design question, Patrick's purple-accent comment). ### Demonstrated Content Format - **Live model-review streams on release days**, walking through a published written "vibe check" on every.to, with screen shares, benchmarks, and live demos. - **Structured evaluation rubric:** "reach test" ("do you reach for it every day?"), medal scoring (green = "really good model," gold = "paradigm shift"), plus named internal benchmarks: "senior engineer benchmark," "LFG bench," "writing benchmark," "cozy island" test, PowerPoint/deck benchmark. - **Recurring subscription pitches** mid-stream ("if you're here, welcome... subscribe to every"), framing the video partly as marketing for the paid publication. - **In-house banter and personality** ("keep yourself and your agent liquid cooled," hydration jokes, "chaos muppet" exchange), presenting a casual, team-based culture. ### Demonstrated Opinions/Bias Toward Candor - The team shows **willingness to criticize** alongside praise: critique of the Claude desktop app ("kind of a mess... it's slow, it's confusing"), acknowledgment that "Codex is still my daily driver," and caveats about FOMO: "you can just chill. GPT 5.5 is very good, too." ## Notes on Distinguishing Claims from Demonstration - The "frontier AI lab for the future of work" framing is a **creator claim**; what the video *demonstrates* is hands-on benchmarking, written model reviews, product ownership (Kora), and live commentary consistent with that claim. - The ~30-person team size, week of early testing, and product suite are **asserted, not independently verifiable** from this transcript alone. # Delivery Observation: LIVE VIBE CHECK: Opus 4.8—IT'S A MONSTER ## Format & Structure - **Live stream review format** with a host (Dan Shipper) moderating two panelists (Kieran, Katie), built around a published "vibe check" article they walk through on screen. - The host repeatedly **re-orients the audience and re-runs a promo script** at intervals. He states the company pitch near-identically multiple times: > "Every is the only subscription you need to stay at the edge of AI. You can think of us a little bit like a frontier AI lab focused on the future of work." This appears at roughly three separate points ([13:xx onward], ~[1708], ~[4847]), a clear stable habit of live-stream sponsorship/subscription insertion. - Segment structure is explicit and telegraphed: reach test → coding → writing → knowledge work tasks → live demos → outro. ## Host's Communication Patterns (Dan Shipper) - **Heavy filler and self-repair in speech**: frequent "um," repeated word starts ("we're going to we're going to," "it it it has some depth"). - **Enthusiastic superlatives and intensifiers**: "really really really freaking good," "very very very very exciting results," "it's a beast model." - **Signposting and handoffs**: constantly delegates with direct prompts — "Kieran, do you want to start?", "Katie, let us let's give us your reach test," "Kieran, do you want to tell us about your results." - **Interactive with the live chat**: addresses commenters by name ("Sam, I see you in the chat," "Dominic asked what about UI design," "Christian is asking..."), and invites guests on stream: > "Dario, if you're there, we'd love to we'd love to chat." - **Recurring comedic runner — hydration**: "Keep yourself and your agent liquid cooled" appears at least twice, and the stream closes on "make sure that you stay hydrated... Cheers." ## Panelist Delivery (Kieran) - Uses a distinctive **extended analogy habit** when characterizing models: > "the GPD5 is like a very eager, brilliant 25 year old" ... "4.8 Eight feels like someone that's seen the world." - Speaks in **evaluative, benchmark-driven fragments** with numeric evidence (times, scores): "Gemini 3.1 Pro finished the entire task in 3 minutes... GPT 5.5 took 13 minutes... Opus 4.8 extra high took 42 minutes." - Recurring phrase: calling the release a **"paradigm shift"**, repeated multiple times with explanation. ## Panelist Delivery (Katie) - **Self-deprecating and confessional tone**: "after going off and hiding in a corner and crying for about 15 minutes," "maybe I'm just like a chaos muppet." - **Repeated vocabulary around AI writing flaws**: "AI tells," "AI smell," "not X but Y," "timid scribe analogy" — a stable specialist lexicon used several times. - Explicitly flags uncertainty and limitations: "I don't 100% know if it's new model behavior because it just hasn't occurred to me that that might be possible." ## Shared Group Habits - **Collective repetition of the core verdict**: "Anthropic is so back" and "they should have called this Opus 5" recur across multiple speakers. - Casual slang is consistent across the group: "sick," "cold move," "banger," "nerd sniped," "glaze you." - Frequent **laughter-tagged banter** recorded in the transcript, indicating a relaxed, conversational register. ## Uncertainty Notes - Timestamps are bracketed transcript line markers, not verified wall-clock times; interval claims above are approximate. - The host's stammering pattern is highly frequent, but distinguishing deliberate verbal style from transcription artifacts of a live auto-transcriber cannot be fully resolved from this source. # Vibe Check: Opus 4.8 — Observation Record ## Participants - **Dan Shipper** — co-founder/CEO of Every (host) - **Kieran Klaassen** — GM of Kora - **Katie Parrott** — staff writer at Every ## Overall Positions ### Dan Shipper - **Headline claim:** "Anthropic should have rounded up to five" — the model is good enough to warrant a version-5 designation; subhead "Anthropic is so back." - Context: Claude had lost ground internally at Every — Dan is "super Codex"; even Kieran and Katie (diehard Claude users) had become GPT 5.5-curious, with Kieran using it 50/50 and Katie switching to Codex for much writing. - Opus 4.7 was "hard to use, pretty slow"; GPT 5.5 "in that Codex harness is just sick." - **Reach test: gold + green.** Gold for the model itself ("opens up way more use cases"); green only because the **harness** (Claude desktop app) doesn't get the most out of it — "Codex is now my operating system for knowledge work." - Notes labs usually trade off axes (coding vs. writing, agentic vs. cautious); this model "hit some magic thing" without such trade-offs. - Claims the "Claude is winning" narrative is inflated: Anthropic had a head start (Claude Code, ~May–Dec 2025); investors are "behind," feeding off vibes; OpenAI are "relentless executors." - **Caveat:** the release isn't a "4.5 moment" — GPT 5.5 users can "just chill"; anti-FOMO message. - Cites a NYT push notification mid-stream: Anthropic surpassed OpenAI as most valuable AI startup at $900B valuation (attributed to the notification, not verified by speakers). ### Kieran Klaassen - **Reach test: gold (paradigm shift)** — also gave gold to 4.5. Reasoning: 4.7 was usable but flawed; 4.8 is "very hard to find anything I don't like." It's "a model that just does everything and then the best." - **Paradigm shift rationale:** it "punches you in the face if you do something stupid," breaks out of familiar AI patterns, focuses on *why* it does something rather than pleasing the user — enabling more autonomous decision-making. - Prompting: 4.7 read prompts too literally ("too GPTish"); 4.8 reads between the lines again — "we didn't need to change our prompts in any way." - **LFG bench:** extra-high thinking is the sweet spot ("high is as good as GPT 5.5; extra high does things I've never seen GPT 5.5 do"); max is over the top. Example: breathwork app that adds a safety confirmation for Wim Hof (intense) but not coherence mode — contextual, human-like judgment. - Characterizations: GPT 5.5 = "very eager, brilliant 25-year-old"; 4.7 = too much fluff, boring; 4.8 = "someone that's seen the world… without having to show off"; plan writing is half the length. - **Design:** previously preferred Gemini; 4.8 is "as good maybe better," least AI-looking in the Proof redesign comparison (likes "every word has an author. Now you can prove it."). Current AI tell in design: "too much going on" — eye candy without value (purple accents are dated). - **Caveats:** Claude desktop app "very mediocre" (vs. Dan's "really bad"); GPT 5.5 is 2–3× faster; fast mode is probably expensive. Cozy island timing: Gemini 3 min, GPT 5.5 13 min, Opus 4.8 extra-high 42 min — "It's better, but it's also slow." - Cozy island benchmark: 4.8 first model whose birds "fly as a real bird" (prior models flew sideways); no prompt leakage into UI (GPT 5.5 "loves to put GDPR compliance boxes in designs"). - "Is 4.8 the best at design? Humans are." ### Katie Parrott - **Reach test: green (the holdout)** — calls herself "overly conservative"; splits time between Opus 4.8 and GPT 5.5/Codex (was "Codex pilled" writing the Codex guide). - Had been in an "LLM malaise," uninspired to code or write; with 4.8 spent days in a "vibe coding haze" (added an MCP agentic endpoint; built a site where AI Dan debates AI Pope Leo — "It's a discussion, Dan"). - **"Wow moment":** while writing about her agent editor, the model noticed the draft contradicted the code, flagged it, suggested a fix, and returned to writing mode — "I've never seen that before." Qualification: she's not "100% sure it's new model behavior because it never occurred to me that that might be possible." - **New writing benchmark** (work in progress): Opus 4.8 at high scored highest, 79.6 vs. 73 for GPT 5.5; second place went to Sonnet — "maybe I'd been sleeping on Sonnet." Caveat: extra-high not yet tested ("I'll improve that next run"). - Writing is subjective: she preferred GPT 5.5's intro line ("The strangest thing about automating a company is how much work it creates") over 4.8's. - **AI tells:** 4.8 left 13 tells across eight tasks (vs. 25 for 4.7, 21 for GPT 5.5); lingering tell: "not X but Y" constructions. - Strong on style pickup via style guides, which she's "near evangelical" about (pulled from her Tastemaker app MCP). - On the desktop app: disagrees it's bad — "I was perfectly happy… I wouldn't have known the Claude desktop app wasn't good if it hadn't been for Codex"; self-describes as "a chaos muppet." She moved from web app/projects to desktop MD files but won't use the terminal. - Live demo: prompted 4.8 to draft a piece on her "guardrails" skills, pointing it at local skill files ("go find my skills") — it found them, described them accurately, and found a stray Codex draft she had to tell it to ignore. Dan highlights that the model's visible reasoning is itself valuable to read. ## Shared Benchmarks Cited - **Senior engineer benchmark** (Dan): 4.8 scored 63/100 (~30 points above 4.7) — but *only at extra-high reasoning*; at high it scored in the 30s–40s. Beats GPT 5.5 by one point. 5.5 more willing to slash-and-burn; 4.8 more conservative with a transition plan, and "fairly honest about what it had done and not done." - **PowerPoint one-shot** (Dan, via Mike Taylor's consulting benchmark): best deck he's seen in one shot; minor contrast/spacing issues but presentable. - **Cozy island / Proof redesign** (Kieran): comparative spatial/design test described above. ## Unresolved Tensions - **Model vs. harness:** all three praise the model; Dan and Kieran stay on Codex largely due to harness quality (slow, confusing multi-tab Claude app vs. Codex's in-app browser and `send_input` cross-thread messaging). - **Speed vs. quality:** 4.8's extra-high quality comes at 42 minutes vs. 3 for Gemini. - **Gold vs. green:** genuine disagreement on paradigm-shift label (Kieran gold; Dan gold/green; Katie green). - **Writing preference split:** Katie prefers GPT 5.5's intro despite 4.8's higher aggregate score. # Speaker Attribution & Provenance Record ## Video Overview - **Title:** LIVE VIBE CHECK: Opus 4.8—IT'S A MONSTER - **Type:** Live stream model review (Every's "vibe check" series) ## Identified Speakers & Roles ### Dan Shipper — **Creator / Host** - Self-identified: co-founder and CEO of Every - Leads the stream, conducts reach tests, presents senior engineer benchmark results and PowerPoint/deck demo ### Kieran Klaassen (transcribed as "Kieran Classen") — **Guest / Panelist** - Self-identified: GM of Kora (Every product) - Presents LFGbench results, cozy island benchmark, design comparisons across models ### Katie Parrott (also transcribed as "Katie Parrot" / "Katie Parish") — **Guest / Panelist** - Self-identified: staff writer at Every - Presents writing benchmark results, AI tells analysis, live writing demo ### Chat Participants (unseen, unnamed in transcript) - **Christian** — asked about prompt literalness in 4.7 vs 4.8 - **Dominic** — asked about UI design - **Patrick** — commented on purple accents as AI design tell - **Sam** — declined to join on camera ("better experienced through the medium of text") ### Mentioned, Not Present - **Dario (Amodei, implied)** — invited to join stream, did not appear - **Mike Taylor** — credited as creator of the consulting deck benchmark; leads the tech side of Every's consulting practice - **Kate, editor-in-chief** — referenced in Katie's demo draft text (as written content, not a speaker) ## Promotional Segments (Flagged) Two explicit mid-stream promotion blocks delivered by Dan Shipper: - Promoting **Every subscription** ("the only subscription you need to stay at the edge of AI") at every.to/subscribe - Promoting products: **Kora, Monologue, Spiral** - Cross-promotion of content: **Codex guide**, **After Automation** report, prior vibe checks Note: entire video functions as promotional context for Every's model-review content. ## Referenced Sources & Claims ### First-Party Benchmarks (Every's own) - **Senior engineer benchmark** — Opus 4.8: 63/100 at extra-high reasoning; ~30s–40s at high; beats GPT 5.5 "by one point" - **LFGbench (Kieran's compound engineering flow)** — single 40-minute prompt run; best results at extra-high reasoning - **New writing benchmark (first run, work in progress, built by Katie)** — Opus 4.8 at high: 79.6 vs GPT 5.5: 73; Sonnet second place - **AI tells count** — Opus 4.8: 13 across eight tasks; Opus 4.7: 25; GPT 5.5: 21 - **Cozy island benchmark timing** — Gemini 3.1 Pro: 3 min; GPT 5.5: 13 min; Opus 4.8 extra high: 42 min - **Deck benchmark** — one-shot PowerPoint via Mike Taylor's consulting benchmark script ### Third-Party Claims (unverified in source) - **New York Times push notification** (received live by Dan Shipper): Anthropic surpassed OpenAI as most valuable AI startup at $900B valuation - **X (Twitter) algorithm** — Katie states the X team externalized the for-you-page algorithm on GitHub; used in tweet-writing test ### Referenced Models (compared, third-party products) - Opus 4.6, 4.7, 4.8; GPT 5.5; Codex; Sonnet; Gemini 3.1 Pro; Claude desktop app; Cursor; Wim Hof method (breathwork app demo content) ## Provenance & Availability Notes - Speakers had **early access** to Opus 4.8 (~1 week, received "Friday afternoon") - Full vibe check published at every.to (not yet on homepage at stream start; link shared via tweet) - Katie's writing benchmark scores at extra-high reasoning **not yet run** at time of stream - Benchmark writeups stated as "coming to Every soon"; benchmark suite described by Dan as disorganized - **Comment availability:** live chat active during stream (near 1,000 viewers ~13 min in); no indication of post-publication comment availability - Transcript contains transcription errors (e.g., "Katie Parrot/Parish," "Kieran Classen," "EveryY," "open 4.7") — names normalized above with uncertainty noted Hello and welcome. It is a model release day. I am Dan Shipper, co-founder, CEO of EveryY. I'm here with Kieran Classen, GM of Kora, and Katie Parrot, staff writer at EveryY. And we are psyched about this model, are we not? \>\> We are. \>\> Um, so we've been testing it for uh for about a week now. And one of the, you know, we're going to we're going to do a little bit of warm-up. If you if you really want to know everything that we think, our vibe check is live. It's on every.TO. Um, actually, let's see if it's on the homepage yet. It is actually not on the homepage yet. Um, but I'll drop a link. I just did a tweet that has has a link. We have the full we have the full vibe check already happening on on X, but on sorry, on every and and a tweet on X. Um, we've been testing it for about a week, which is really great because when open 4.7 came out, I don't know if you guys remember, I certainly remember getting a message from Anthro being like, "The model's live." And I was like, "What model?" Um, and then we did the full vibe, we did the full vibe check live, which was a bit stressful, but we are here. We are here to tell you this model is awesome. Again, we've been testing it for about a week. Um, one of the things that we've been saying internally and we've we actually said this to Anthropic is like you should just called this Opus 5 because it's like it's really really really freaking good. Actually, I think Kieran Kieran even said it should have been Opus 5.5. Um, \>\> which that will be funny. \[laughter\] \>\> That would be very very funny. So, uh, I'm gonna get I'm gonna just gonna take two seconds to get myself set up here so that I can screen share through the, um, through the vibe check that we did. Well, what we're going to be doing today is we'll go through all the testing that we've done. Um, we will be drinking a lot of water because we're obviously staying hydrated. Hopefully, everyone's got got their hydration. Keep yourself and your agent liquid cooled. We're going to go through all the testing that we're that we've done. We're going to tell you exactly what we think, what this model's good for, what it's not good for, all that kind of stuff. Um, let me see if my screen share Oh, man. My screen share seems to be busted, which is not what I not what I was hoping for. \>\> Who needs screens? Just \>\> who needs screens? You're totally right. Um, \>\> imagine it as a podcast and you just sketch out the world. Would someone mind either Kieran or Katie, would someone mind screen sharing the Vibe check page? I'm going to throw the link in the uh in the chat so that y'all \>\> who anyone who's watching and listening can can can take a look at it. But I do like to go through that um with everyone. Here we go. We've got Oops. Uh we've got this is uh that's the homepage. Oh, there we go. Sweet. Who who is controlling this right now? Kieran. Okay, so vibe check. Opus 4.8. Anthropic should have rounded up to five. That's your headline, folks. Sub head. Anthropic is so back. Um, you know, I I feel like Anthropic was in this like interesting spot where obviously they've been riding the the whole claude code wave. like they started this this this way of thinking about engineering and then that has bleeded into the rest of knowledge work and they've been riding that wave for like a year. I feel like they've got they've got the mandate of heaven right now publicly with Opus 47 and maybe the last couple months. It's it was the it's the first time internally at every in a long time where I think that you could you started to feel some of the cracks of um we weren't using Claude for everything. Um, I personally am super I was and still am super super codex code. Like I love love love codecs. Um, but Kieran and Katie are both real like diehard Claude stands. Kieran for coding, Katie Katie for writing. And even you guys were a little like getting a little GPD 5.5 curious. Um, and so the the like \>\> more than that I would say even I I used it 5050 which is like \>\> which is that's a big deal. And Katie, you switched out of Claude and into codeex for a lot for a lot of your writing, which is also a big deal and had not happened before. And um I this model with 4.7 was that great. Like there's some there are definitely some good things about it, but it's like hard to use, pretty slow, um hard to get the like power out of it. And GP 5.5 in that harness in the codec harness is just like sick. So uh it's really great. I feel like Enthropic is so back with this release because it is a model that is both incredibly good at programming. It beats um it beats GBD 5.5 by a hair on our senior engineer benchmark. It did really well on uh Kieran's LFGbench tests. We'll get into all of that pretty soon. It's also an incredible writer. So, what we found is that it um it writes very human-like pros that doesn't have as many of the like AI tells or AI isms as uh as you as you might expect. Very good at copying style or mimicking style. It's also just generally really good for knowledge work. It like produced the best the best PowerPoint I've ever seen in one shot. So, there's a lot of like really interesting stuff to get into. We're going to get into all of it, but first we're going to start with the reach test. Let's scroll down. So, this is our this is our best um indicator of whether a model is good, and if so, what it's good for. It's like, do you reach for it? Do you use it every day? And if so, for what. Um and uh I've I've been doing a lot of talking, so I'm going to let I'm going to let you guys tell us your reach tests. Kieran, do you want to start? Um, it looks like you gave this a gold medal paradigm shift model score. \>\> Yeah, and that's a big one. Like I gave that to four or five as well. And and I feel like it's harder to see that shift, but I really think there is some kind of shift happening here. Uh, and it will be hard to feel that shift since we'll get into that as well. like the codeex apps so so amazing and the like the the code cloth code app is kind of behind there \>\> but it's my favorite model and and that is that is a very important thing 4.7 was not my favorite model I I used it and it worked but there were things I didn't like about it and with 4.8 eight. It's very hard to find anything I don't like about it. And I think that's very special. Like 5.5 is amazing at certain things, but there are plenty of things about it that I don't like. And I think the par paradigm shifts really is in it's a model that just does everything and then the best, which is kind of cool. and and we'll get into the best and how it does the best because there are thinking levels that will influence this and you need to pick the right thinking modes to get the best out of this for the task you do. Um, but so we we got early access and for some reason the model just sucked me in. Like it just captured my attention and I was like, whoa. Like it's just a feeling you have sometimes where you're like, wow. Okay. Uh, and it wasn't because it was saying what I wanted. It was actually because it was saying what I didn't want. And it was making was breaking out of the like I'm familiar with AI stuff like it it just broke out of something that I always thought AI was and it showed a new thing and and it's very early and we'll see what it is. But that's why I think it's a paradigm shift. I think this is the first model that can break out and like punch you in the face if you do something stupid instead of like other things. And for me, that's why it's a paradigm shift. And \>\> yeah, it's it's good at coding. It's good at writing. I think it's it's less verbose than 47. It's a little bit more than GP 5.5. And it's also more about why it does something instead of this is what you need to do. So like that's really the difference between GP 5.5 which is just like it's a task list. Let's do it. And open for 8 and other anthropic models is why I love them so much. It's more like hey but okay why are we doing these tasks? Because if you know why it means it can make more decisions. And now since it can make decisions not because it wants to please you only. I feel that's a paradigm shift. \>\> I totally agree with you. Like the way that I would uh one way that I would I would frame it is um it's it's good at pushing your frame. Um and I think you see this in coding. I actually saw I was using it for just some like interpersonal psychology stuff recently and it's just so good at being like here's a perspective that you might not have considered that uh it it I feel like it's been hard to get a model that doesn't glaze you and also doesn't just like take the other side just to do that. And I I like this model's ability to gently push your frame in a way that feels helpful. Katie, let us let's give us your reach test because you're um you're using a lot for writing, you're using a lot for editing, you're moving away from the clawed universe. Tell us like tell us what your reach test is. Tell us where you are now. What are you using? What did you not like? \>\> Yeah. So, I really love this model. I should say that up top. Um you'll see that I I did I am the hold out at a green instead of a paradigm shift. And that does come down to the fact that I have been splitting my time between CL between this model between Opus 48 and GPD55 and the codeex app. I have been codec pill largely through preparing our codeex guide. Go check it out. But you know I like that was really me being overly conservative I think with with handing out a paradigm shift label. Um because you know I really I really love this model. To be honest, I had kind of been in a malay an an LLM malaise. I wasn't coding as much. I wasn't even writing as much because I just wasn't that inspired by the models that were available to me. And we when we from the time we got access to Monday or Tuesday morning when I had a meeting with my med management provider, I was basically in a vibe coding and vibrating haze with Opus 48. I added a new MCP agentic endpoint to my app. I made a little agentic website where you can have AI Dan Shipper and AI Pope Leo debate AI and the future of work. \>\> Who won? \>\> Um, it's a discussion Dan. \[laughter\] Um, it's actually really fun to to watch it. Um, and then I also wrote like I wrote about that experience and this is like something that really this is like my wow moment. I had I had a a a you know a chat open in the cloud code desktop app which is the surface I use for working with cloud and I was coding up this agent editor and editing it to my website. Please don't go look at it or you will bankrupt me. Um but um and then I just started writing about it and it noticed that what I was saying in my draft was out of line with what the code actually said and like flagged the issue. \>\> Sick. suggested that a fix for it and then we went right back into writing mode and I was just like I've never seen that before. I don't 100% know if it's new model behavior because I just hasn't occurred to me that that might be possible. Um but with this model I was like I wonder if I could just do this in one chat instead of switching over and it worked. \>\> Well, it's really interesting because that that thing you just said like I don't know if this is possible before because it never occurred to me that it would be possible. It it I think that is what a good model does. It allows you to do things that you didn't even realize were possible that maybe you could go back and and an old model would be able to do it if you prompted it right. But this this kind of like what a real good frontier model does is just does where you're like, "Oh, I didn't even realize that was possible." Or it occurs to you to try something that you didn't think was possible before. So, I think that that's a that's a big deal. Sam, I see you in the chat. If you want to join or anyone from Anthropic, if you want to join, we'd love to have you on the stream. Uh we're we're getting to almost almost a thousand people. We're only like 13 minutes in. So this is going to be I think this is going to be a big one. We'd love to have you on there. Um \>\> Dario will would be cool. \>\> Dario, if you're there, we'd love to we'd love to chat um and uh hear how you're using this model. Um and maybe hear what I wonder what the pope thinks. I'm sure the pope was using this model uh to write uh to write his um end cycle. What it's called? End cycle, right? Encyclical \>\> encyclical. It's an encyclical. \>\> Um monitos. \>\> Sam says he's better experienced through the medium of text. Um Sam, that makes sense. Uh I experience you. I think you're amazing through every medium, but if there's someone else who is more video forward, please let us know. DM me on Twitter. We'll get we'll get them in the stream. Um okay, let's keep going. I want to talk through the coding performance. Um, so the headline is it's a powerhouse at at coding, especially at extra high reasoning. One thing that you should know is that we found this on on every um task that we did across coding and writing and all that kind of stuff, we did find like real differences at at different reasoning levels. And um so on our senior engineer benchmark which it might be a little hard to see but on our senior engineer engineer benchmark which tests the model on which tests models on senior engineer type task which uh which in this benchmark means we give it a vibecoded slap codebase that is a real production app and then we ask the model can you fix this from first principles and uh and see how it does and Opus 4.8 scored a 63 out of 100, which is about 30 points better than Opus 4.7, which is really great. However, it only did that on extra high reasoning. On on high reasoning, I think it scored in the 30s or 40s. So, like a little bit better than Opus 4.7, but it really came out with Opus 4.8. And I think the the one of the differences is uh sorry, it really came out with extra high reasoning. And I think one of the differences is extra high reasoning seems to give it a little bit more confidence. Like the um the the failure mode on high reasoning for this task was it would sort of just like I would like we ask for a total rewrite and instead it would be like oh that's too much effort. It would take many weeks to do that. So I'm just going to like write some I'm gonna like paper over the issue and just write some patches basically. And 4.8 was like no no I'm just going to go do it. um it beat uh 5.5 by one point. So really there it's they're sort of neck and neck. There's there's good and bad parts to there. There are pluses and minuses for for each one in in this in this particular benchmark. Um 5.5 seemed to be a little bit more willing to just like slash and burn and and delete a bunch of code and um truly start from scratch. Uh 4.8 Opus 4.8 was a little bit more conservative, a little bit more like I'm going to like I'm definitely going to redo this, but I'm going to make a transition plan. What one thing that I found is it felt fairly honest about what it had done and what it had not done, which is always a problem with on like really long running tasks where it kind of forgets at the at the end like what what was I supposed to do and did I finish and all that kind of stuff. It's it's pretty good at that. Um so very very very very exciting results on on the senior engineer benchmark especially because it also um it also got better on a lot of other axes like a lot of what we see usually with with major model releases from labs is you get better at one thing but you kind of get worse at another and and they're always kind of like flip-flopping between different poles of an axis like is it better at writing or is it better at coding and if you make it better at coding it just gets a little bit more autistic so it's like not as good at writing anymore Or is it uh uh did you make it more agentic? Yes, but like now it's now it's kind of too agentic. So it's going off and doing things you didn't ask for. So now they pull it back and now it's too cautious. Like they're you're kind of flip-flopping. For whatever reason, when we all tried this, we were like, "No, no, this is they they like hit some magic thing here that just feels really good to use." Kieran, do you want to tell us about your results on the benchmarks and maybe show us some stuff? \>\> Yeah, there was a good question actually that's related to this. Uh Christian is asking with 4.7 we saw that it reads prompts more literally and we needed to change our prompts a lot. Does Opus 4.8 read more between the lines as 4.6 did? It's kind of related to that like do what did you feel there \>\> that if it uh does it read between the lines more? \>\> Yeah. So so like what what I always like about anthropic is that you you don't need to be too specific. uh like it reads between the lines and with 47 suddenly it stopped doing what you wanted it to do because it was just too GPTish like and so yes Christian like we're back this is the title of the thing that thing that annoying thing is gone it's just doing whatever we expect from models from anthropic to do uh we had no issue we didn't need to change our prompts in any way they just worked. So yeah, you you can loosen up the prompts a little bit again. Yeah. Um for Okay, so let's see for um yeah, what I what I found uh with mine I can also show things. Uh but what I found really is um I tested the LFG bench uh which is the compound engineering LFG flow with just one prompt I give it and it's basically running for 40 minutes and then something comes out. So it's really pushing like longunning task, factory style work, the future basically in this benchmark and whatever comes out I just score on all kinds of things. And like the the thing I learned is that it's very important to choose the right thinking level. And for me the best thinking level is extra high. Um, and high. High and extra high are the ones I mostly tested. Max is just a little bit over the top. You don't get a lot out of it, but the sweet spot was with extra high. Um, and the time running was not that much longer. It was like maybe 35 minutes for high and then 40 minutes for extra high and some. So it's like a little bit more, but that was actually the thinking level that pushed me to think, hey, this is a paradigm shift instead of it's as good as GPT 5.5. So I would say for coding tasks in autonomous running like harnesses or like however you you do it, I think high is as good as GP 5.5. Extra high does things I've never seen uh GP 5.5 do. And uh this is also on uh on high there. So this is kind of like \>\> things like that that that stood out to you that you thought were like really cool. You looked at it and you're like, "Oh, wow." \>\> Yes. Uh yeah, you can see this, right? \>\> Yeah, we can. Mhm. \>\> So yeah, so this is a this is uh like a breath work uh simulator. Um, and one of these things I've never seen is like, so you can go through different uh different types. And really what I said, hey, I want a breath work uh app that uh just uses some methods and these are the methods I want to have in there. And one of the things was like if you go to Vimhof, shout outs to the duchies and the ice people. Uh, if you click begin, it says, "Hey, before you begin, this is an intense breathing practice. A few things to keep you safe." and it says, "I understand." But if you go to coherence, it just starts. And that's something that I think is special because it did something that is breaking out of a frame specific to this thing, contextual for that thing while also working on the whole thing in one shot. So I think that's an example where um yeah where where high works really well and if you see the extra high version here um it doesn't do that. So you can see like the the depth there. Um, and yeah, you can you can see it it really does something extra that is very human or like this is something that someone like a like a PM would say, oh, actually we should add this or like we should have this because it's a little bit different context. Um, and for AI, it's very easy to just implement it on like a service level and this feels very deep. So I I think that's that's an example of this. Um and like how I do this is like the the the GPD5 is like a very eager, brilliant 25 year old. like they can do anything and \[laughter\] they're very like they they're super smart and open 47 uh just writes too much fluff writes too much uh and is a little bit too professional boring uh and 4.8 Eight feels like someone that's seen the world that just knows things but doesn't say it but just it's present kind of like presence kind of like you understand okay it's seen it it gets it what without having to show off or like showing the plans and how that is is like writing plans uh plan writing in Opus 4.8 eight is like half the length, which is great. Uh, so it's not as lengthy as 47, but also it's more about the meta layer than GP 5.5. And I think that that's kind of the difference here. \>\> Yeah, that's that's really interesting. A couple things that are striking me that that you're saying. One is the eager brilliant 25-year-old is what we used to talk about like a sonnet and and early opus, you know? it was like, "Oh my god, I'm going to go do something. It is amazing." \>\> And and that's it's really interesting that you feel that way about uh GVD 5.5. And yeah, for I feel like they they went in the opposite. They got more codeexy. Codeex got a little bit more claudy. They sort of switched places, but they it feels like they they hit some they hit a sweet spot here with this model. And uh and I I think the meta layer part of it is is really part of it. There's a there's a questioning the frame a little bit but in a gentle way that's like I just I want to like insert a little bit more like a little wink to like maybe there's another way to do this in case that's helpful but like we'll do it we'll do what you want which I think it's really hard to achieve and and they did a really good job about good job with Dominic asked what about UI design Kieran did you I mean we you did some on your LFG bench there is some UI design stuff so tell us take us through UI UI design. \>\> Yeah I I like it. So I I think so my favorite UI model is Gemini mostly. Uh and I did not like 47 for design. It was just too fiddly and like they tried something but like it it wasn't for me. I think they toned it down and it's actually pretty good. I think it's either as good or better than Gemini, which is great because if one model can do everything \>\> because you've been I mean you've been using Gemini a lot for and this is specifically for translating you've been using it specifically for translating Figma to code, right? \>\> Yes. Yeah. Yeah. So, I've been using it for translating Figma to codes. I've been using it for iterating and just saying make it better. Um, but I do think um I think 4.8 is really good at design and I would say as good maybe and it just feels like it's less fiddly, more minimal but also with some character. It's actually it is actually pretty good. So this is one uh this is an other this one is not there. Oh yeah, this is this is actually a a store. This is the most complicated one is the rubber duck store where you can design your own ducks. Uh so it has a designer here and I want a glossy. I want some spectacles. You can see it's not perfect \>\> but it looks good. I mean this whole thing looks good. \>\> Yeah, it looks like it could be an actual website. Like I if I look at this, my AI sensor is not getting like it's not going off other than \>\> like yeah it's a duck and it should probably be more 3D or anything like that but like that's very hard. \>\> Um \>\> what would make you think and I ask this what would make you think that it had like AI smell? I know what that means for writing but like what does it mean for code or design? \>\> Um a little bit too fiddly. Like I can I can do a side by side with five five and some older models uh later as well maybe as a as a fun uh just I I have to set those up. But like it's a little bit more fiddly. Uh \>\> yeah, purple purple accents. Uh Patrick says, \>\> "Yeah, but I I think purple accents are already old AI things for design because um I I think now the AI thing is that there's too much going on. there's just lots of visual stuff that like looks like eye candy but isn't adding any value. I think that's kind of what I think now about AI stuff. And yeah, like the gradients and the the stuff like that in the past, but that's really like an opus for five or like a four zero thing. That's not there anymore and Gemini doesn't do that either. Um, I think it's just making sure whatever is there makes sense in design. And I feel it's the same with writing. Like 47 gave you like lots of text and 48 gives you less text, but it feels the same or better, which I think is a little bit similar in the design uh side. \>\> Cool. Um, let's keep going. Kieran, anything anything else you want to talk about for our first pass through coding? um it's is very good at orchestration as well still which I think is maybe the most important thing for any model uh right now or at least there should be an orchestrator model and uh yeah it's good to note that it's very good at that as well. \>\> Awesome. So we're going to get to writing in one second but before we get there if you're here welcome. This is our vibe check of Opus 4.8. It's a beast model uh where we are every I'm Dan Shipper. I'm the co-founder and CEO. We're joined by Kieran Classen who's the GM of Kora and Katie Parish who's a staff writer. Every is the only subscription you need to stay at the edge of AI. You can think of us a little bit like a frontier AI lab focused on the future of work. Uh we do three things. We test models and write about them. So today we have a really long hands-on vibe check of Opus 4.8. We've been testing it for about a week now. Uh but we do tons of other stuff. Every day we have something new about how we're using these models in our work and in our lives. We also build products. So we have a suite of products. Kieran builds Kora which is an an AI agent for your email. Uh but we have like we have many others. We have one called Monologue which is a speechtoext app. We've got Spiral which is a writing agent. So we build a lot of products with AI. And then we also do a lot of training. So, and we bundle it all together. So, you pay one price and you get access to everything that we make. All the writing that we do. So, you know, we've got these vibe checks on new models. We just published a really great guide to codeex yesterday or two days ago that that Katie did is absolute banger. I love codecs. She loves codecs. Um uh uh so so we do a lot of publishing like that. We we build these tools for ourselves that we that we give to our subscribers and we do a lot of training. We do live streams like this. We do courses. We do camps. So if you want to be at the edge, there is there are there's really two things you need to do. One thing is subscribe to every every.0/subscribe. The other is stay hydrated. Um it's really important to keep yourself and your agent liquid cooled. Am I frozen? \>\> Great. \>\> You're drinking water. \>\> Drinking water. So with that, \[clears throat\] I would like to get into another very important part of our program, which is to talk to you about the writing capabilities of this model. And luckily, we have our very own Katie Parrot, writing tester extraordinaire, writing tester, vibecoder extraordinaire. Um, maybe Katie, can you share the the vibe check so folks can see it and then uh maybe talk through some of what you found as you were testing this model. \>\> Oh, let me find the unmute button. \>\> There it is. \>\> Here it is. So, we have previously on Vibe check, we were looking at the coding section. So, let's scroll on down here. Passed all of our beautiful our reach test, our coding write up, which was a joy to get to write up for me. Um, I'm very happy to have had um to have had uh GPT or not GPT, Opus 48 as my writing partner on this vibe check. Um, that's something that I should highlight. I always when when laying down the first draft of these um these um reviews, we always try to use the model in question. Um and the kind of headline for this release is that we actually uh feeling inspired to try things that I wouldn't have tried otherwise, I decided to take a stab at building our first writing benchmark. So this is very much work in progress. It's not something that like we'll continue to evolve it as we learn more about what exactly we want to know. But for this for this run through, I kind of went with a a suite of tasks that I know I use writing for uh or AI writing to write to do a first draft on other other people on our on our writing team, editorial team um do the same. And so that's writing an intro from scratch, filling in text that's left out of the middle, like if you're like you don't know quite what you want to say in the middle of a section, it can help bridge that gap for you. writing a promotional email, writing a tweet using the um the algorithm that uh the X team actually ex like externalized on GitHub for the for you page. Um we sort of ran it through all of these things and what we found is that opus 4.8 at high uh scores the highest across all of these tasks um at a 70 what does that say 79.6 six compared to uh 73 for GPT55. But the interesting thing that we found is that second place actually goes to the Sonnet model. So this does make me wonder if maybe I'd been sleeping on I had been sleeping on Sonnet a little bit for writing. Um but this just gets down to something with my personal behavior with models is I like to just settle down in one model and even one effort level if I can and just charge ahead with that. So for this test, for this for GG, for why do I keep doing that? Opus4, for example, I settled right in at high for writing myself personally, that's what we did for this um benchmark. Um I'm actually in the process of running it on on extra high just now because uh I just didn't do that test because I was so, you know, I was so impressed with what it did on high. So that's something that we will make sure that we improve for the next run of this benchmark. Um, but of course, you know, writing is really subjective. And so, something I want to highlight is that I actually preferred GPT55's take on writing the intro. So, what you'll see here, the t the ask is to write an intro um for a piece. It's actually modeled on Dan Shipper's recent opus, if you will, after automation. Uh, this is the model's attempt to write an intro to that piece. Um, and so you'll see just looking at the first line, this is what Opus4 came up with. I run a company of about 30 people and we have spent the last two years automating ourselves as hard as we possibly can. And that's, you know, that's pretty good. That's pretty close to what Dan put in the beginning in the actual opener if I if I remember correctly. But I really love this line from GPT55. The strangest thing about automating a company is how much work it creates. Like to me that's just like a little bit more interesting of a first line um than what Opus48 came up with. But overall like just as like like especially like drafting, you know, in my experience like you're never just drafting this much of a post, right? You're carrying it through an entire draft or at least you are if you're me. and just across the the full flow of the right of writing with uh Opus 48, I just found that it reliably delivered uh content that I was like, "Yeah, there's some things to fiddle with here. There's some things that I want to rough up to make it mine, to use the timid scribe analogy that I love to use, you know, that AI can just be a little too predictable." Um, that's still here. But the pros is just like it's a good clean copy to lay down and then go in and make it your own. Um, one thing I do want to caveat, we've got to talk about AI smell. You know, we have to um if you like one of the things that the benchmark scores is how many AI tells were in the copy across all of the eight different tasks. Um, Opus 48 came in at, I believe, 16 tells. I can't remember. Is it up here? It's not. It's It's somewhere in here, but I'm not going to Oh, here it is. Um, yeah. So, Opus 48 left 13 tells across eight tasks below Opus 47's 25. So, it's a big improvement over Opus to Opus. Um, GPT55 had 21 tells. It really really loves to tell you something matters. It really loves to tell you not X but Y. And speaking of not X but Y, that is the one thing that I found that Opus 48 just can't quite let go of. Um is that impulse to set up what the thing is not before it goes on to tell you what the thing is. So I'm fascinated by why that's a thing. You know, we know a AI models are trained on writing humans have done. So, it must be something about humans where we just need to tell you what we don't mean before we tell you what we do. And then the last thing I'll cover on writing and then I'll pause is just that it's really good at picking up style from context. Um, I have been near evangelical about the value of having a style guide for writing with AI. This is something I discovered probably a year to a year and a half ago. If that if you just hooked up a document with some basic rules and preferences to the model that you're working with, whether in the chat interface in the project files or in a pro a file in your folder if you're working on cloud the way I do now. Um once it has access to that style guide, um it really does a really good job of picking up style. So, um, I actually for this particular run, because I built the MCP into my Tastemaker app, uh, I went ahead and pulled styles directly from there. So, my personal style tends to be a little bit more flowery. It tends to be a little bit more literary. Um, when I'm writing for my newsletter, the curiosity gap, or it's really mostly based on writing that I admire, that I aspire to. Um, and so we've got some, you know, we've got, you know, we're talking about Leonardo da Vinci and apprenticeship and the importance of the entry- level job across both of these pieces, these um, versions of the copy, but the personal one picked up that much more literary style where the working overtime one is more conversational. \>\> I love it. Um, yeah, I mean I I I I used this model a lot actually for I think we got it after the after automation piece came out, but there's a lot of stuff that I had written for that piece that I needed to then like change or did we get when when did we get it? We got it on Friday, right? \>\> Last Friday. \[snorts\] \>\> Friday afternoon. \>\> What is time? I don't even know. I don't even know what time is anymore. But anyway, I use it a lot for writing and thinking, and it's got like a I think of GPT 5.5 as being very straightforward and clear, but that means it can sometimes lack depth and richness, which I think comes through in its writing for actual writing tasks and in its plans. Like the GPD 5.5 plans are are pretty sparse. And \>\> with Yeah, with Opus 4.8, eight. It it it has some depth and a richness, like a multi-dimensional richness that that I think GDP 5.5 lacks. And I think that's actually really helpful in a lot of writing tasks. Um because it it manages to do that without going off the rails and just like being way too flowery and like bad. And it's also really good for coding because I I I think the plans are just yeah, they're just more detailed, which is really important. Um, so, uh, so very very good writer. So, I'm going to now I think I figured out how to how to share my screen. Um, and I'm going, what I want to do is share with you all. Um, one thing that we do is we always test these models on knowledge work tasks, too. And I want to show you something that this model made in one shot because it's like really good. Um, so one of one of the tasks that we always test on is how does it how well does it do making a deck? And this is a oneshot deck. Um, it's for compound engineering, which is our engineering philosophy championed and created by Kieran. Um, and this is a deck made in one shot by Opus 4.8. So it's got compound engineering, the AI native way to build software or every unit of work makes the next one easier. Like that's actually that subhead is really good. It's it's exact it's probably exactly what I would have written. It does you know the problem with software today what compounded is the force. It starts with the problem. It defines the solution. It goes through the loop. Like most code bases get harder to work with over time. Each feature you add injects more complexity. After 10 years teams spend more time fighting their system than building on it because each new feature is a negotiation with old ones. com engineering flips this on its head. Like this is good. It's like a good story. Um that I looked at this and I was like thank god I just wrote a long essay about why even as model capabilities improve it it makes more work for human experts. Uh which which we'll talk about in a second. Um, but I think usually when when when models make decks, they it like it it is a deck, but then there's like stuff that's out of place or if you look at the actual writing, it's like not that good and it doesn't really tell a story. This has the this combination of the visuals are really are really good, solid. Like there's there's nothing obviously wrong here. You know, some of the contrast stuff I would probably change here, but I I think I could give I could plausibly present this deck to someone and not feel totally embarrassed. And the writing and the way that it lays out the ideas is also very good. Like I I might struggle to do a better version of this on on like with a couple hours of of work than than this thing is. There's some spacing problems here, you I'm sure if we really like there's some more spacing problems here. So, if you really get into the details, there's still stuff to do, but it's it's incredibly impressive that it's able to do this. And I think this gives um this should give you a sense for the range of stuff that it's able to do because deck making is it it it requires a bunch of different skills, right? There's like um there is uh writing, there's visual design, there's the actual like putting it together into a deck. So there's just a lot of stuff going on that I think is that I think is really impressive. \>\> Um if you are \>\> if you do this in in what app did you do this? Was this the \>\> this was uh this is Mike Taylor. So Mike Taylor is our um he leads our the tech side of our consulting practice and uh this is his his uh consulting benchmark. So, it's just a script that I ran. Um, I don't know. I honestly don't know the harness that it runs in, but it obviously did a great job. \>\> Writeups on all of our benchmarks coming to every soon. \>\> Yeah, we actually we really need this because I think we've we've started to like move into a zone where we're like we have benchmarks for everything, but then it's all just like this total mess. So, someone's going to have to come in and clean it up because I know that it's not going to be me. Um, \[laughter\] \>\> might be four. I think the three of us like to make the mess and we need to find someone else. \>\> Yeah, we needed someone \>\> going to clean it up. \>\> Too many pirates. \[laughter\] \>\> So, if you are here, welcome. This is our vibe check of Opus 4.8. We've been testing it for a while now, for about a week. It's a great model. Um, we uh we are every we are the only subscription you need to stay at the edge of AI. You can think of us a little bit like uh a frontier AI lab for the future of work. Um we publish vibe checks like this all the time. So our vibe check is now live on our site if you want a really in-depth explainer of all of the testing that we've done from coding to writing to uh to knowledge work tasks. Like it's all on the website right now. We do these on the day models come out with empropic models with with GBT models with lots of other stuff. We actually also just published something yesterday, not yesterday, last week that I'm quite proud of called after automation which um is a really in-depth long report on how we use agents internally because our you know our raise on detra uh I don't know if that's probably that's like the bad bad French accent there but our our reason for being is we love to play around with this stuff and then figure out how does it get incorporated into our work and our lives as an organization and we have about 30 people so we get to like how do these models get get incorporated into our work? And so uh after automation is about this fundamentally interesting thing that we've seen which is that even though we've we're automating everything that we we see we uh actually hire we're hiring a lot more humans. Like why is that? That seems like a paradox. Um does it apply to other companies? Does it not? If so, how? It goes through how we use agents internally at every um and then it goes through uh a core argument for why actually automation with today's AI and and AI going to the future actually creates more work for humans. So check it out every.pa- um but today we are here to talk about opus 4.8. This is the vibe check right here. It's at every.048 vibe check. It's a too many vibe checks. Uh we'll we'll have to change that. Uh I wrote it with Katie. Katie is our incredible is incredible staff writer. Um and the the headline is anthropic is so back. We think this model is really good. We think it's we think it's really really good. It's good enough that they bumped it from 4.7 to 4.8, but I think they could have rounded it up to five and we would have not been surprised or baded an eye. Um incredibly good at coding. Uh, it beats GD5.5 on our best on our hardest benchmark. It's really good at writing. It's it's manages to be very clear and expressive, but it's also very good at matching style. Um, and it it's just really good at general knowledge work tasks. We were just talking through um how it how it did on uh uh the a PowerPoint presentation creation benchmark, like a oneshot PowerPoint. It's like fantastic. Um, so that's where we are. We've gone through the vi we've gone through the reach test already. So the reach test is that's our our main way of just overall telling how good is this model like and that's do we reach for it every day like just automatically and if so for what um I'm a gold and a green so uh gold is paradigm shift green is really good model. Um, I I would say I am gold in the sense that this is it's just really it's great. It it's it's just a different kind of thing than other models that I've tried and I think it it opens up way more use cases and and new things. Um, and I'm agreeing because the harness just isn't that good. I love Codeex. Codex is an incredible an incredible harness. It's very smooth. It's very fast. It's like it I think it skips all the history of how we got here. Whereas the the desktop the cloud desktop app is like it's got it's got chat. It's got code, it's got co-work, and I'm like, where do I where am I going? Like, I don't I don't know which task this really is. And codeex, it just works. And I it is now my operating system for knowledge work. Um, so I I I give this model a a gold green only because I don't think the harness gets the most out of the model. Hopefully, they will change that. Kieran is a gold. Um, Kieran and And I got to tell you, these are rare spores. I I I feel like we've only given out golds like maybe once or twice in the last year or two. It's like every six months something happens. You're like, "Ah, this is sick." Um, and I think probably Augustus 45 was the was was in the gold territory and that was in November, December. Um, so rare event. You should check it out. Katie's a green. Uh, really really good for writing. So we're we're definitely reaching for this model all the time. I will say CODC's still my daily driver. Also, one thing that I want to say while we're while I have you, um, is I think that there is a whenever a model comes out like this, there is this feeling of FOMO that you get, especially if you've committed to an ecosystem where you're like, a like I just subscribed to Codeex, like now I got to go subscribe to Claude to have the latest. And I I I have to say we love new models, right? We we love testing it. that's part of our job. Um, we love playing around with all this stuff. And so, if you're one of those people and you just love playing with stuff, like you should totally use this model. It's amazing. And also, if you're like, I just need to get my job done. It's worth paying attention to this. It's worth maybe trying, but also like just just you can you can just chill. Like GP 5.5 is very good, too. Um, and I I I want to like just take a second to like have everyone just settle out. settle out a little bit of the FOMO and and realize that these tools are incredible. This all this new stuff is incredible. And if you're worried about switching back and forth too much and all that kind of stuff, like don't worry about it. It's fine. Um uh but you should try it if you if you want to play around. So, uh it wasn't it wasn't a 4.5 moment where it was like, "Okay, everyone, whatever you're using, this is the thing you need to try. This is the absolute best." Now, it's \>\> it's a little bit better. and I like it more. And some people probably prefer 5.5 because that's how that's how they work or that's how they know how to work as well. So \>\> I do prefer it because of the harness. And I think that's a that's \>\> like if it if if 4.6 was true AGI, it wouldn't matter the harness, you know. So um uh Okay. So Kieran, I know you're I know you have some stuff you want to show us uh on the design side. Do you want to take it away? \>\> Sure. Yeah. Yeah. So people ask how good is it at design? Let me just show you the different models and how I read the AISMs in design. Um I will show 46, 47, GP 5.5 and Opus 4.8 if you can share the screen and Gemini 3.1 Pro. Cool. So this is this is part of the LFG bench and basically it's uh I say hey compound engineering plugin I uh have this website called proof can you redesign it and it's a very minimal prompt. Uh so uh the whole point is like how much can the model do? This is opus 4.6 six. And what are I looking at for design? Is is there anything that's weird and anything that stands out? And then yeah, does it look good? Stuff like that as well. You can see the color here is still a little bit like weird. And this thing here on the right is also strange. Um, also this animations everywhere, drop shadows. So I'm like, yeah, like this looks like AI in in my head. This is Opus 4.6. Let's go to Gemini 3.1 Pro. Boom. Well, looks better in my opinion. Like, it's more of a statement. It's more like design. And there is some animation, but it was done I think pretty okay. There. I mean, I don't love the animations. It's a little bit AI, but this looks like this looks way more. This looks better. And there's nothing where I can see, oh, this this is broken or this is not not good. Um, let's see who was next. GPD 5. Uh, no, Opus 4.7. Let's do GP 5.5 next. Okay, GPD 5.5. Um, I mean, this is what I mean by 5.5. It's a little bit like yeah like there is stuff they they just like it's like filling it up. Also the position is a little bit weird but like it looks a little bit busy. Um it's not bad. It's like structured but like it feels like there's it's not super balanced. It's a little bit fiddly but it's not bad. \>\> Is also less imaginative. um often shared documents like that's \>\> that sounds like business speak to me as opposed to something that would get me excited to try a product. \>\> Exactly. Yeah, that Yeah, absolutely. I I'm just looking at design, but yeah, like the copy for sure is also uh is a big difference. So, let's let's switch from GP 5.5 to Opus 47, which I thought was the more business model, like the the boring business model. Well, accounted it says in the time \[laughter\] every word accounted for. So, this is like it's interesting as a concept, but it's not executed well and it's well like it's it it looks also like it's there's some it looks a little bit off and and why it's because this is big. It's the colors are a little bit better. I think it's not the like the the standard colors we had. There's no I mean the background color is a little bit uh uh like that. But let's go to 4.8. Uh 4.8. Poof. Um so every word has an author. Now you can prove it. That's cool because it is called proof and it's not accounted for. It's like I think from all the designs I think this this is not as screamy big in your face as the GP5. It's a little bit more balanced, more minimal. I would say if I look through this, this looks the least like AI. There's no like these scroll like the there is the hover the the the background shadow but there's not the animation that's in everything the the like that appears. So they're really like I I think it's a nice balance of design working together. I think this and Gemini so let's go back to Gemini as well are my favorites. Like I kind of like this as a like a mark but it's also not like it's not very it symbolizes Yeah, the headline sucks, but like it looks like it looks meaty. Uh, but yeah, like for me the winner here is 4.8. So, is 4.8 the best at design? Humans are, but \[laughter\] if you want the design, 4.8 is pretty good at extra high, I think. \>\> I love it. I love it. \>\> Yeah, \>\> go for it. \>\> No. and and like and look if you get the copy that's better in it as well. Like that's kind of handy similar to what you're presenting with the presentations. Uh like it's handy if everything works because it's just that's what you need in the end. You need something to work all together. You don't need just good design. You need everything. \>\> Totally. Um, I I wonder if we should do some like live stuff and Kieran uh and or Katie, I wonder if there are if there are things that we can show people that might be interesting um either building stuff or writing stuff. Uh, anyone anyone have things in mind? And if not, that's okay. But, uh, curious we could tee it up and I can keep talking while we're while while you guys are setting it up. Any any ideas? What what should we do? I can get I have an interview that I just ran on a piece that I'm just sub that I just filed. If you let me just pull the exact same questions that I used, we can throw up um what G what op gosh opus 48 does. Um so \>\> let's do it. I love that idea. I love that idea. \>\> Um so while you're while you're getting that set up and Kieran, do you have do you have anything you want to you want to show? I can I can quickly share the differences for the cozy island benchmark as well. It's \>\> Oh, that's a good idea. If you have that pulled up, I would actually love to see. I think that's a really good little little thing to show. \>\> Yeah, it's it's pulled up. So, if you share, so the cozy island uh is a 3D. So, I just have a prompt where like make a cozy island uh use 3JS and like that's that's it. And it's kind of fun because you can see spatial reasoning and like it we'll just go through some details. Uh you're looking at GP 5.5 and what you can see here there is an island with birds uh and a house uh with smoke with animations. The birds fly sideways which is interesting. Um but in general like it \>\> which is looks kind \[laughter\] Yeah, but it it like it's not bad. It doesn't look too bad. Uh but but just try to spot things. One other thing is the cozy island floating ecosystem like it has this like UI thing on the left which is I call this like prompt uh like leakage or something like that where like it has a prompt and it thinks oh I need to put that in the UI instead of like it understands we don't want the words or the title in the UI we just want want that. Um so this is 5.5. Let's go to Gemini. Okay, Gemini. Also interesting. There are cracks in the island. Uh it's a little bit different style. Like it's more heavily stylized. Uh there's also smoke. Also, the the birds fly sideways. Um so there are some faults here, but it's like it's kind of cool that models can do this. There are different trees. Let's go to the next one. Let's do 46. Opus 46 is here. Disco. Um, the birds are flying sideways. The There's some uh earth problems going on here with the water. Uh, but the house like that the roof is nicely on the house. There's some flowers here like little tulips or something like that which is cool. different kinds of trees. Go to 47. Okay, the birds fly straight. First time ever that they fly straight. Uh they they flap a little bit weird. Um there is some collision but lots of detail and and like it's like there's a dog. People on the chat are saying nothing is changing. So are you are you \>\> There we go. \>\> There we go. \>\> Oh, okay. Sorry, I did not share any of that. Uh shall I just redo the other two or \>\> I would say go quickly through the through the historic ones so people can see \>\> live streaming. \>\> Okay. Yeah. Um Okay. So we had 5.5 then I did Gemini. This is Gemini. The birds are weird. Uh but it's cool. Uh and then we have 46 which has some issues with the earth but there is some good detail. There are some flowers here. You can see some flowers and uh the house is kind of good because there's no floating uh floating roof. Then we go to 47, which you can see it now, right? Yeah. Yeah. Yeah. 47 is very detailed. There's a lighthouse, which is really cool. And the birds finally go straight, but they canoe in the air instead of fly. So that's still something \[laughter\] funny. I I like the rocks, the like the little bushes, the detail. So you can see 47 like went deeper and that is kind of tracking with like it rides longer, does more, but it's also makes it way less usable. Uh and then we have 48 which is number five. \>\> 48. Look at the birds. \>\> Wow. \>\> Super cool. \>\> I mean there's animation in the trees as you can see here. Um, the house doesn't float in the air. There's no smoke coming out, but there's light at the house. But especially the birds, they look like they look really cool. They're the first ones to fly as a real bird. So, uh, yeah, good job on this benchmark. \>\> And I think there's there's some interesting questions here. It's like, okay, if if I'm not going to make a cozy island, like, why should I care? And yeah, uh, and \>\> first of all, you should care because it's just cool. Um, but second, but second, \>\> it looks fun. I rather have this than boring \>\> Cuz the island like why do you have to have everything have a use? But \>\> yeah, \>\> um beyond that, one thing that we like to test like making something like this, it requires stitching together so many different skills. Um from coding to design to kind of like thinking about what what would be interesting or fun or what would best represent this idea and and it and when you saw that I think a lot of people in the chat and me too, I was like, "Wow, this actually looks it just feels better. the whole thing feels better. \>\> Yeah. You you see the difference. \>\> Exactly. And it doesn't show up necessarily in a benchmark number, but when when you see something like this for for a task like this and you can put it side by side and go, "Wow, that \>\> Oh, there's a fish. I saw a fish. There fish." \>\> See, like there's there's this depth to it. And I think that that that'll tell you something about um what you can expect from the model in other scenarios. So even if you're not making a cozy island, which you obviously should, um it you'll see this kind of this kind of behavior in performance when you're doing writing tasks or when you're doing coding tasks um in in the appropriate dimension. \>\> Yeah, for sure. And and there's so many things that can go wrong here. And it's cool to compare models like you can just see the difference. And you can see there's no leakage of like UI elements here. uh which is nice and and this is like a GP 5.5 thing like loves to put GDPR compliance boxes in designs because that's in the data for some reason but like like this emotional thing of like oh yeah like this doesn't feel vcoded necessarily or there is something here that's why we're so enthusiastic about this \>\> totally and I have to say I just got a push notification from the New York Times sayingthropic surpassed OpenAI as the world's most valuable AI startup with a valuation of 900 billion. So \>\> dang, \>\> I mean like it's kind of crazy. That's that's just like pretty cold to just to drop this model and your fundraising news in the New York Times uh on the same day. That is uh yeah, cold move there from Daddy Daario. Um, \>\> also to balance that off a little bit, what do we think? Like, have we tried the cloth code app? Uh, and \>\> a good question. Thanks for teeing me up on that. There are caveats here. There are caveats. And I actually I actually think Thank you for putting me up on my soap box, Kieran. Um, I actually think that there's a whole narrative right now where like Claude is like winning, but that's kind of because the entire investment community is way behind and and they are feeding off vibes that Anthropic and Claude have generated from Claude Code starting about a year ago. I it took it took OpenAI until about December to like really understand what was going on with cloud code and really change their strategy. And so Anthropic had this head start from probably May of 2025 through December. And I think that's where that's where all of the investor hype and all the like just generalized hype is. But that's because they're the market's kind of behind and people who are really paying attention and use this stuff every day. You you just saw in December OpenAI turn something on and they are just like relentless executors. And what's really interesting about it is with Codex, they had the misfortune of being behind, but that has turned into sort of like the Zen master story that has turned into a a benefit for them because with Anthropic, they have the cloud desktop app and you can kind of see the history of how uh how the model has progressed and how the harness has progressed in the way the desktop app works. Like it has three different tabs. The three different tabs do different things. They're sort of targeted for overlapping but slightly different use cases. There's the chat tab, the code tab, the co-work tab, and the desktop app is kind of a mess. Uh it's slow, it's confusing. And Codeex, they just saw what Anthropic was doing. They because they were behind, they could see, okay, this is where it's going and then they just built that and it's only that. And that's why I that's why I love codecs. And I think that's a it's a really important thing to temper some of this stuff with because the model is only part of the story. It's only like half the story. The harness is is the other half and the codeex harness is just so freaking good. And they've made some of these little design choices that um just make it really nice to use. So for example, the inapp browser of Codeex, you can um you can go on any website. So like I do all my email inside of the inapp browser and codeex and that's amazing. in cloud you can't because you can only uh go on local host and it's just stuff like that or here's another one that I love in codeex both cloud desktop and codeex have sub aents and and claude and codeex can both the orchestrator can communicate with the sub agent but the way that the codeex team designed the codeex desktop app is they made a messaging tool called send input that allows the uh the codeex orchestrator thread to message it sub aents, but it can also message other codeex threads. So, I've got like this router thread that's like routing emails and messages to like different codeex threads that are that are for different projects. Like I've got my writing thread and my coding thread and it just it can just send messages to each one to like do different types of work and it's like so cool. So, um I really hope that Anthropic uh takes another pass at the app because it's just yeah, it's just really bad and it it makes uh it it means that even though this model is great, I'm still just using codeex as my as my daily driver. Um I Yeah, \>\> I wouldn't say it's very bad. I would say it's very mediocre. And now the the bar has raised like \>\> yeah we were \>\> I was perfectly happy with the claude app desktop app until Dan nerd sniped me by saying that codeex was so good and then assigned me I got and then assigned me to write about it and I learned it but like I didn't I wouldn't have known that the claw desktop app wasn't good if it hadn't been for codeex. \>\> That's interesting. Yeah. I Well, I mean that that's interesting. You \>\> there are bugs. Yeah. \>\> You didn't feel the bugs and the slowness and the like I don't know which tab to do this in. \>\> Um maybe I'm just like a chaos muppet and I'm not disciplined enough to do things anyway. I just go, you know, I just do things. I don't really know why. So, \>\> I mean, I'm the same way. Like, you can't out chaos muppet at me. I mean, maybe you can, but we're like we're we're maybe neck and neck, you know? Uh uh interesting. I just found it was just like slowing my computer down and and I was just like that little piece of friction that's like um which tab does this go in? It just like it kills me. But yeah, I I I do think there's there is another element of when you use codeex you see you can see something that you didn't even know you needed which is a sign of a good product. \>\> Yeah. And yeah, I I agree. Like I use cursor and that's nice and speed speedy. And I was trying cloth code out and I just sent like 20 things that I thought were not as good as cursor and and codecs. Like I like it's it's just uh yeah, it's that and what one other thing that I don't think they win at is speed. I know there's a fast mode, but also fast mode is probably very expensive. GPT 5.5 is two to three times faster. And \>\> that's also to call out that speed is important. Yet, you can get it faster if you set the effort level to slower. And maybe we can like show a little bit of the difference of speed uh in a live uh live thing here where we do like low effort versus extra high. Uh so we get a feel for that as well. \>\> We could totally do that. Yeah, it it is a bit a bit slower. You can still sort of feel the GPU rich versus GPU poor dynamic of OpenAI versus versus anthropic. Um, and uh, yeah, Katie, do you wanna do you wanna go through \>\> maybe one one thing about the speed and then there like the cozy island uh, Gemini 3.1 Pro finished the entire task in 3 minutes, \[laughter\] \>\> which is fast. GPT 5.5 took 13 minutes. \>\> Wow. \>\> And Opus 4.8 extra high took 42 minutes. Wow. So, \>\> okay. \>\> It's better, but it's also slow. \>\> Totally. Totally. Katie, I know you're going to show us some writing stuff. Uh, do you have things queued up? \>\> Yes, I do. In the desktop app. \>\> Wow. Um, brave. Very brave. Very brave. \>\> Well, this is where I do my work now. That's the thing is like I used to be I used to live in the web app, you know? I like projects were a huge unlock for me just like in AI overall. The ability to like you know as I said I'm evangelical about style guides and the ability to hook up a style guide and examples of past work to the desktop app or not the desktop top app the web app. you know, I was perfectly happy until, you know, I really started coding more seriously, you know, orchestrating more seriously, and then I just wanted the stuff that was on my files on my desktop. And so I just moved everything that I had in Google Docs into MD files and now everything lives on my desktop. So I very much like But I'm not I'm not going in the terminal. Are you kidding me? Like I'm I'm not technical enough for that. I mean, I used to be in the terminal, but like I just like the desktop app as a as a non-technical person. So, that's my little preamble about why I'm on the desktop app. Now, let me talk at you while I try to figure out how to share my screen. Um, \>\> there's a uh screen share. I know. I know. You know, \>\> uh I just have to remember \>\> screen share button \>\> window. No, I want entire screen share. Is this the screen? Okay, wait. \>\> We got you. We got it. We got it. \>\> This This is Claude. You see Claude? Okay. So, this is something that I just teed up. I'll just read off the prompt because I don't know any other way to um to explain it. You're all about to get a a sneak peek at the next working overtime piece because that's \>\> Well, actually, can you just tell us like set up the scenario for us like what what led you to start doing this prompt? Like what are we looking at? \>\> Yeah. So, I want to write a piece about something that I call AI guard rails, which are basically just the set of skills that I have developed to sort of push back against both my and AI's worst instincts when it comes to AI writing. Um, I actually got, you know, some very helpful feedback from our editing team that I was letting things through that I shouldn't be and that just, you know, some of those were AI tells, some of them were my own ticks and weirdnesses. And you know, after going off and hiding in a corner and crying for about 15 minutes, I came back and sat down with the claw with the claw app, then said to Opus 47, actually, and said and just said, "Here's the list of the feedback that we've gotten. I'd like you to make a skill called guard rails that we can run to to stop it." And that's part of a suite of of of skills that I've built around the same idea of checking things and making sure things are as good as I want them to be. uh and then some skills that push things to be even better. So basically I want to write a piece about that um that process and so actually now I don't need to read my prompt because most of that's reflected here. But the important thing to note and this is something that's you know this is you know this is very much how I write and like my lived experience as a writer at every um because I'm writing about these skills that live on my desktop um I can just point the AI model at the repo at the folders where these things live I'll just tell it to go find it like I don't even have to tell it anymore right like this is something that's been true \>\> I love that right you're just like oh yeah figure it out and it does \>\> yeah so I was like go find my skills so you know what they say and what they do and like then I don't have to explain it to you. You know, my usually when I start a draft, I'll start by doing an interview process, which I wouldn't necessarily do live because it's just takes a while and um with an Opus uh 48 isn't necessarily the model I do it with live because, you know, it's faster than Opus 47 by a huge margin, but you know, quick back and forth is still like not what you want it for. But you can see here it found the skills to make sure it described them accurately. It says, "Katie's local skills are the centerpiece. Let me read them directly." And then it's just, you know, I just love reading like watching the model like find its way through stuff and make decisions. And it exposes a lot of like of its reasoning and rationale here. So, you see it found my compound writing skills. Um, which uh I haven't been calling it compound writing publicly so much, but because I'm still working out how to get it to compound reliably, but that's what this is. This is me trying to this is me having forked Kieran's repo and you know trying to build a version for for for writing and then so it found um it found all the skills um and and um it actually another funny thing is that it found a previous draft that I actually done with codeex. So I had to tell it to ignore that and do its own version here. But that's that's another reality of what my workflow looks like. Like I actually real like I realized that I had this like both Codex and Claude looking at the same draft and they were like being like there's a new file. Here's what I think about it. And that was fun to watch. Um so let's just see what And then I just had it draft. Um and here's what we got. Um let me pull this open a little bit. I built an editorial board that lives in my terminal. Um that's Yeah, man. Like usually I have our AI editorial intern um handle the heads and decks at this point because she has access to what's performed well for us in the past. Um and I just said like give her the draft and say come up with heads and decks that you know align with what we've seen be be successful. Um but so there were good um the embarrassment of riches in my writing stack that actually comes from my prompt. I have committed to putting the skills out there that I'm talking about here at least some of them. And I said in the prompt, I want to inundate people with an embarrassment of riches. So it picked up that language directly. Um, and so yeah, we've got some some good decks, too. The skills I run at every stage of the writing process to keep AI assisted work rigorous, weird, and mine. And a repo so you can build your own. Think that I might take that for the actual draft. And then we get into the copy. And I'll just read a little bit because that's the only way to do this. But it says, "There's a specific kind of professional embarrassment that arrives quietly in a shared dock before a one-on-one. You open the dock to prep and there is a list. The list is about you and the list is I came from Kate editor and chief. The note was small and surgical. My dress had started picking AI not was wrong. Exactly. The sentence were symmetrical in that suspicious way. The transitions were too smooth. Pros and frictionless quality frictionless quality of paragraph rather than argue for like very competent assistant impression of me. The worst part is that I write about the I write about this failure mode. I know neuroscience the speed fluency feel like the quality like quality what is not way are it turns out different that's I don't know that's the writing thoughts \>\> oh I love it I love it one thing that you said that really resonated with me is I truly truly love reading this model's thoughts Um, like it just feels different because you can feel it kind of going through all these different little nooks and crannies of what it might say and how it might say it and why it might say it that way. \>\> And often that itself beyond just the finished answer is what I want to see because it help it helps push my brain in a lot in a lot of different um a lot of different uh ways. And then I can also say like I know he I know I just I just anthropomorphized Claude. I know uh the I know the model is sort of worried about XYZ thing. Like I'm just going to push it in that direction because it it's worrying unnecessarily or whatever. So the way that it writes is it's compelling enough that I want to I want to read how it thinks. \>\> Totally. \>\> All right everybody. So this has been a pleasure. This has been our live vibe check of Opus 4.8. Um, we do these vibe checks every time a new model drops. We get it early. So, we put our hands on it. We use it for coding. We use it for writing. We use it for knowledge work tasks. And the day it drops, we have a vibe check. The vibe check is it happens live. We've also got a recording on YouTube or video that we did in advance. We've also got a long long vibe check uh co-written by Katie and uh and I and me. Is it me or I? I I think it's just I think it's uh me. Um co-written by Katie and me. And uh we are we love model release days. We love doing this for you all. Um if you're not a subscriber to every, you should subscribe. Every is the only subscription you need to stay at the edge of AI. You can think of us a little bit like a uh frontier AI lab for the future of work. We test these models. We uh we write about them. We build software using them. Uh and we do a lot of training and consulting. It's all available for one subscription. It's on the every website every.subscribe. Um and we will have much more of these. There's a there's a lot. There's a lot coming. There's a lot coming, so stay tuned and make sure that you stay hydrated. \>\> Cheers. \>\> Cheers, everyone. Have a \>\> great day, everybody. Bye.