A live first look at Anthropic's Claude Sonnet 4.6, covering its claimed upgrades across coding, computer use, long-context reasoning, and agent planning, a 1M token context window (beta), and Opus-level performance at Sonnet pricing, as the hosts test the model live. # Channel Presentation Observations ## Video Identity - **Format:** Live-streamed, conversational "vibe check" featuring two hosts (the main host and **Kieran**) testing a newly released AI model (Claude Sonnet 4.6) in real time. - **Stated context:** A recurring show format — *"We were here last week"* and *"Someone said last week that we should make it a daily show."* ## Explicit Self-Description Claims - **Channel purpose / frequency:** *"We do them [tests] as soon as a new model comes out. Generally, we get early access to it."* - **Subscription pitch:** *"Make sure you subscribe to every.TO. Every is the only subscription you need to stay in the age of AI."* — indicating the channel is **"Every"** (also referenced as "Every Group," which *"manage[s] four different products"*). - **Methodology claim:** *"There is no magic in our tests. It's just us who do this often and who really form an opinion"* — informal, experience-based evaluation rather than benchmarks. - **Testing philosophy:** *"Here are some tasks I do on a daily basis. Like, how does he deal with that?"* — practical, workflow-integrated testing. - **Sign-off habit:** The recurring *"don't forget friends, stay hydrated"* closing, echoed mid-stream with a hydration reminder bit. ## Intended Audience - **AI developers and builders:** Discussion of pricing per million tokens, production use cases — *"if you're developing applications where you want to integrate an agent... Sonnet now makes possible."* - **Heavy Claude/Opus users:** References to subscription tiers (*$20/month*, *$200 plan*), Claude Code, sub-agents, MCPs — assumes existing Anthropic ecosystem familiarity. - **AI-curious general followers:** Hosts repeatedly re-explain for newcomers — *"if you're new here, Anthropic has just launched Sonnet 46..."* - **Interactive live audience:** Viewers are directly addressed and answer questions (Nicholas, Randy, Dar); audience is invited to edit a live shared document: *"feel free to go here if you want to get into the document."* ## Claimed / Demonstrated Scope - **Scope demonstrated:** Live model evaluations, coding workflow demos (pull request triage, compound engineering plugin work), personal product demos — Proof (markdown editor at proofeditor.ai), Kora (AI email assistant), Spiral (ghostwriter agent). - **Claims insider access, when available:** *"Normally, we receive these models in advance"* — but notes this launch had *"no early access program."* ## Presentation Patterns (Observable) - **Casual, unpolished style:** Real troubleshooting on stream (broken MCP installs, screen-share mishaps, *"we're going to crash the server"*). - **Verdict framing:** Ends with a rating scale — *"Red, yellow, green, or a paradigm shift?"* — both hosts answer "yellow." - **Dual-purpose content:** Blends genuine work (*"we're just trying to move forward with our work"*) with audience entertainment and product exposure for their own tools. # Delivery Observation: Live Vibe Check — Claude Sonnet 4.6 Test Stream ## Format Overview - **Two-host live stream** (primary host plus "Kieran"), informal, conversational, testing a model release in real time with screen sharing. - **Recurring audience onboarding**: the host periodically re-explains the premise for new viewers: > "So, if you've just joined us, what you need to know is that Sonnet 46 has just been released." > "Again, if you're new here, Anthropic has just launched Sonnet 46, which they're sort of presenting as Opus, but at Sonnet prices." ## Tone & Register - **Casual, unpolished, stream-of-consciousness**: heavy filler ("uh," "um," "like," "yeah"), self-corrections mid-sentence: > "Uh, I mean, how does it compare? I think it compares to, as they say, Opus 46..." - **Spontaneous humor and asides**, including a recurring hydration bit: > "It is very important to stay hydrated to stay at the forefront of AI." > Closing line: "don't forget friends, stay hydrated." - Playful reactions to chaos: > "No no no no no. He...he does whatever he wants." ## Teaching / Testing Habits - **Live, unscripted testing with narrated reasoning** — the host explains *why* each test matters: > "The reason I like it is that it's a kind of creative problem-solving question." - **Standardized prompt reuse**: "the extraterrestrial civilization that never invented the number two" is presented as a recurring benchmark: > "Imagine this is one of my tests for the new cutting-edge models." - **Comparison against prior models** as the core evaluation frame: > "What we're trying to determine is: is he really as intelligent as Opus?" - **Explicit honesty about methodology limits**: > "There is no, there is no magic in our tests. It's just us who do this often and who really form an opinion." > "But I really don't know. I'm just guessing that's why it's happening." ## Recurring Phrases & Verbal Tics - **"Let's see"** as a transition into tests — appears many times ("Let's see how he deals with that," "Let's see if it works," "Let's see what he's doing here"). - **"If you're new here" / "if you've just joined us"** — repeated audience-recapture phrase. - **"Okay" and "All right"** used constantly as pacing markers. - **Personification of models** ("he goes all the way," "he does whatever he wants," "he's token-hungry"). ## Audience Interaction Patterns - **Reading and answering chat questions by name**: > "Nicholas asked what the verdict is so far." > "Randy asks, 'What do you mean by "they launched Sonnet 3.5"? They released it last week.' Yes. Welcome to our daily life." > "There is an interesting question from Dar." - **Inviting audience participation**: > "If you try it out yourself, let us know what you think." ## Structure & Closing Conventions - **Hands off mid-task to co-host** to fill waiting time: "Kieran, do you have anything to show while this is cooking?" / "while it's charging?" - **Structured closing verdict** using a color scale: > "Where do you stand on this model? Red, yellow, green, or a paradigm shift?" - **Call-to-action ending** with subscription pitch: > "just make sure you subscribe to every.TO." ## Stable Patterns vs. Isolated Moments ### Stable (repeated across the stream) - Re-explaining the premise to newcomers (3+ instances). - "Let's see" / "Okay" pacing language throughout. - Compare-to-Opus framing for nearly every test. - Naming chat participants when answering questions (multiple instances). ### Isolated - The hydration joke (setup mid-stream, callback at close — only two instances, so its status as a running gag rests on thin evidence within this transcript). - The "Red, yellow, green, or a paradigm shift" verdict scale — appears only at the end; cannot confirm it's a recurring format element from this transcript alone. # Position Record: Live Vibe Check — Claude Sonnet 4.6 ## Context Two hosts (the main host, unnamed in transcript; and **Kieran**) conduct a live, unscripted test of the newly released **Sonnet 4.6** (transcript inconsistently names it "Sonnet 3.5"/"Sonnet 46"). No early access was provided; they test on the day of release. ## Central Claim Under Test - **Anthropic's framing (relayed by host):** the model is "a bit like Opus, but at the price of Sonnet." - The hosts' stated goal: verify whether it delivers **Opus-level intelligence at Sonnet pricing**, hopefully at Sonnet speed. ## Host Positions & Findings ### Creative reasoning test - Host used his recurring "alien civilization that never invented the number two" prompt. - **Assessment:** "That's actually the most interesting answer I've seen" — praised the depth of the explanation of the aliens' perceptual system (no category-grouping cognition; mathematics built on ratios/continua rather than discrete counting), going beyond prior models' "one → several" answers. - **Qualification:** "I'm not blown away by it, but I shouldn't be." ### Speed - Both hosts repeatedly observed it was **slower than expected for a Sonnet**; thinking time "closer to the level of Opus." - Host noted it "seems to be working much faster now" later in the session — an inconsistent observation, possibly attributed to launch-day load ("maybe they're overwhelmed with requests"). ### Work behavior / reliability - **Kieran** on PR triage and brainstorming tasks: it completed the work, sorted pull requests, identified themes, wrote changelogs — "I haven't found any reason to believe it isn't as clever [as Opus 46]. And if that's true, it's a victory." - Host observed mixed behavior: sometimes **overly cautious** (asking what to name a working tree) and then **overly zealous** (starting to rebuild the homepage when only asked to plan). Conclusion: "you have to be a little careful with this tool." - **MCP repair test (Kieran):** Sonnet failed repeatedly to fix the Figma MCP; **Opus solved it in one pass**. Kieran: "whenever I feel frustrated by a model, it's a deal-breaker." - Host's summary of that test: "if you need to solve more difficult problems, use Opus, but probably for many things, Sonnet is great." ### Token usage - Host's tentative impression: token consumption is closer to **Opus 4.5** than the more token-hungry **Opus 4.6** — potentially "a more balanced version." ## Pricing & Strategy Claims - **Prices cited by host:** Opus at $5/$25 per million tokens (in/out); Sonnet at $3/$15 — "not quite half the price, but it's getting close." - **Both hosts** on Anthropic's strategy: keeping tier prices stable while migrating performance downward (Opus performance → Sonnet price), which they say Anthropic has done before. Host sees this as giving developers **predictability** and enabling production use; Kieran agrees and notes it may be "taking a bit of a jab at Google." - Host's cost anecdote: their product **Spiral** (ghostwriter agent on Opus 46) costs ~$500–$1,000/day in tokens; switching to Sonnet 4.6 is "a huge win" — "things become cheaper without you having to do anything." ## Release Cadence Speculation - Host expects **monthly-or-two incremental updates** as "the new normal," viewing smaller predictable jumps as safer for production upgrades. - **Kieran, explicitly guessing** ("I'm just guessing"): new RL pipelines and better post-training allow gains without retraining from scratch. Host: "I really don't know." ## Final Verdicts | Speaker | Verdict | Reasoning | |---|---|---| | **Kieran** | **Yellow** | Frustrating failure on the MCP task; but "very usable and I think many people should use it" | | **Host** | **Yellow** for daily coding use; strongly **for** product/agent integration | "100%" agreement it's good for products — much cheaper than Opus; for real coding work, stick with Opus "if you can afford it" | - Consensus: it is **clearly not better than Opus**, probably somewhat less capable, at roughly half price — "quite significant." ## Preserved Uncertainties & Contradictions - Model name stated inconsistently ("Sonnet 3.5," "Sonnet 46," "Sonnet 4.6"). - Speed observations conflict across the session (slow → faster → slow again). - The 1M-token context window was discovered but not activated; both hosts unsure how to enable it (beta header vs. app setting). - Attribution of the RL-pipeline explanation is explicitly flagged as speculation by Kieran. - Host was unable to confirm whether Sonnet would handle the fiscal-year edge case in the P&L task before the stream ended. # Video Attribution Record ## Video - **Title:** Live Vibe Check: Testing Claude Sonnet 4.6 — Opus-Level Performance at Sonnet Pricing - **Format:** Live, conversational "vibe check" test stream with two hosts. ## Speakers & Roles ### Speaker 1 — **Host / Co-creator (unnamed)** - Primary host of the live test; opens and closes the stream. - Runs the creative problem-solving test (extraterrestrial civilization without the number two). - Presents the **fictitious profit-and-loss spreadsheet** task; explicitly states "All these figures are fictitious." - Discusses **Every Group** products ("we manage four different products," including **Spiral**, a ghostwriter agent running on Opus 4.6). - Discusses his project **Proof**, a markdown editor (described as Mac-native, now web-collaborative at proofeditor.ai). ### Speaker 2 — **Kieran — Co-host / guest** - Tests the model via **Claude Code**, including forcing the model identifier via environment variable. - Works on the **compound engineering plugin** repository (reviewing/merging pull requests, changelog work, brainstorming issue themes). - Tests the **Figma MCP** integration; struggles with MCP installation, ultimately resolves with **Opus 1M**. - Developing **Kora**, an AI-powered email assistant (prototype environment "Baby Kora"). ### Audience (chat participants, named only) - **Nicholas** — asked "what the verdict is so far." - **Randy** — asked about the Sonnet 3.5 launch timing. - **Dar** — asked comparison to GPT 5.3 low/medium. - Unnamed viewers — joined the shared Proof document live. ## Referenced Sources & Third-Party Claims - **Anthropic (vendor claim, unverified):** the model is "a bit like Opus, but at the price of Sonnet" — relayed by the host as pre-release information. - **Model pricing (as stated by hosts):** Opus at $5/M input, $25/M output; Sonnet at $3/M input, $15/M output. - **Context window:** mention of a 1M-token beta header / model identifier (2025807) sent via platform documentation and text message; hosts could not initially activate it. - **Tools/products referenced:** Claude app, Claude Code, Codex (OpenAI), Cursor, GitHub Copilot, Figma MCP, Proof, Kora, Spiral, compound engineering workflows, "Open Claw." - **Named individuals (mentioned only):** Eric (Cursor plugin author), Natash (colleague with prior 1M access). ## Promotional Segments (flagged) - **Hydration interlude** mid-stream ("stay hydrated to stay at the forefront of AI") — host promotion/bit. - **Closing call-to-action:** explicit subscription promotion for **every.to** ("Every is the only subscription you need to stay in the age of AI"), plus reminder to subscribe. ## Provenance & Availability Notes - **No early access program** existed for this model; hosts state they normally receive early access but tested this live. - Transcript is inconsistent in model naming (Sonnet 4.6 vs. Sonnet 3.5; Opus 4.6 vs. 45/46) — recorded as spoken, not reconciled. - Live audience interaction was possible (chat questions answered; shared live document editable by viewers). **Comment availability beyond the live chat is not stated.** - Hosts give explicit uncertainty: "I really don't know. I'm just guessing" regarding faster release cadence. Alright. Good morning. How are you, Kieran? How are you doing ? Yeah, great. Uh, here we are again. We were here last week. We're back. Someone said last week that we should make it a daily show, and we're starting to believe that it will be necessary. There are many new model launches, but we are here. We're going to do a live test of Sonnet 3.5. With this particular model, we knew it was coming, but we didn't have early access. So, we're going to test it live right now. There was no early access program for this model. So, we're just going to test it, see what we discover, and you'll see how we proceed. If you try it out yourself , let us know what you think. We are really excited about this . What Anthropic told me before the release, uh, is that it's a bit like Opus, but at the price of Sonnet. So we're going to check if that's really the case. I currently have it in my model selector on my Claude application. I know you have it too, but you were trying to get it from Claude Code. Are you able to do it? Uh, I'm trying in Claude Code. He's not here yet . But I'm trying to force the model number to see if that works. I have it in my app too, so it should work. Yeah, I'm just going to start, uh, damn. I'm going to share my screen while you install this. And I'll see. Okay, let's do some tests. First, something simple, maybe, uh, we can't see your screen yet. Can't you see my screen? All right. Yeah. So, there you have it, that's my Claude. I'm going to ask: can you switch me to Sonnet 3.5? It's just been released. Let's see if it works on my Claude. And here is my Claude app. What are the usual things to start with? Okay. Imagine this is one of my tests for the new cutting- edge models. Imagine an advanced extraterrestrial civilization that never invented the number two . How did they become so advanced? And why did they never invent it or need to use it? No cheating. They can't just walk over them. This literally shouldn't exist. Alright . Let's see. The reason I like it is that it's a kind of creative problem-solving question. And I think that touches on everything, from mathematics to the imagination and creativity of the model. I think Opus 46 has provided one of the most interesting answers I have seen so far . And each generation of the model gets better and more interesting. What I'm already noticing is that it's actually slower than I expected for a Sonnet model. It seems that the thinking time is a little closer to the level of Opus, which is quite interesting. It's a really difficult compromise, because "two" isn't just a symbol, it's the concept of another identical thing, which seems almost inevitable. For this to work, we need to get to the heart of what would make the number two useless. I find that interesting. The Vorith civilization evolved in a world of extreme sensory richness and radical ontological individualism. Their perceptual system, refined over millions of years, has never developed the cognitive habit of grouping distinct objects into a category and counting the instances of that category. Where a human sees two stones and thinks 'two stones', a Vorith perceives this stone and that other stone as fundamentally different phenomena that share no useful mental grouping. They don't philosophize about it. That's simply how their pattern recognition works. The stones are too different to be piled up in a heap called " two stones". This means that their cognition has ignored the fundamental step that produces counting, namely treating distinct things as equivalent instances of the same type. That's actually the most interesting answer I've seen. I 'm really curious to see how this continues. They have "one" and then they have "several". In fact, the shift from "one" to " several" is something I've seen before. But it never went this far in explaining the perceptual system and categories. Their mathematics developed as a system of pure ratios and continuous proportions. Imagine a civilization that invented infinitesimal calculus before it even invented addition. Their numbers are positions on a continuous spectrum, not discrete objects. They don't see "twice as much". They say the full doubling ratio applies to this. I don't know. Interesting. Uh. All right. It's good. But how do relationships work ? This question about the functioning of relationships and reproduction comes from the original Opus 46. It's as if it was the first model that said to itself, "My God, if there aren't two of us, how do we reproduce and how do relationships work?" " Uh, so he didn't understand that, which is quite interesting." Reproduction is a collective act with varying levels of participation. All right. Yes. Alright . Uh, the dyadic link still exists, but actually no. Okay, I think that's fine. That's, I think, pretty good . I'm not blown away by it, but I shouldn't be. I think what we're looking for here is to know if it's really an intelligence on the level of Opus, uh, at the prices of Sonnet and, hopefully, at its speed. Uh, oh, here we go. Stay in the flow with Opus 46. Kieran, were you lucky to get it? Yes, I ran it. I coded the model by hand and it works . Okay, great. You want to, you want to show us, uh, yes. Show us your screen. Yeah. So what I'm trying to do is, uh, what Opus is good at, uh, it's not shared yet, but uh, you can share it on screen. Uh yeah, understood. Great. Yeah. So what I'm trying to do is see how good it is at managing multiple things without losing track and making mistakes , because that's kind of Opus's strength. So, now uh I just have uh the composite engineering plugin. It's very active. Many people add things. So, we have a lot of new uh targets here. And I just want to review the pull requests , to see if everything is okay. So, Copilot added Kro and I think Cursor too. Uh yes, there's also a Cursor plugin made by Eric. So, I want to see how it can help; the thing is, I only use a template when it helps me do my job. I mean , I need to get things moving. If it works, it works. And I don't care whether it's Opus, Sonnet or Haiku. If it does the job, I'm very happy. And why not Opus here, compared to, uh, a model like this one ? Well, it's cheaper. So, you probably don't reach your limits as often. So, if this is a model that does the job for me, why choose Opus now? That's what wins. I just see it as work that I have to do. It's just, uh, yeah. So, it identifies small victories, uh, easy objectives. This one is mine. Plugin metadata and discussions. So yes. Okay, fine, recommendation: merge or close. All right. So, I like those ones. Let's go. All right. I like quick wins for number 80. Can you close it and say that we're going to, like, it's better to get it working rather than adding documentation. For the other two, can you examine them in the sub-agents and see if we can merge them with the main branch so that we can publish them and manage the exit processes? So, regarding speed, I found no results for the flows, I don't know what that means. Streams, okay, we found something new, the streams there, they published something , I don't know what "streams" means, but oh, output process, no match, what? Let's start again. Okay, I don't know what it is . All right. It was trying to evaluate something, I suppose. Oh, maybe because I had done a reset before. All right. So, in terms of speed, I agree. It's not , like, it's not mind-blowing. It's not like OpenAI's Spark that we saw with Codex, but also, yeah. So, the speed isn't breathtaking. I am very curious about the million, because it might be something I would prefer, even if it is not as clever as Opus, although they say it is. Even if it's a little less intelligent, if it really is a million usable tokens. Uh, there's a context indicator of one million, 2025807, that I'm going to send you, Kieran. All right. Because, yeah, I like it , especially with. Okay, forget it . Okay, that's just the indicator. Okay, great. Yeah, I just received that via text message from them. Uh, like this or the model or... Uh, I don't know if it's the model . Um, I sent you the platform documentation. Uh, what is it... It's, I think it's an indicator, not a model. All right. Yeah, the model identifier. All right. This is a beta header. Oh, okay. Beta header. All right. They make it really easy. Yeah. I know there was, yeah, there was already an indicator or there was a beta version of one million tokens. Uh, let's see. Oh, it's a header. Yeah, but the beta header, I think that's for the API. To be honest, I don't know how to do that in Claude. I think they need to add it somehow . I never had access to the million-dollar version, I think it was the 35. Maybe that was it. I know Natash, one of our dear colleagues, he has had it for about a year and I have never had it . So, um, that could be very useful, because what I've noticed with, uh, Opus 46, especially for sub-agents since it 's a bit token-hungry, uh, Opus 46, I often lack context, and there's a bug: if you go beyond the context, it doesn't automatically compact. Hmm, look, proof, proof just opened. That's interesting. Oh, that could be the hook, because it modified a markdown document. Oh yes , yes, yes, that's right. I should probably change that. I thought about making proof available only on the web, because I find the web version to be the most interesting. Yeah. Yes, I agree. And then, I use, I run everything via SSH , T-mux style. So, it's not on my local machine, and it's not where I usually code. So, for me too, having a web version would be great, because I could open it anywhere. Yeah. I think it's great and it works now, if you just ask to open it, like go to proof editor.ai and click create, create a document, it should work. specific uh okay , yeah okay I'll, I'll try that here too, I'll just create a new one uh draw, let's see what the model was already I'll just let it be Hum, this is the moment for me to remind you all that models are released every day . It is very important to stay hydrated to stay at the forefront of AI. So, I'm going to drink this because I feel like I'm getting dehydrated just from looking around here and working here. Work, work. So, if you don't drink about a glass every hour, you're not drinking enough. Please make sure you stay hydrated. Hmm, Nicholas asked what the verdict is so far. We are trying to determine that. So, if you've just joined us, what you need to know is that Sonnet 46 has just been released. Anthropic says it's essentially Opus, but at Sonnet prices. We are running tests to verify. So far, it looks pretty good, but it's still very, very early. And um, we don't see it, we expected it to be faster. Yeah. Qu Opus, but for now, it doesn't seem radically faster. Um, we're still studying, like, his actual coding, how good he is at programming. And I'm also upgrading my Claude to use it. Normally, we receive these models in advance. Well, we knew it was going to be released before it actually happened, but, well, this particular model didn't have an early access program. I think they're just moving super, super, super fast. So, uh, we do that often. We conduct our tests live, so you can see exactly how we play with these models and what we do. Basically, we're just trying to move forward with our work, we're just working, and then every now and then we add something fun or some other random element. Because for me, the most important thing is: can it help me do my job on a daily basis as well? Yeah. And there is no, there is no magic in our tests. It's just us who do this often and who really form an opinion. So, what I read in there is that I asked him to review all my pull requests on the compatibility engineering plugin and I said, "Hey, can you sort them ? "He came back with a plan and I said, 'Okay, that works for me.'" "And he did it ." So, he goes all the way. He did the job. It would be interesting to see how long it will run as well. I don't think it's suitable for streaming, as it could probably last a very long time. Um , I'd like to see how it compares to... Um, because what we saw with Opus 46 is that it's a bit more token-hungry . Uh, he goes into a bit more depth, which means he simply used more context. So perhaps this is a more balanced version, which still goes quite far but without using as many tokens or consuming as much of its context. Um, so maybe I'll start a search or let's brainstorm. That's a good idea. I'm also going to set up a collaborative work task that we can try while you do this. Yeah, let's do that. So you want to do it now then ? I'll prepare that quickly. Yeah , I need a little time to set everything up, but yeah. Okay, so uh, I'm going to do a quick brainstorming session. Great. Uh, uh, yeah, so brainstorming. I want to see because I have a bunch of problems that have come up here on... So I have a bunch of... I have a bunch of problems that have come up on compatibility engineering here. You can look, uh, you can retrieve all of that in parallel using agents and see what themes emerge? Um, and can you also make sure we label them all so that it's perfectly clear. Uh yeah, let's brainstorm what we should be working on. What are the feelings here? What we saw with Opus 46 is that it, uh, yeah, likes to push things towards tasks a little more often. It likes to do a few more things in parallel. Let's see. Yeah. And all these things, we do them randomly, but I have slash commands configured for that. Uh, but that's normal, what I actually like when we're testing is moving on to VIP coding. Instead of using, well, I use compound engineering workflows here, but I also like to just test things out. Because an assumption you had for creating a certain slash command may suddenly no longer be useful, either because the model is good enough, no longer makes errors, or perhaps it is fast enough that it no longer needs parallelization. So, going back to doing things manually from time to time, i.e., launching a slash command or having your own prompt and seeing how it changes, is a fun thing to do, and you can learn from it. But also using it in your workflows and with your skills and seeing how they change, I think both are very good to do. Yeah. Totally. Let me see if I have any. It takes a while actually, I have it...it's working on one of the things I did this weekend, I had to create a version of our profit and loss statement to send to some of our investors. And I, uh, I asked Opus 46 to take our profit and loss statement and make a sort of summary document out of it. So I just launched this with Sonnet 46. It seems to work, but it's working quite slowly. So I'm wondering if maybe they're overwhelmed with requests right now because they just launched the model. Uh, but I'm just sitting here waiting for it to come back. It's difficult to judge its quality, but the speed isn't great. Yes, same here. If you told me whether it was a new model or not, I would say no. It looks identical. It looks like Opus. Yeah. Yeah. It looks like Opus. Yeah. Which, I mean, is much cheaper than Opus. Gender. Yeah. If it's much cheaper and identical, that's still very good . I think the main thing is to say to ourselves, if it's the same as Opus and we can use it much more often, that's an excellent result. Opus costs $5 per million tokens in and $25 per million in. And the price of Sonnet is $3 for entry and $15 for exit. So, it's not quite half the price, but it's getting close . That's really interesting. And it 's also interesting to note that they don't seem to be lowering the price of the model levels. What they seem to be doing is keeping the same prices, but they release an Opus, then a Sonnet which offers the same performance. And they've done it before. Yes, they did it last time. Exactly, their new strategy is to have a consistent hierarchy and offer the same performance at a lower level while pushing the limits of the most powerful model , Opus. Yeah. This is quite interesting, because for a developer, it offers more predictability: if I want to stay on the cutting edge with Opus, I know what it will cost me in the near future. But as they improve the features, I'll be able to use Sonnet for specific tasks. Uh. It's true. Yeah. Yeah. And that's...Yeah. It looks like this is the one that will push people to use it in production, because Opus is great, but very expensive. I wonder if they'll do the same thing with Haiku, so that it reaches a similar level. This is interesting because the goal seems to be to say, "It's ready for production," which is ideal for those developing AI applications. They might be taking a bit of a jab at Google here . I think so too. Yeah. All right. Randy asks, "What do you mean by 'they launched Sonnet 3.5'? They released it last week." Yes. Welcome to our daily life. We've lost count of the number of tests we've done in the last two weeks, but it's enormous. It's about eight or something like that. And so I think that's a really important and interesting thing: they are able to make progress on the models much faster now. They therefore offer gradual updates, like once every month or two, instead of what was done before, perhaps every six months or every year for a major model launch. So I think we should expect this to be the new normal, with new model updates every month or two, and they will come in groups, as they launch Opus and then Sonnet shortly after. Yeah. And I think that's a good thing because the differences are less significant and more predictable, which I think is better for those using agents but also for those developing applications in production, because it's easier to upgrade. It's quite risky to upgrade a model that is very different. So I think I appreciate that they're doing that. I don't know why they didn't do it before. Or perhaps it's a new technological advancement or something like that. Do you have any idea why this has changed, or is it just the laboratories trying to keep up with each other or something like that? You know, I really don't know. Hmm, yeah, I think they have new RL pipelines that make things um, that allow them to do it faster. It's just more refined, and I think they've figured out how to handle post-training, so they don't need to train a whole new model from scratch to achieve gains. But I really don't know. I'm just guessing that's why it's happening. Yeah. Yeah. There is an interesting question from Dar. How does this compare to GPT 5.3 low/medium? It's interesting that Anthropic is keeping its Opus, Sonnet, Haiku organization. Um, uh, I mean, how does it compare? I think it compares to , as they say, Opus 46, and it probably gives that impression, but cheaper. Hmm, but they also have modes of reflection in Opus 46 now. So they have that, but the question is: why are there other models instead of ways of thinking? Uh, what's interesting, but I think it's more like, for example with Google's Flashlight, it's just a smaller model rather than a reflective mode, so it's for a different use. Um, I'm going to share my screen while we wait for yours to come back, because I have the anonymized profit and loss statement that I can show. So, take a second. All right. So basically, what I was saying, if you 're new here, one of the things I think we want to see how good this model is for is, uh, how good it is for, uh, uh, tasks that are, there you go . It's good. All right. How good it is for tasks that are a little more oriented towards collaborative work. So what I did is this is a profit and loss statement spreadsheet. This is not , this is not based on our actual profit and loss statement. because I experimented with this over the weekend in Opus 46. All these figures are fictitious. Um, just FYI, but basically, what I'm going to do is I'm going to take this spreadsheet. It's a very complex spreadsheet. And I'm going to put it in Co-work, and I'm going to perform a task that I literally did this weekend, which is to say, "Can you take a simpler version of this very complicated sheet that contains many different sheets inside and turn it into a summarized profit and loss statement that I can send to my investors?" And I'm going to use Sonnet 46. I'm going to say "here we go". Well, we'll see how he does. Opus was very good at that. I actually used Opus rather than Codex for this this weekend. And I think it was a good task for Opus. I asked Codex to change the numbers, or rather to check the numbers on it. But yes, I just think Opus understands better that, if you do a profit and loss statement like this, you are trying to tell a particular story and you know how to present it, you know, not in a way to "fake the accounts", just in the sense that "there is something you are trying to show". And uh, so I asked Opus to do it, then I asked Codex to check the figures and it was pretty good. It seems to be working much faster now and uh yeah, it looks like the intelligence, it seems pretty intelligent. I have the impression that version 45 would have already dropped out at this stage, well, version 45 has been out for a while. Yes, and I get the impression that in terms of token usage, it's a bit like, it's more like Opus 45 in terms of token usage. So, the 46 increases the tokens a little. So if you encounter problems like "too many tokens, it takes too long". Mhm. In terms of tokens alone, I think this one is for you. If you liked Opus 45, this is probably very similar, maybe even to 46 , but in terms of token usage, it's more like it. Yes. Okay, fine. So that indicates your forecasts. So that's a good thing. He asks that questions be asked of him . So, optimistic scenario, base scenario. I'm going to choose the optimistic scenario, why not? What level of detail about revenue do you want to display? I think that's fine . Uh...yes. So, one point I didn't ask for and which intrigues me: I asked for a report on, uh, the last 12 months. And I wonder if he will understand that he needs to do a full fiscal year for 2025 and then a month in 2026 in this profit and loss statement, or if he will simply do February to February, an error that Opus made yesterday and that I was only able to spot thanks to Codeex. Uh, let's see what he's doing here. Again, if you're new here , Anthropic has just launched Sonnet 46, which they're sort of presenting as Opus, but at Sonnet prices. And Sonnet isn't quite 50% cheaper, but it's almost there . So if you're thinking about where and when to use this model, for now, it doesn't seem ultra- fast. It doesn't necessarily look like it, it seems to be the same speed. It seems a bit slow, but that's okay if he 's intelligent, if he gets the job done. Yes. Yes. I think what we're trying to determine is: is he really as intelligent as Opus? Because if that's the case, then, uh, the use cases it opens up are numerous , especially if you're developing applications where you want to integrate an agent. There are many cases where Opus would be too expensive, because it is really very costly, and which Sonnet now makes possible; or maybe even for your Open Claw, for example, which I tried earlier and completely broke, so I have to figure out how to fix it but, uh, even for your Open Claw, you know, having your Open Claw billing every, every, uh, every month, it's really important if you can get the same level of performance. And so far, the performance seems reasonable. It's clearly not better than Opus, but it's not clearly bad either at the moment. Uh, but we 're still testing the feel. Normally, we receive these models in advance. For this one in particular, there was no early access program, so we're doing this live, folks. You 'll see how we do our " vibe check," which is basically saying : here are some tasks I do on a daily basis. Like, how does he deal with that? Mhm. And we're using the normal context window , not the million-dollar one. So the speed corresponds to that. Uh, we haven't figured out how to activate the million. Yeah. In any case, not me. Hey Kieran, do you have anything to show while this is cooking? Yes, he finished everything. So basically, uh, let's see. So I work on compound engineering and uh first we worked on pull requests and published stuff, uh we uh merged pull requests. We fixed some, uh, uh, merge conflicts. Then I started, uh, okay, so now we have Copilot. So, uh, Copilot, uh, Cursor is added, uh, to the Cursor marketplace, which Eric kindly did, and we also have Cursor here. Yeah. So yes, we merged that, so native Copilot in Cursor, which is great for anyone whose company pays for it and wants to try it. It's, uh, it's really cool. Uh, he wrote the changelog. So, what I see is that it's intelligent. It's very clever, in the sense that it knows what to do. Also, I kind of trapped him here because I'm not in this warehouse. I'm at my Corora depot here. It's something I enjoy doing. Where am I? Just in another random project. And then I said, "Hey, can you go work on another project?" "Because I think that, uh, if he can do that without getting confused, it's something that has been introduced in, uh, in, uh, yeah, in the more recent models." So, I like to do this to see how well it works on different elements and how it finds the solution. So I did all that and then I said, "Hey, can we do a little brainstorming?" I want to see what the problems are. "So he looked at everything and said, ' Hey, the first theme is broken workflows,' with the whole list, and I went through it and thought, ' Yeah, actually, this is good work. "The second theme is actually where we reach the limits of context, and it is literally something that will, I hope, be resolved if you use this model or the one of a million. So, uh, maybe we can just mention, uh, that kind of thing, like, uh, multiplatform, stuff like that. Uh, I think, uh, yes, and then I said, "OK, can you combine that and do it?" "And, uh, yes, he went through everything , uh, corrected those points where he said 'always do this'." Uh , I removed the Ralph Wigum skill, which, uh, isn't really necessary in this case, which I agree with, and, uh, yes, he pushed that and published it. So it gives the impression of being very capable and it's, uh, perhaps as clever as Opus 46. I haven't found any reason to believe it isn't as clever. And if that's true , it's a victory, because I can do twice as much with my limitations, or maybe even more. Like, I can just get more work done for the same price or subscription, which is great. I have reached my limits and I have already exceeded my consumption for December and January. So maybe this is the month where my $200 plan will cover everything I do, which is pretty cool. I can buy a second one obviously, but, uh, it's a great victory to be able to do all this. Uh no, I completely agree. Uh, yeah, I mean, for us, you know, at Every Group, we manage four different products, and, uh, one of our products, Spiral, which is a ghostwriter agent, runs on Opus 46 and it costs us, like, I think, between $500 and $1,000 a day just in token costs, so it's, uh, completely out of control, and we're about to start a big project to, uh, make it cheaper, but it's gone down in price all by itself because I think it happened today, I just put Spiral in and it's a huge win. I find it, uh, really interesting to build an AI where, in general, things become cheaper without you having to do anything, which is really nice. Yeah. Another thing I want to try is, uh, there's also the Figma MCP that was launched, uh, the Claude code to Figma. So, I'm going to transfer a design to Figma. Let's see how he deals with, uh, the visual loops. Uh, let's see . By the way, how do you get Sonnet 46 working? You just do the "-m" and then the model, the model token. Yes, I just put it in an environment variable, like, what is it ? Claude, a model or what? Anthropic model or something like that. Let's see. I can, I can do it here. I can show you here. Anthropic model claude 46. Okay, noted. I'm going to paste it into the chat. CC is my code . Okay, just let me go to the right directory and I'll launch it. So, we're here in Baby Kora, my test environment for Kora. Um, which is the application I'm currently developing? It's an AI-powered email assistant, and I prototype things in Baby Kora, then when I like them, I transfer them to the real Kora. We are in the design phase and I want to see if we can transfer the settings page to Figma. The Figma MCP is therefore connected. Let's see how he does. All right. Can you convert the settings page we have here in Baby Kora? Can you use the Figma MCP and uh create this design in Figma so that we can pass it on to the designers? Okay. While this is filming, I have a few things to show you. So, here's where we are on the P&L task. And you'll see that it only takes a few minutes. I think that's wrong. I think it's been longer than that. This is only the most recent part. It's true that it takes some time, you know. So, if you're expecting a major acceleration, I don't think you can expect it. But it does things like build a fairly complex spreadsheet, and then be able to recalculate everything inside it. Well, I'm going to check the work here to make sure everything is correct. But it's promising; we can probably count on him for complex, multi-stage tasks. Another interesting thing is that I just configured my cloud environment on it. So I can say that it runs on a cloud plan in 4.6. Let me give him a small, classic task. Can you tell me when my next flight is and what I should probably plan for? He has access to Kora, the email application that Kieran is developing, which has a CLI . He should therefore be able to search in Kora and give me an answer. That's the kind of thing Opus could do, we'll see. We'll see how he does. Uh, on that subject, another thing, I'm working, uh, as you know Kieran , on a project called Proof, which is a native markdown editor. It was an entirely Mac tool, a Mac application, and now it's become, uh, like a Google Docs but for agents and humans. Here is an example of a Proof document. You can share it. It manages a complete presence and works equally well for agents and humans. So you can collaborate on documents very easily . But this home page is really specific to the Mac experience. So what I want to do is just say, uh, redo, oops. That's interesting. Basically, I asked Sonnet 3.5 to open a working directory tree. He asked me what to name it, which is unusual. Normally, a Claude would just say: "It's started. I'll take care of it ." Yeah. It adopts a behavior closer to OpenAI, which is probably what you want. Are you sure? Exactly. Exactly. So now I'm going to say: "Okay, basically, the homepage...I feel like it doesn't really reflect all the new things we've done with, uh, Proof on the web, uh, and the live collaboration." And, uh, it's not really designed for agents to work well on. So what I want you to do is put together a plan to migrate the homepage to the web, with maybe a separate subpage for the Mac experience, uh, and, uh, slash, uh. That's really interesting, actually; If you look at this, uh, he's been way too cautious here. And then he said to me, "What should we name this working tree structure ? Give me a short name," and I said "redo homepage," and he created the tree structure, then he started to redo the homepage all by himself. That's really interesting. He is both too cautious and, like, too zealous, and too impatient. Uh, so I'm just going to delete that. Okay. That's interesting, I don't think Opus would have done that. So, uh, I would say that you have to be a little careful with this tool. He might just go for it and do something you don't expect. But I'm going to get Kieran's composite engineering workflow going here so he can come up with a plan and we'll see how good he is. Uh, great. Kieran, do you have anything to show while it's charging? Yes, I'm trying to do the thing with Figma and, as always, the MCPs are not installed correctly, they don't work or they're not there. So I'm trying to install the Figma MCP here, but it's struggling. So uh...yeah, I'm not sure about that, but uh the ...Yeah. It looks like...let me see if I can still fix it . You can share my screen so I can show you the... Yeah. So. Yeah. So, I'm trying to get the MCP working here, which I think should just...just work. Uh, but he's struggling. Let's see how good he is at self-reflection and if he knows how to fix things himself. I think Opus is really good at that. So, let's see if we can push it a little further. Okay, for some reason, it's still not showing up . Um, can you repair the MCP? Because we really need it, and then I'll see if switching to Opus solves the problem. Because obviously, I can uh...uh revoke access to the Figma MCP and reinstall it. Yeah, it didn't...it didn't even show up in the MCP list here. And that's also why I hate MCPs. I love command-line skills. It's so much easier. I understand why MCPs exist, but...Oh, he's done something now. Okay, so restart Float Code. Let's restart. Let's see if he's here now. Okay, he didn't do it the first time, and he's still not here. Look, he's not here. Uh, let's see. Okay, he's not there. Let's try again . Still not here. Is this the...take it back? It's quite frustrating. Uh, still not here. I...let me see if I go after that . I'll skip over Opus. Yeah, " resuming" is convenient. Uh, it's basically continuing, but starting over. You can choose whichever one you want. You can just resume any previous session. Hmm. Okay. It looks like a step backwards, but maybe...maybe it will be done the same way with Opus. What ? No no no no no no. He...he does whatever he wants. Okay. That's right...that's right...Yeah. Okay. He's doing whatever he wants. Let's move on to the model . Uh, oh yeah. Okay, he's actually here . Opus 1 million. Oh. 1 million, since when has Opus 1 million existed? Wait, there's... Well, there's also Opus 1 million, folks. Uh, interesting. I wonder if I have it too. No. Well, someone at...Thanks to whoever is watching. It had already been a year since I had access to these things, even though I knew people who had them. So, thanks to the person who activated this, or to the algorithmic gods of chance. Okay, let's see if Opus will solve this. I also have Opus 1 million. Interesting . Nice. Awesome. Well, that's cool . Opus 1 million is really something I need because I'm running out of tokens every second. So, uh, because not all the compaction was working, which was a bug, but now the bug is fixed because we no longer lack space. Okay, let me see. Um, can you take a look and get this working? While you're doing that, I have things to show you. Yes, go ahead. I'll get back to you. One thing. So before we, before we stopped, I had asked him to make a plan for redesigning the homepage. That's Sonnet 46, and you remember, uh, he did that thing where, when I asked him to start a new working tree, he said, "Before I do that, what should we call it?" —which is kind and prudent. And then I said, "Redo the homepage," and he created the site map, but then he really started to redo the homepage. So, he took the plunge when he shouldn't have . So we see him doing stupid things. Uh, which means we need to keep a little bit of an eye on it . But, I asked him to make a plan, basically, to make a plan to redo the home page, uh, for this web markdown editor that I called Proof. And I asked him to open it, write the plan , and then open it in Proof. And you'll see, it's right here. And in fact, I think I could probably give you the link. I'd be curious to know if it actually works. So , I'm sending you the link. Everyone can now go to Proof and we should see the number of viewers start to climb here. Uh, it's a live document, so you can make changes, you can change it. Um, so feel free to go here if you want to get into the document, but let's see, let's look at this plan. So the redesign revolves around the lead from the homepage. Okay, great. Good title, blah blah blah, static screenshot. Oh, Kieran is here. We have Nick with us. Allen's over there . Awesome. Uh, that's pretty cool, isn't it? So, it's really live. Yes, very cool. Yeah, just point your agent to proofed editor.ai and you can create live markdown documents. I did that over the weekend. Uh, okay. Let's see. Let's see how many websockets you can handle. Yeah, I bet we're going to crash the server. Uh, okay. So, one thing I notice with Opus, as we were saying, is that it seems slow with Sonnet or Opus. Okay then . So, we said that Sonnet 3.5 seems a little slow. I'm switching back to Opus 3.5. Much slower. Oh really ? Yeah. It seems much slower. And it seems much slower because it takes longer to execute the sub-agents. I think Opus is more geared towards sub-agents and uh yeah so I think, yeah, it's definitely faster than Opus, but I think it's mostly faster because it does less. In addition, I always use Opus in high effort mode , so that may also make a difference. Okay, so I'll add it to the plan. Okay, so I think it looks good, uh, everyone is so polite, nobody is touching this plan. Okay, I've made a comment. Please add it and execute the plan in order. Okay, cool. So, let's see if it works. Oh, now we have people who tinker. I like it. Uh, oh, wow. That's interesting. Yeah, I placed order Z and I saw that. Okay, that's a good bug to fix for me. Sorry, that was me. Yeah, yeah, yeah. No worries. No, it's just the proof metadata. Basically, what I know is that it follows who wrote what. Um, and somehow, command Z caused this. So, you're going to see a little bit of my workflow. Uh, that's something I would ask Codeex. So, I was in this document , Kieran was there too, Kieran typed something, pressed command Z , and the proof metadata appeared at the bottom. Can you diagnose that? Find the root cause. And I just think Codeex is good for this kind of thing. And I'll try a little too. Uh, yeah, cool. Um, is there something you want to show? Yeah. So, uh, Opus came back in one go and found the problem. So there you have it, that's the difference. It's as if Opus is right, if it gets a little too meta or something, he just says: "Oh, that's a known bug. He found it here, which is an incredible source, and it's the wrong config file, just do this to fix it. "Let's see if that 's really true." What I like about it is that it doesn't go around in circles and it doesn't pretend to know how to do things. It just says do this, uh, very practical. Uh... let's see if it works now. Yeah. All right. Okay. Well, so far, so good. Okay. So, so Opus won on this one. And Sonnet had a hard time, and it's that feeling where you feel like you're running into an invisible wall and it's there, like it's always there. So if you need to solve more difficult problems, use Opus, but probably for many things, Sonnet, uh, is great. Let 's come back here. Let's move on to Sonnet. Also, uh, the million is good, but you pay extra, so you need to have an additional usage activated, similar to slashfast. Hmm, so I don't think many people will do it, but it's very interesting to be able to do it and see what it's like. So, let's go back to Sonnet and see if we can resume our design project. Dan, Julia. Oh, it's a Zoom. Okay, cool. Uh, sorry, I, I, I'm crashing the party. Uh, so Kieran, I apparently have a Zoom meeting I need to join now. Yeah, me too. Okay everyone , a quick final summary. Hmm, Kieran, where do you stand on this model? Red, yellow, green, or a paradigm shift ? Hmm, I'd say yellow because of the last thing. It wasn't great, it was just frustrating. And whenever I feel frustrated by a model, it's a deal-breaker. But I think it's very usable and I think many people should use it. Yeah, I think I'm kind of the same, in the sense that I'd say yellow for my actual daily use. I'm totally against integrating it into a product you're developing. Yes, 100%. Hmm, this will allow you to do things at a much lower cost that you would have had to pay full price for with Opus. So, yeah, to create a product, I highly recommend trying this. If you're coding or doing real work, I'd probably stick with Opus if you can afford it. Well, if you have the $20 a month plan and it gives you more, that's great. It's...it's really good. So what I expect is that it won't be, uh, better than Opus. It's probably a little less good in some respects, but it costs a little more or a little less than half the price. So, you see, it's quite significant. That's it for our atmosphere test today. If you want more tests, we do them as soon as a new model comes out. Generally, we get early access to it. Um, just make sure you subscribe to every.TO. Every is the only subscription you need to stay in the age of AI. Uh, and don't forget friends, stay hydrated.