A hands-on review of OpenAI's GPT-5.5 after three weeks of testing at Every, with a focus on its coding ability. On Every's Senior Engineer Benchmark, GPT-5.5 scored 62.5/100—about a 30-point leap over Claude Opus 4.7, while human senior engineers score in the 80s and 90s. Key findings: GPT-5.5 rewrites code from first principles rather than patching existing code, performs strongly when paired with a contract-style plan written by Claude Opus 4.7, and is a restrained, effective business writer and voice mimic; Claude Opus 4.7 is still more trusted for sharp insight and evaluation and wins vibe coding from underspecified prompts. Topics include the benchmark methodology, why 5.5 executes better, real-world build tests, where 5.5 falls short, language strengths and weaknesses, writing quality and voice, knowledge work with the Codex desktop app, and a daily-driver verdict. # Channel Identity Observations: "We Tested GPT-5.5 for 3 Weeks. It's a Beast." ## Explicit Self-Description (Creator Claims) - **Organization identity — "Every":** The creator repeatedly frames the channel as part of Every, describing a paid offering: "if this is interesting to you, you should go to every.to and subscribe." Team members are identified by role: "Naveen, who's the GM at Every who runs Monologue," "Kieran Classen, the general manager at Cora," "Mike Keller, who runs technology consulting at Every." - **Scope of the organization's work, as stated:** - Daily written AI coverage: "We write new things every day to keep you at the edge of AI." - Software products: "We have a fleet of six apps that we've built with Codex and Claude Code" — naming Cora (email agent), Sparkle (file organizer), Monologue (vacation app) — "release[d] to our audience as part of our subscription." - Education/training: "we run camps and courses to teach you everything you need to know to code, to write, to do design with AI," including "for big companies and executive leadership teams." - **Publishing format claim:** Timely model reviews called "vibe checks," released on model release days, in both video and written form: "you can get it into your work and into your life today… you go to the link down below, you're going to see our written version… a couple of thousand words." ## Observable Presentation (What the Material Demonstrates) - **Intended audience:** AI-forward professionals — developers, writers, and knowledge workers evaluating frontier models for practical use. Evidence: benchmark results framed against workflows ("what its drawbacks are… get it into your work and into your life today"); categories covered are coding, writing, and knowledge work. - **Claimed scope demonstrated in practice:** Hands-on comparative model evaluation, not just news. Evidence: internal testing "for about 3 weeks" before release day; a proprietary benchmark ("the SE bench… a benchmark I invented") comparing GPT-5.5, GPT-5.4, and Opus 4.7 against human senior engineers; a colleague's "LFT bench" for product-forward engineering tasks. - **Team-based, product-backed credibility:** The review incorporates named colleagues' real usage (Naveen's ~900 million tokens of pre-release testing; Katie Parrot, "a staff writer at Every," switching models). This demonstrates that model reviews draw on an operating software business, not solo opinion. - **Practical, prescriptive framing:** Findings are converted into usage advice: "if you want to get the most out of 5.5, using Opus 4.7 as the planner… is going to be the thing that draws out this boldness"; "if you're in TypeScript or you're in Swift, you're going to be pretty happy." - **Commercial conversion pattern:** The video funnels viewers to the written report and subscription: "you should go to every.to… We also have a suite of apps for subscribers." - **Tone and sign-off conventions:** Enthusiastic, insider-flavored frontier framing ("That's our report from the frontier") and a recurring outro ("stay hydrated. See you next time on the moon"), suggesting an established series identity. ## Audience & Scope Summary | Dimension | Evidence-Based Observation | |---|---| | **Audience** | Professionals applying AI to coding, writing, and knowledge work | | **Claimed scope** | Daily AI journalism, AI-built software products, training/courses for individuals and enterprises | | **Demonstrated scope** | Independent multi-week model testing with proprietary benchmarks and team-based evaluation | | **Creator claims vs. demonstration** | Claims of rigorous testing are corroborated by concrete benchmark numbers (e.g., GPT-5.5 at 62.5 vs. humans at 80–90 on SE bench); no contradiction observed in this video | # Delivery Observation: "We Tested GPT-5.5 for 3 Weeks. It's a Beast." ## Overall Tone - **Enthusiastic, casual, tech-reviewer style** throughout, with conversational asides and informal vocabulary. - High-energy opener: > "It's model release day. Let's go." - Recurring slang/superlatives: **"beast"**, **"sick model"**, **"vibe-coded slop"**, **"oomph"**, **"ripped through"**. ## Vocabulary & Recurring Phrases - **"vibe check"** — used as a branded recurring segment name: > "those are the days where we publish our vibe checks. This is a video version of the vibe check" - **"step change"** — repeated evaluation language: > "it's actually like a real step change" ... "it is a step change in terms of how good it is versus other models" - **"benchmark"** — extremely frequent; paired with a named, self-invented metric: > "It's a benchmark I invented... the SE bench" - **"plan/planner"** as the central conceptual frame of the review: > "using Opus 4.7 as the planner... is going to be the thing that draws out this boldness" - Anthropomorphizing language for models: **"boldness," "agency," "confidence," "personality," "daily driver," "psychology of this model"**: > "this tells us a lot about the psychology of this model" ## Structural / Teaching Habits - **Announced categorical structure** up front: > "I've got four main categories for you. We've got coding, we've got writing, we've got knowledge work, and we've got open claw." - **Explicit transitions between sections**: "Okay. So, let's start with coding." → "Okay. Now onto writing." → "Okay. Now onto knowledge work." - **Headline-first delivery**: states the key finding before explaining methodology: > "The headline is a senior engineer benchmark." - **Comparative scoring** with concrete numbers repeated for reinforcement (62.5, low-30s Opus, 80–90 humans). - **Citing named team members as evidence sources** (Naveen, Kieran Classen, Mike Keller, Katie Parrot), with credibility framing: > "He means an incredible senior engineer, and if he likes this model, it's worth paying attention to." - **Balanced assessments**: consistently pairs praise with caveats: > "There are some things that it's not as good at." ... "It's not perfect yet. It's far from perfect." ## Communication Patterns - **Direct audience address and calls to action** (repeatedly): > "you should go to every.to and subscribe" ... "please leave a comment" - **Mid-roll promo block** describing the company's products (Cora, Sparkle, Monologue, camps, courses). - **Signature sign-off** (appears to be a habitual closer, though only observable once here): > "remember, stay hydrated. See you next time on the moon." - Filler and speech disfluency present in live-style narration: "um", "cuz", "like": > "I feel like it's Christmas every couple weeks around here cuz we keep getting access to new stuff" ## Stable Patterns vs. Isolated Moments - **Stable (repeated)**: numeric benchmark comparisons, plan-centric framing, section signposting, promotional CTAs, casual slang. - **Isolated / uncertain**: the sign-off phrase and "Let's go" opener occur only once, so their status as habitual delivery features is **uncertain from this single transcript**. # Observation Record: "We Tested GPT-5.5 for 3 Weeks. It's a Beast." ## Video & Speaker - **Video:** GPT-5.5 vibe check from Every (every.to), published on model release day. - **Speaker:** Unnamed narrator (transcribed host, presumably Every's lead; speaker identification not explicit in transcript). Other contributors quoted/attributed by name: Naveen, Kieran Classen, Mike Keller, Katie Parrot. ## Headline Claims (Coding / SE Bench) - GPT-5.5 tested internally for ~3 weeks; host calls it "a real step change in a lot of abilities." - **Senior Engineer Benchmark ("SE bench")** — invented by the host: model given a "vibe-coded slop" code base (from an app called **Proof**) and asked to rewrite it from first principles. Gold standard: rewrites by two human senior engineers. - Scores claimed: - Human senior engineers: **~80–90 / 100** (benchmark not saturated). - **GPT-5.5 with an Opus 4.7-written plan: ~62.5** (best score ever seen on the bench). - **GPT-5.5 without external plan: low-to-mid 40s.** - GPT-5.5 with self-prompted planning: **low-to-mid 50s** — host concludes **Opus 4.7 is the better planner**. - **Opus 4.7: ~33** (low 30s), producing a ~30-point gap vs. GPT-5.5. - **Key qualification:** the headline gap only holds with an Opus 4.7 plan. ## Reasoning About Model "Psychology" - GPT-5.5 can identify core principles/invariants in a plan and then **execute boldly**: delete files, avoid "patch mode," carry an idea through over hours. "It's not perfect yet... still about 30 points away from senior engineers." - Opus 4.7 writes **beautiful, contract-driven plans** (specific: "this file will only be 100 lines") but, given its own plan, "picks off a small slice" and patches instead of rewriting. GPT-5.4 does the same, slightly better. - GPT-5.5's own plans have right concepts but are long, human-friendly, lacking contract-style specificity — host infers tuning toward humans; Opus 4.7's terseness suits 5.5's execution. - **Practical recommendation:** use Opus 4.7 as planner, or prompt 5.5 with exact detail. ## Team Testimonials - **Naveen** (GM, Monologue): built native iOS/Mac to-do app "Dayline" — impressed; used ~900M tokens pre-release on a Monologue release; calls it his favorite model; claims he hit his deadline only with it. - **Kieran Classen** (GM, Cora), LFT bench (product-forward/front-end tasks): 5.5 did well, but **Opus 4.7 better for design-forward tasks** — "better aesthetic sense." Kieran (a "Ruby stan") says 5.5 **doesn't write good Ruby**; strong at **TypeScript and Swift**. - **Mike Keller** (tech consulting): vibe-coding a complex app from an under-specified plan — 5.5 didn't do as well; Opus 4.7 "ripped through the entire task." ## Writing - Less personality than older Opus (4.6) but strong for **business writing**; host one-shotted a near-ready investor update. - **Katie Parrot** (staff writer): first GPT model in a long time she uses over Opus/Sonnet; she and Mike praise its **voice replication** — restrained, subtle, doesn't overdo style. ## Knowledge Work / Desktop Agent - Host: OpenAI "was so behind" 3–6 months ago but now the **Claude X desktop app + GPT-5.5 is best-in-class** desktop agent experience: fast, uses computer apps, browses, builds dashboards, data analysis. - **Opus 4.7 "feels slow"** by comparison — attributed to OpenAI hardware advantage. - **Trade-off:** training for digestibility cost some detail/insight sharpness — for sharp-insight tasks host recommends Opus 4.7; e.g., he trusted 4.7 more for grading SE bench trajectories even while preferring 5.5 as daily driver. ## Open Claude / Harness - Host switched desktop/agentic work mostly to Claude X; harness "still built to work better in Opus," but improving daily; more stable (less forgetting, fewer mistakes) recently. - Free under ChatGPT subscription — accepts power trade-off. ## Contradictions / Unresolved Tensions - **5.5 as step change vs. 5.5 underperforming Opus 4.7** on design-forward, under-specified, and detail-sharp tasks. - Benchmark performance **depends on a competitor model's planning**, which the host flags as a workflow insight rather than resolving. - "It's a beast" headline coexists with "not in the senior human engineer range yet." ## Promotional Content - Plug for every.to subscription: written vibe check (~2,000 words), six in-house apps (Cora, Sparkle, Monologue, etc.), camps/courses and corporate training. # Speaker Attribution Record — "We Tested GPT-5.5 for 3 Weeks. It's a Beast." ## Video Provenance - **Video ID:** `01a0f15a-8922-7dd2-af62-91fa9c893742` - **Format:** Single-speaker video essay / "vibe check" report, released on GPT-5.5 launch day - **Curated metadata:** None supplied; all attributions below derive from the transcript ## Speakers & Roles ### Primary / Sole Speaker - **Role:** Creator / host (unnamed in transcript) - **Evidence:** First-person narration throughout ("We've been testing it internally… I invented this benchmark"); addresses audience directly ("you should go to every.to and subscribe") - **Affiliation:** Every (media company) — references "our senior engineer benchmark," "our written version," "the Every subscription" ### Referenced Third Parties (quoted/paraphrased, not speaking) - **Naveen** — GM at Every, runs **Monologue**; tested GPT-5.5 on to-do app "Dayline" and a Monologue release (~900M tokens pre-release). Paraphrased, not directly quoted. - **Kieran Classen** — GM at **Cora** (Every's AI email agent); tested on "LFT bench." Opinion paraphrased: prefers Opus 4.7 for design-forward tasks; GPT-5.5 "doesn't write good Ruby." - **Mike Keller** — runs technology consulting at Every; benchmarks paraphrased (Opus 4.7 better on under-specified vibe-coding tasks); also praised voice replication. - **Katie Parrot** — staff writer at Every; paraphrased as switching from Claude models to GPT-5.5 for writing. ## Referenced Sources & Claims - **"Senior engineer benchmark" (SE bench)** — benchmark invented by the creator, using a real code base from an app called **Proof**; gold standard from two human senior engineers' rewrites. - **Model comparisons cited:** GPT-5.5 (~62.5 with Opus 4.7 plan; low-mid 40s without; low-mid 50s with own prompted plan), **Opus 4.7** (~33), **GPT-5.4**, **Opus 4.6**. - **Third-party model releases referenced:** OpenAI (GPT-5.5, GPT-5.4, ChatGPT subscription, Claude X desktop app / Open Claude — names as given in transcript), Anthropic (Opus, Sonnet models). ## Promotional Segments — Flagged - **Opening/subscription promotion:** Written vibe check link "down below"; every.to subscription; fleet of six apps (Cora, Sparkle, Monologue); camps, courses, corporate training. - **Closing promotion:** "go to every.to," subscription apps, "like and subscribe," comment solicitation. ## Audience Interaction & Availability - **Comments:** Explicitly invited ("please leave a comment"); no comment availability status supplied. - **External links referenced:** every.to; written vibe check link (content not available in this record). It's model release day. Let's go. Today is the release of Spud GPT-5.5 from OpenAI. We've been testing it internally and every for about 3 weeks. It's a sick model. We've tested it on everything from coding to writing to knowledge work tasks, and it's actually like a real step change in a lot of abilities. The big headline for me though is its coding ability. On our senior engineer benchmark, it scored 62.5 out of 100. And by comparison, human senior engineers score consistently about 80 or 90 out of 100 on this benchmark. It's not in the senior human engineer range yet, but it's getting up there. Opus 4.7 by comparison scored in the low 30s pretty consistently. So, there's a 30-point gap between the work that GPT-5.5 does and the work that Opus 4.7 does. But, there's a catch. The best performance that we got out of GPT-5.5 came when we ran it on our senior engineer benchmark using a plan written by Opus 4.7. Write a plan with Opus 4.7 and GPT-5.5 is an absolute beast. We're going to go into a ton of detail on all the tests we ran and what we found, what this new model's useful for, what its drawbacks are, you can get it into your work and into your life today. I love model release days because those are the days where we publish our vibe checks. This is a video version of the vibe check, but if you go to the link down below, you're going to see our written version. It's a couple of thousand words with all of our detailed thoughts on exactly what we did with this model, what we thought, released on the day of. We write new things every day to keep you at the edge of AI. We also build things ourselves as part of the Every subscription. We have a fleet of six apps that we've built with Codex and Claude Code with these models to help us in our work. Things like Cora, which is an agent for your email, or Sparkle, which is an antique file organizer, or Monologue, which is a smart vacation app. All these are things we build ourselves and release to our audience as part of our subscription. We also do a lot of training, so we run camps and courses to teach you everything you need to know to code, to write, to do design with AI. And we even do this for big companies and executive leadership teams. So, if this is interesting to you, you should go to every.to and subscribe. Okay. So, let's get into the vibe check. I've got four main categories for you. We've got coding, we've got writing, we've got knowledge work, and we've got open claw. How does GPT-5.5 stack up versus 5.4 and versus Opus 4.7 and other models? We're going to go into that in detail. So, let's start with coding. The headline is a senior engineer benchmark. As I said, GPT-5.5 scores about 30 points higher than Opus 4.7 on the senior engineer benchmark, but it only does that when it is used with a plan written by Opus 4.7. I think this tells us a lot about the psychology of this model, what it's good at, what it's not good at, and when to use it in your workflow. To understand this a little more, I want to talk to you about what this senior engineer benchmark is, the SE bench. It's a benchmark I invented, and the benchmark basically gives the model a vibe-coded slop code base. It's a real code base from an app that I built called Proof, and it says to the model, "Hey, like this is vibe-coded slop. How would you rewrite this from first principles?" Basically, is it able to rewrite a code base doing it in a clean first principle way where there is conceptual clarity and it looks like a senior engineer did it? And the gold standard for this benchmark is actual code written by senior engineers. I had two different engineers do their own rewrites of this code base. So, we get to compare what these models do versus real human engineers. And first of all, this benchmark is not saturated. There's still a lot of room on the frontier. The best human engineers get around 80 and 90 points on this benchmark out of 100. The best score we've ever seen is GPT-5.5. It got about a 62.5 using a Claude Opus 4.7 plan, and it got in the low to mid-40s without, which is still much better than Opus 4.7, which has got about a 33. If you do some prompting, you can get 5.5 to make its own plan good enough that its performance on this benchmark starts to approach its performance when it uses an Opus 4.7 plan. But, it only gets to like the low to mid-50s. So, for my money, Opus 4.7 is just a better planner than 5.5. What this model can do that other models don't seem to be able to is it's able to identify the underlying core principles or invariants that need to be true in the code base, in the plan that it writes. And then when it starts writing code against that plan, it doesn't get as distracted by the existing code base. It doesn't go into patch mode and start like patching little holes. It has the assertiveness, the boldness, the agency to actually go and delete a bunch of files and and really start from scratch and then carry through the idea it has from start to finish over several hours. It's not perfect yet. It's far from perfect. Like I said, it's still about 30 points away from senior engineers on this benchmark, but it is a step change in terms of how good it is versus other models. What happens with Opus 4.7 is Opus 4.7 actually produces really good plan. It's very good at writing something that's conceptually clear and clean, something that feels contract-driven and gives the model a sense for if this rewrite is good enough, this big file will only be 100 lines. That kind of thing, that level of detail. That's what drives GPT-5.5's performance. That's what makes it so good is if you give it a plan like that, it just actually does it. It has the confidence to do it. But, Opus, if you give it its own plan, it's really beautiful, well-written plan. It says, "Oh, this is like too much effort. Like uh I'm just going to pick off a small little slice." And then it just starts to patch around the issue rather than actually do the rewrite as asked for. GPT-5.4, the old OpenAI model that's being replaced today, does the same thing. It does it better than Opus 4.7, but it still does the same thing. But, GPT-5.5 on extra high reasoning just has that little bit of extra oomph that allows it to actually go in and execute. Okay, well, what is it about the Opus 4.7 plan that makes GPT-5.5 better? And I think that's really interesting because it says something about what makes a good use of 5.5. If you look at the plan that Opus 4.7 wrote for this benchmark versus the plan that GPT-5.5 wrote, it has all this all the right concepts, but it's pretty long and it doesn't have a lot of the really specific programmary contract style. Okay, here's what good looks like, here's what you should delete, here's how many files should be left, all that kind of stuff. It's really actually just really written well for humans, and I think they've done a lot of tuning on 5.5 to make it good for humans. And if you look at 4.7, part of the reason people don't like it is because it's kind of terse and it's kind of it feels a little bit more robotic than you're used to with Anthropic models. But, it turns out that terseness, that exactness, that contract style of plan is really good for 5.5. So, if you want to get the most out of 5.5, using Opus 4.7 as the planner or making sure you prompt 5.5 with a lot of exact detail is going to be the thing that draws out this boldness, this agency, this ability to carry a big plan through from start to finish. It's really impressive. Naveen, who's the GM at Every who runs Monologue, tested it on building a to-do app called Dayline, and he was really impressed by how it was able to build a native iOS and Mac app that was beautiful, works really well, and it can just sort of turn through a bunch of features in a plan until it's done. It's really, really good at that. Especially if the plan is well specified. Naveen also used it to build a release of Monologue on the app that he runs, and he used about 900 million tokens with GPT-5.5 in pre-release as part of his testing. He thinks it's his favorite model for everything. He said he was only able to hit the deadline that he needed to in order to get a new feature out for Monologue by using this model. He means an incredible senior engineer, and if he likes this model, it's worth paying attention to. There are some things that it's not as good at. Kieran Classen, the general manager at Cora, which is our AI email agent at Every, tested the model on his LFT bench, which essentially replicates his real development process to see how model does on more product-forward engineering tasks, where you're not necessarily refactoring a whole code base, but you're building a feature um in in something that includes a lot of front end and design and product thinking. And he found that for for these types of tasks in the LFT bench, 5.5 did pretty well, but Opus 4.7 design-forward tasks. It just has a better aesthetic sense than 5.5. And Mike Keller, who runs technology consulting at Every, had a similar feeling about it in his benchmarks, where getting it to vibe code relatively complex app from scratch with a under-specified plan, it just didn't do as well as Opus 4.7, which is ripped through the entire task. So, whether you're vibe coding or you're trying to do some senior engineer type stuff, a better specified plan is going to be the way to get the most out of this model. One other thing that we noticed is that GPT-5.5 is really good at writing TypeScript and writing Swift, but Kieran, who's a huge Ruby stan, thinks it doesn't write good Ruby. So, if you're doing a Rails project, you may not be happy with the quality of the Ruby that gets written, but if you're in TypeScript or you're in Swift, you're going to be pretty happy. Okay. Now onto writing. It's got a little bit less personality than Opus, especially the older Opus models like 4.6, but it's actually really good for doing business writing. So, I used it to write our investor update, and it basically one-shotted an update that was like close to being ready to send. Katie Parrot, who's a staff writer at Every, has been using Claude models for probably a a year or two now almost exclusively for writing. This is the first GPT model in a long time that she has started to use for writing tasks over Opus or Sonnet. In particular, Katie and Mike both really liked it for its voice replication. It was pretty good at at mimicking a style without overdoing it. It's a little bit more restrained, which I think is why it's good at business writing, and that makes it a little bit more subtle as a writer. Okay. Now onto knowledge work. This is one of those things that OpenAI was so behind on even 3 months ago or 6 months ago, and they have just rapidly iterated to the point where the Claude X desktop app is great for any kind of knowledge work, and using it with GPT 5.5 is best-in-class agent experience you can have on your desktop. For one, it's it's really fast. It's really powerful. It can use any of the apps on your computer. It's good at browsing the web. It's good at doing things like writing dashboards or doing complex data analysis. And again, it's really fast. Um in in all of these tests, I haven't talked about this that much, but in all of these tests, we were pretty shocked at how slow Opus 4.7 feels in comparison. You can really tell the hardware advantage that Open AI is working with right now. You can feel it. One thing about this model that I kind of ding it for versus Opus is some of the training that's being done to make it feel digestible has come at the cost of some of its eye for detail and knowledge work. If you're using this model for tasks that require like really sharp insights, I would definitely consider using Opus 4.7 over 5.5 for that. For example, even the senior engineer benchmark, like grading the model trajectory, I just trusted 4.7 more than I trusted 5.5, even if I basically want to use 5.5 more as my daily driver. It's a great daily driver model. So, that's basically the gist of 5.5. I feel like it's Christmas every couple weeks around here cuz we keep getting access to new stuff, and it keeps being awesome. And I feel like my powers and everybody else on the team, like all of our powers are just increasing day by day. You should definitely give this model a try. It's the most usable way right now to get the power of a frontier model in a user-friendly, human-friendly, collaborative package. And that's a big achievement. Until I've really switched my usage where I use Claude a decent amount on mobile, and I like the mobile app, but for anything on my computer, anything agentic, I'm pretty much in Claude X for everything. And I got to say, I like it in Open Claude. If you're an Opus stan, the harness is really still built to work better in Opus, but they're releasing stuff every day that makes it pretty good in Open Claude, and it's been one of those stable experiences for me over the last couple weeks where it's not forgetting stuff as much. It's not doing dumb stuff as much. It still does it a little bit. But because it's free under my ChatGPT subscription, I'm okay with a little bit of a trade-off in power, and I think that they'll fix that hopefully soon. So, if you didn't like 5.4 in Open Claude, I recommend giving it a shot in 5.5. So, that's it, folks. That's our report from the frontier. If you liked this, you should like and subscribe, but more importantly, you should go to every.to. We have a couple thousand words on this model with everything that we tested and everything that we found. We also have a suite of apps for subscribers, stuff that we build ourselves that help you work better with AI, stuff that helps you write your emails, that helps you organize your files, that helps you write better. We've got a ton of good stuff in the Every subscription. If you try this model, we'd love to know what you think. So, please leave a comment, and remember, stay hydrated. See you next time on the moon.