--- schema: 1 type: video title: 'Jev CEO: I made ChatGPT, now I''m building what''s next' description: >- Diogo Almeida, a GPT-4 co author formerly at OpenAI and now CEO of TypeSafeAI, creators of Jev, argues that RLHF made models that are extraordinary at pleasing the human in the loop, and that is exactly the problem. Optimizing for human preference optimizes for engagement and for overpromising, the same pressure that makes a model confidently agree that a fart audio file is a symphony. That produces two camps: one where models act as assistants with a human catching mistakes, where RLHF shines, and one where they operate autonomously with real stakes, where the same instinct to please quietly becomes a liability. So what comes next is not the Claude Code era but a shift in what you optimize. Almeida frames it through Sutton's bitter lesson: the task matters more than the data, and reinforcement learning with verifiable rewards points the model at real automation instead of human approval. He is careful that pre trained models are already incredibly capable and that the trap is bolting preference optimization on top, which teaches confidence and drops modes. The through line is that assistance and automation pull in different directions in optimization space, and the field is only starting to say plainly which one it is building. Speaker info: http://ai.engineer/speakers/diogo-almeida - https://x.com/CompleteSkeptic - https://www.linkedin.com/in/diogomda/ - https://typesafe.ai/ Timestamps: 0:00 - Not the Claude Code era 1:40 - The state of the field 3:14 - Two camps: assistance and autonomy 4:31 - Why models please the human in the loop 6:37 - How RLHF actually works 7:31 - Preference versus what's true 8:10 - When the consequences get real 8:47 - So what's next 9:35 - Assistance is not automation 14:31 - Is pre-training the problem? 15:43 - RLVR and Sutton's bitter lesson --- Join AI Engineer CODE in San Francisco, November 10–12: our last flagship conference of 2026. This selective, application-only gathering brings together top agentic engineers to explore software factories, system-one models and large-scale rewrites. https://ai.engineer/code/2026 Explore this talk's slides, transcript, key takeaways and resources on its official AI Engineer page: https://ai.engineer/talks/cJ0EOzey--o-what-s-next-after-rlhf --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/jev-ceo-i-made-chatgpt-now-i-m-building-what-s-next-d7d012b02c3e markdown: >- https://liquidgold.cc/channel/aidotengineer/video/jev-ceo-i-made-chatgpt-now-i-m-building-what-s-next-d7d012b02c3e.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T06:29:06.413Z' language: null children: [] videoId: cJ0EOzey--o durationSeconds: 1085 publishedAt: '2026-07-31T23:30:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 24 endSeconds: 100 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=24' heldReason: rights - ordinal: 1 text: null startSeconds: 110 endSeconds: 200 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=110' heldReason: rights - ordinal: 2 text: null startSeconds: 211 endSeconds: 277 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=211' heldReason: rights - ordinal: 3 text: null startSeconds: 277 endSeconds: 289 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=277' heldReason: rights - ordinal: 4 text: null startSeconds: 289 endSeconds: 404 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=289' heldReason: rights - ordinal: 5 text: null startSeconds: 289 endSeconds: 404 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=289' heldReason: rights - ordinal: 6 text: null startSeconds: 416 endSeconds: 458 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=416' heldReason: rights - ordinal: 7 text: null startSeconds: 469 endSeconds: 500 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=469' heldReason: rights - ordinal: 8 text: null startSeconds: 510 endSeconds: 534 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=510' heldReason: rights - ordinal: 9 text: null startSeconds: 546 endSeconds: 578 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=546' heldReason: rights - ordinal: 10 text: null startSeconds: 546 endSeconds: 578 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=546' heldReason: rights - ordinal: 11 text: null startSeconds: 591 endSeconds: 672 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=591' heldReason: rights - ordinal: 12 text: null startSeconds: 672 endSeconds: 682 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=672' heldReason: rights - ordinal: 13 text: null startSeconds: 682 endSeconds: 804 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=682' heldReason: rights - ordinal: 14 text: null startSeconds: 682 endSeconds: 804 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=682' heldReason: rights - ordinal: 15 text: null startSeconds: 880 endSeconds: 892 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=880' heldReason: rights - ordinal: 16 text: null startSeconds: 892 endSeconds: 948 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=892' heldReason: rights - ordinal: 17 text: null startSeconds: 948 endSeconds: 959 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=948' heldReason: rights - ordinal: 18 text: null startSeconds: 959 endSeconds: 1085 watch: 'https://liquidgold.cc/watch/cJ0EOzey--o?t=959' heldReason: rights --- # Jev CEO: I made ChatGPT, now I'm building what's next Diogo Almeida, a GPT-4 co author formerly at OpenAI and now CEO of TypeSafeAI, creators of Jev, argues that RLHF made models that are extraordinary at pleasing the human in the loop, and that is exactly the problem. Optimizing for human preference optimizes for engagement and for overpromising, the same pressure that makes a model confidently agree that a fart audio file is a symphony. That produces two camps: one where models act as assistants with a human catching mistakes, where RLHF shines, and one where they operate autonomously with real stakes, where the same instinct to please quietly becomes a liability. So what comes next is not the Claude Code era but a shift in what you optimize. Almeida frames it through Sutton's bitter lesson: the task matters more than the data, and reinforcement learning with verifiable rewards points the model at real automation instead of human approval. He is careful that pre trained models are already incredibly capable and that the trap is bolting preference optimization on top, which teaches confidence and drops modes. The through line is that assistance and automation pull in different directions in optimization space, and the field is only starting to say plainly which one it is building. Speaker info: http://ai.engineer/speakers/diogo-almeida - https://x.com/CompleteSkeptic - https://www.linkedin.com/in/diogomda/ - https://typesafe.ai/ Timestamps: 0:00 - Not the Claude Code era 1:40 - The state of the field 3:14 - Two camps: assistance and autonomy 4:31 - Why models please the human in the loop 6:37 - How RLHF actually works 7:31 - Preference versus what's true 8:10 - When the consequences get real 8:47 - So what's next 9:35 - Assistance is not automation 14:31 - Is pre-training the problem? 15:43 - RLVR and Sutton's bitter lesson --- Join AI Engineer CODE in San Francisco, November 10–12: our last flagship conference of 2026. This selective, application-only gathering brings together top agentic engineers to explore software factories, system-one models and large-scale rewrites. https://ai.engineer/code/2026 Explore this talk's slides, transcript, key takeaways and resources on its official AI Engineer page: https://ai.engineer/talks/cJ0EOzey--o-what-s-next-after-rlhf --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/jev-ceo-i-made-chatgpt-now-i-m-building-what-s-next-d7d012b02c3e · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.