---
schema: 1
type: video
title: 'What Do Models Still Suck At? - Peter Gostev, Arena.ai, BullshitBench'
description: >-
  What type of real world model responses do users still hate? We get to see
  millions of user's prompts - and we let users 'dislike both' on the Arena.
  We'll show you trends and examples of the tasks that LLMs still suck at
  despite the relentless hillclimbing. Speaker info: - https://x.com/petergostev
  - https://www.linkedin.com/in/peter-gostev/ --- Watch along with the corrected
  transcript, summary, timestamps and resources on this talk's official AIE
  page: https://ai.engineer/talks/R7A8rX-09Zw-what-do-models-still-suck-at Join
  our upcoming AI Engineer conferences (see https://ai.engineer landing page): -
  AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and
  engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference
  returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS
  2026 in Sydney ---
canonical: 'https://liquidgold.cc/watch/R7A8rX-09Zw'
markdown: 'https://liquidgold.cc/watch/R7A8rX-09Zw.md'
llms: 'https://liquidgold.cc/watch/R7A8rX-09Zw/llms.txt'
parent: 'https://liquidgold.cc/channel/aidotengineer.md'
channel: aidotengineer
updated: '2026-10-24T14:56:57.244Z'
language: null
children: []
videoId: R7A8rX-09Zw
durationSeconds: 1224
publishedAt: '2026-04-24T14:30:06.000Z'
searchChunks:
  - ordinal: 0
    text: null
    startSeconds: 26
    endSeconds: 120
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=26'
    heldReason: rights
  - ordinal: 1
    text: null
    startSeconds: 130
    endSeconds: 251
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=130'
    heldReason: rights
  - ordinal: 2
    text: null
    startSeconds: 130
    endSeconds: 251
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=130'
    heldReason: rights
  - ordinal: 3
    text: null
    startSeconds: 130
    endSeconds: 251
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=130'
    heldReason: rights
  - ordinal: 4
    text: null
    startSeconds: 262
    endSeconds: 373
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=262'
    heldReason: rights
  - ordinal: 5
    text: null
    startSeconds: 384
    endSeconds: 476
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=384'
    heldReason: rights
  - ordinal: 6
    text: null
    startSeconds: 384
    endSeconds: 476
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=384'
    heldReason: rights
  - ordinal: 7
    text: null
    startSeconds: 488
    endSeconds: 623
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=488'
    heldReason: rights
  - ordinal: 8
    text: null
    startSeconds: 488
    endSeconds: 623
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=488'
    heldReason: rights
  - ordinal: 9
    text: null
    startSeconds: 488
    endSeconds: 623
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=488'
    heldReason: rights
  - ordinal: 10
    text: null
    startSeconds: 634
    endSeconds: 737
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=634'
    heldReason: rights
  - ordinal: 11
    text: null
    startSeconds: 749
    endSeconds: 859
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=749'
    heldReason: rights
  - ordinal: 12
    text: null
    startSeconds: 871
    endSeconds: 993
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=871'
    heldReason: rights
  - ordinal: 13
    text: null
    startSeconds: 1003
    endSeconds: 1107
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=1003'
    heldReason: rights
  - ordinal: 14
    text: null
    startSeconds: 1003
    endSeconds: 1107
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=1003'
    heldReason: rights
  - ordinal: 15
    text: null
    startSeconds: 1107
    endSeconds: 1119
    watch: 'https://liquidgold.cc/watch/R7A8rX-09Zw?t=1107'
    heldReason: rights
---


# What Do Models Still Suck At? - Peter Gostev, Arena.ai, BullshitBench

What type of real world model responses do users still hate? We get to see millions of user's prompts - and we let users 'dislike both' on the Arena. We'll show you trends and examples of the tasks that LLMs still suck at despite the relentless hillclimbing. Speaker info: - https://x.com/petergostev - https://www.linkedin.com/in/peter-gostev/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/R7A8rX-09Zw-what-do-models-still-suck-at Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney ---

Canonical: https://liquidgold.cc/watch/R7A8rX-09Zw · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md)

> Search chunk text is held until public-source provenance is verified.

## Document chunks

> Missing: no stored document chunks are available.