---
schema: 1
type: video
title: 'How agent o11y differs from traditional o11y — Phil Hetzel, Braintrust'
description: >-
  Traditional observability answers one question: is the system up? Phil Hetzel
  from Braintrust argues that question is not the right one for agents. An
  individual agent trace can exceed a gigabyte. A single span can hit 20
  megabytes. The data is semistructured, packed with unstructured text, and
  still arrives in real time. None of the systems built for uptime monitoring
  were designed to ingest, index, and actually use that. Braintrust built a
  custom database from scratch for this problem: a write ahead log for instant
  visibility, analytical indexes for fast filtering, and a forked version of
  Tantivy (a Rust based full text search library similar to Apache Lucene) so an
  engineer can query every trace that mentioned a specific word. The other
  difference is who does this work: clinicians, lawyers, and wealth advisers now
  open traces directly to grade whether an agent responded correctly, and their
  written justifications become the training signal for automated scoring
  functions. The human annotations surface the failure modes. The scoring
  functions scale them. Speaker info: -
  https://www.linkedin.com/in/philliphetzel/
canonical: 'https://liquidgold.cc/watch/XBaznoTRDFI'
markdown: 'https://liquidgold.cc/watch/XBaznoTRDFI.md'
llms: 'https://liquidgold.cc/watch/XBaznoTRDFI/llms.txt'
parent: 'https://liquidgold.cc/channel/aidotengineer.md'
channel: aidotengineer
updated: '2026-10-24T13:05:18.453Z'
language: null
children: []
videoId: XBaznoTRDFI
durationSeconds: 1243
publishedAt: '2026-05-28T23:00:06.000Z'
searchChunks:
  - ordinal: 0
    text: null
    startSeconds: 237
    endSeconds: 371
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=237'
    heldReason: rights
  - ordinal: 1
    text: null
    startSeconds: 117
    endSeconds: 226
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=117'
    heldReason: rights
  - ordinal: 2
    text: null
    startSeconds: 237
    endSeconds: 371
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=237'
    heldReason: rights
  - ordinal: 3
    text: null
    startSeconds: 384
    endSeconds: 462
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=384'
    heldReason: rights
  - ordinal: 4
    text: null
    startSeconds: 384
    endSeconds: 462
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=384'
    heldReason: rights
  - ordinal: 5
    text: null
    startSeconds: 462
    endSeconds: 473
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=462'
    heldReason: rights
  - ordinal: 6
    text: null
    startSeconds: 473
    endSeconds: 583
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=473'
    heldReason: rights
  - ordinal: 7
    text: null
    startSeconds: 593
    endSeconds: 701
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=593'
    heldReason: rights
  - ordinal: 8
    text: null
    startSeconds: 593
    endSeconds: 701
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=593'
    heldReason: rights
  - ordinal: 9
    text: null
    startSeconds: 712
    endSeconds: 824
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=712'
    heldReason: rights
  - ordinal: 10
    text: null
    startSeconds: 712
    endSeconds: 824
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=712'
    heldReason: rights
  - ordinal: 11
    text: null
    startSeconds: 836
    endSeconds: 922
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=836'
    heldReason: rights
  - ordinal: 12
    text: null
    startSeconds: 932
    endSeconds: 1042
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=932'
    heldReason: rights
  - ordinal: 13
    text: null
    startSeconds: 932
    endSeconds: 1042
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=932'
    heldReason: rights
  - ordinal: 14
    text: null
    startSeconds: 1052
    endSeconds: 1180
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=1052'
    heldReason: rights
  - ordinal: 15
    text: null
    startSeconds: 1052
    endSeconds: 1180
    watch: 'https://liquidgold.cc/watch/XBaznoTRDFI?t=1052'
    heldReason: rights
---


# How agent o11y differs from traditional o11y — Phil Hetzel, Braintrust

Traditional observability answers one question: is the system up? Phil Hetzel from Braintrust argues that question is not the right one for agents. An individual agent trace can exceed a gigabyte. A single span can hit 20 megabytes. The data is semistructured, packed with unstructured text, and still arrives in real time. None of the systems built for uptime monitoring were designed to ingest, index, and actually use that. Braintrust built a custom database from scratch for this problem: a write ahead log for instant visibility, analytical indexes for fast filtering, and a forked version of Tantivy (a Rust based full text search library similar to Apache Lucene) so an engineer can query every trace that mentioned a specific word. The other difference is who does this work: clinicians, lawyers, and wealth advisers now open traces directly to grade whether an agent responded correctly, and their written justifications become the training signal for automated scoring functions. The human annotations surface the failure modes. The scoring functions scale them. Speaker info: - https://www.linkedin.com/in/philliphetzel/

Canonical: https://liquidgold.cc/watch/XBaznoTRDFI · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md)

> Search chunk text is held until public-source provenance is verified.

## Document chunks

> Missing: no stored document chunks are available.