---
schema: 1
type: video
title: 'Backtest Your Agent Against Yesterday''s Web — Ryan Ahern, Exa'
description: >-
  How do you know whether your agent would have made the right call? Ryan Ahern
  of Exa shows how to use version-controlled search to see the web exactly as it
  was on a past date, then backtest an agent's predictions against what actually
  happened. The hands-on example predicts company acquisitions from signals like
  leadership turnover, then tunes the system without leaking future information.
  Chapters: - 00:00 Why search matters for agents - 04:15 Exa's search benchmark
  - 06:17 Versioning the whole web - 10:34 Avoiding look-ahead bias - 14:43
  Backtesting and tuning a predictor - 18:49 Hands-on: predicting acquisitions -
  22:54 Results and overfitting Ryan Ahern — https://exa.ai Join us in SF for
  the second annual AIE CODE summit: https://aie.codes our last flagship summit
  of 2026. Applications live now!
canonical: 'https://liquidgold.cc/watch/L4dUFimVDKU'
markdown: 'https://liquidgold.cc/watch/L4dUFimVDKU.md'
llms: 'https://liquidgold.cc/watch/L4dUFimVDKU/llms.txt'
parent: 'https://liquidgold.cc/channel/aidotengineer.md'
channel: aidotengineer
updated: '2026-10-23T22:22:57.129Z'
language: null
children: []
videoId: L4dUFimVDKU
durationSeconds: 1602
publishedAt: '2026-10-19T07:00:39.000Z'
searchChunks:
  - ordinal: 0
    text: null
    startSeconds: 22
    endSeconds: 264
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=22'
    heldReason: rights
  - ordinal: 1
    text: null
    startSeconds: 22
    endSeconds: 264
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=22'
    heldReason: rights
  - ordinal: 2
    text: null
    startSeconds: 264
    endSeconds: 275
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=264'
    heldReason: rights
  - ordinal: 3
    text: null
    startSeconds: 275
    endSeconds: 384
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=275'
    heldReason: rights
  - ordinal: 4
    text: null
    startSeconds: 275
    endSeconds: 384
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=275'
    heldReason: rights
  - ordinal: 5
    text: null
    startSeconds: 395
    endSeconds: 635
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=395'
    heldReason: rights
  - ordinal: 6
    text: null
    startSeconds: 395
    endSeconds: 635
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=395'
    heldReason: rights
  - ordinal: 7
    text: null
    startSeconds: 395
    endSeconds: 635
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=395'
    heldReason: rights
  - ordinal: 8
    text: null
    startSeconds: 635
    endSeconds: 646
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=635'
    heldReason: rights
  - ordinal: 9
    text: null
    startSeconds: 646
    endSeconds: 888
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=646'
    heldReason: rights
  - ordinal: 10
    text: null
    startSeconds: 888
    endSeconds: 898
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=888'
    heldReason: rights
  - ordinal: 11
    text: null
    startSeconds: 898
    endSeconds: 1139
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=898'
    heldReason: rights
  - ordinal: 12
    text: null
    startSeconds: 898
    endSeconds: 1139
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=898'
    heldReason: rights
  - ordinal: 13
    text: null
    startSeconds: 1139
    endSeconds: 1151
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=1139'
    heldReason: rights
  - ordinal: 14
    text: null
    startSeconds: 1151
    endSeconds: 1384
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=1151'
    heldReason: rights
  - ordinal: 15
    text: null
    startSeconds: 1418
    endSeconds: 1602
    watch: 'https://liquidgold.cc/watch/L4dUFimVDKU?t=1418'
    heldReason: rights
---


# Backtest Your Agent Against Yesterday's Web — Ryan Ahern, Exa

How do you know whether your agent would have made the right call? Ryan Ahern of Exa shows how to use version-controlled search to see the web exactly as it was on a past date, then backtest an agent's predictions against what actually happened. The hands-on example predicts company acquisitions from signals like leadership turnover, then tunes the system without leaking future information. Chapters: - 00:00 Why search matters for agents - 04:15 Exa's search benchmark - 06:17 Versioning the whole web - 10:34 Avoiding look-ahead bias - 14:43 Backtesting and tuning a predictor - 18:49 Hands-on: predicting acquisitions - 22:54 Results and overfitting Ryan Ahern — https://exa.ai Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!

Canonical: https://liquidgold.cc/watch/L4dUFimVDKU · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md)

> Search chunk text is held until public-source provenance is verified.

## Document chunks

---

#### Chunk 0: Document excerpt (0:12 to 4:24)

> Held: transcript rights do not permit this chunk's text.

---

#### Chunk 1: Document excerpt (4:24 to 6:24)

> Held: transcript rights do not permit this chunk's text.

---

#### Chunk 2: Document excerpt (6:24 to 10:35)

> Held: transcript rights do not permit this chunk's text.

---

#### Chunk 3: Document excerpt (10:35 to 14:48)

> Held: transcript rights do not permit this chunk's text.

---

#### Chunk 4: Document excerpt (14:48 to 18:59)

> Held: transcript rights do not permit this chunk's text.

---

#### Chunk 5: Document excerpt (18:59 to 23:04)

> Held: transcript rights do not permit this chunk's text.

---

#### Chunk 6: Document excerpt (23:04 to 26:42)

> Held: transcript rights do not permit this chunk's text.