--- schema: 1 type: topic title: Versioned web search and historical data for agents description: >- A synthesis of the channel's material about Versioned web search and historical data for agents. canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/topic/versioned-web-search-and-historical-data-for-agents-nedwebsearch markdown: >- https://liquidgold.cc/channel/aidotengineer/explore/topic/versioned-web-search-and-historical-data-for-agents-nedwebsearch.md llms: >- https://liquidgold.cc/channel/aidotengineer/explore/topic/versioned-web-search-and-historical-data-for-agents-nedwebsearch/llms.txt parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:12:35.346Z' language: null children: [] topicSlug: versioned-web-search-and-historical-data-for-agents-nedwebsearch searchChunks: [] --- # Versioned web search and historical data for agents A synthesis of the channel's material about Versioned web search and historical data for agents. Canonical: https://liquidgold.cc/channel/aidotengineer/explore/topic/versioned-web-search-and-historical-data-for-agents-nedwebsearch Parent: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) ## Agents search differently — and that raises the bar for evidence Before a single query runs, note who is now doing the searching. Ryan Ahern of Exa observes that many users simply ask agents like ChatGPT or Claude to search for them, and agents search nothing like humans do. They describe what they want in great detail, then run many searches to understand a space before responding. As Ahern puts it, agents tend to do a pretty good job at understanding how to search because they are allowed to try multiple, multiple times — they kind of don't get tired. A human gives up after two bad results; an agent will happily run twenty more queries to satisfy its objective, which is what makes it an effective search broker. That behavior raises the stakes on the search layer underneath. Exa, a lab hyper-focused on how agents search, builds around three tenets: **quality**, **cost**, and **latency**. Quality matters because fewer tool calls means more cost-effective inference and a higher chance the agent succeeds on its trajectory — which is why Exa released a benchmark that assesses search tools separately from inference. Cost should be judged on the total cost of an agent completing its task, not a single tool call; Exa's dynamic highlights pick exact snippets across all search results, delivering higher-quality results at 95% fewer tokens than full page text. Owning its own crawling infrastructure and embedding models keeps latency low, and Exa reports that its recently released ExaAgent Ultra is state-of-the-art on benchmarks for very long-running deep research — a claim to weigh as Exa's own benchmark, but one that matters for the use case this chapter is about: deep research over historical web state. ## Snapshot and version search: retrieving the web as it was The mechanism is deceptively simple. ExaSnapshot, by Exa's account, offers **400 billion plus snapshots of the web**, searchable by date. While indexing, Exa checks URLs for freshness and captures a version at each check — so you can retrieve a URL as it looked at, or before, a given date. Version search then asks the natural follow-up question: what would this search have returned if you had executed it at a prior point? You get only results that would have been served at that point or before. The practical consequence is that an agent can cite not just a page, but a _version_ of a page — pinned to a moment in time. One limit is worth stating plainly: because a version is captured at each freshness check, the earliest retrievable state of any page is bounded by when Exa's crawler last saw it. A page Exa never indexed before your target date simply has no earlier version to return. Ahern singles out financial services as especially interested in this capability — hedge funds, asset managers, sovereign wealth funds, and VCs already use Exa for alpha, risk models, and finding hot startups that are not yet in structured datasets. Historical search extends that from _what is on the web_ to _what was on the web when it mattered_. ## Backtesting without seeing the future The clearest way to see the value is the example Ahern walks through. In early October, reporting put OpenAI's annualized run rate around 70 billion dollars; later reporting put net annualized revenue closer to 50 billion, because the gross figure included vendor pass-throughs from cloud providers. With version search you can ask: what information existed on the internet _before_ the market figured this out? As Ahern asks, was there an SEC filing that would have revealed this thing in an accounting footnote — or less reputable reporting that surfaced the discrepancy early? Then you can backtest whether your model or agent could have found it, checking generalizability against sample sets and a holdout rather than one lucky example. The same move fixes a subtler failure mode: contamination. Suppose you are building a model to predict company acquisitions. A live search today reveals every announced deal, because the news was released everywhere — any training data you gather now is contaminated by the outcome you are trying to predict. The fix is to search snapshots taken **1 day, 7 days, and 1 month before** each announcement, build your company set from those snapshots, and test whether your model reliably predicted the outcome. Snapshot access lets you control the web as a feature, so future information cannot leak into the model. ## Three integration archetypes and the overfitting trap Ahern sketches three ways teams plug versioned search into their systems, and each carries its own backtest. - **Traditional models**: gather unstructured data from the versioned web — software prices, sentiment, terms and conditions — into structured signals, enriching datasets you could not reliably access before. - **Agentic features on existing models**: an adverse media scan or an equity-research agent gathering features like revenue or headcount; backtest it by asking what the scan would have found 1 or 4 months ago. - **Fully agentic prediction**: an agent managing the whole simulation, where snapshot access constrains the web feature itself so nothing from the future leaks in. The trap is overfitting, and the workshop Ahern describes makes it vivid. About 100 people worked on predicting company acquisitions. Strong features included whether an investor or significant shareholder had publicly called for a sale in the past year, and whether the share price sat at least 60% below its 5-year high. And then there was the feature that reduced to: the company is named Dominion. It scored well; it also generalized to nothing, which is obvious to anyone familiar with backtesting and easy to miss in an agentic pipeline. Snapshot-based evals are what make this detectable — you can set up tests for overfitting and generalization offline, something Ahern notes would be very, very hard to do if you are gated by testing live in production. Semantic search supports complex feature descriptions, such as adverse media against reputable VPs at VC funds who appeared on a podcast, which you can then backtest the same way. The payoff is verification: financial services customers can check that search genuinely adds value against their existing processes, rather than just believing it will be better. ## Put it in your agent The integration recipe falls out of the examples. Pick the decision your agent supports, define the web-derived features that would inform it, then run every search against a dated snapshot at or before the decision's cutoff. Cite the snapshot date alongside the source so the evidence is auditable. Reserve live search for genuinely forward-looking work, and use the gap between snapshot results and live results as a diagnostic for leakage. - **ExaSnapshot** provides 400 billion plus dated web snapshots; **version search** returns only results that would have been served at or before a chosen date. - Snapshot searches taken 1 day, 7 days, and 1 month before announcements keep future information out of training data. - Backtest across three archetypes: unstructured-to-structured feature extraction, agentic features on existing models, and fully agentic prediction. - Guard against overfitting with holdouts and offline snapshot evals — a workshop feature that amounted to the company being named Dominion scored well and generalized to nothing. - Judge search tools by the total cost of the agent's task, not a single call; Exa's dynamic highlights cut tokens by 95% versus full page text. An agent wired this way can retrieve and cite how the web looked at a specific point in time — which means its predictions rest on what was knowable then, not what everyone knows now. That is the capability this chapter promised, and the backtest is how you prove you have it. ## Source videos > Missing: no public cited source videos are available.