--- schema: 1 type: video title: 'Computer-use models will agentify the web, not APIs — Dhruv Batra, Yutori' description: >- To learn what a US school district is buying, you file a Freedom of Information Act request. Someone scans the email you sent, puts the scan on Google Drive, and attaches the relevant PDFs. Dhruv Batra's question is whether anyone seriously expects that office to publish an MCP server. He grants the popular claim that agents will drive most of the action on the web, then rejects its usual next step, that the web will meet them with APIs. The head of the distribution might. The long tail, some 200 million active sites where infrastructure changes over decades, will not. Reading the HTML instead does not save you, because much of what you see was never written down anywhere. A basketball score is missing from the page that first loads and arrives later as JSON. A product page contains no text reading sold out, only a quantity of zero and a script that grays the option out. State is calculated and rendered rather than stored, which makes the browser closer to a game engine than a document and makes pixels the source of truth. He calls this the bitter lesson for web agents: scaffolding built per site does not generalize, and the general solution is the one the web was actually built for. Their Navigator model runs screenshot in and clicks out, now writes JavaScript when that is quicker, and checks the result on screen. It misses 8 of 300 trajectories on a benchmark he thinks should be retired. Speaker info: - https://x.com/DhruvBatra_ - https://www.linkedin.com/in/dhruv-batra-dbatra/ - https://dhruvbatra.com Timestamps: 0:00 - The argument, and the part of it that is wrong 3:23 - Restaurant menus on easy, medium, and hard mode 5:31 - School district procurement, up to a FOIA request 7:38 - 200 million active sites that change slowly 8:42 - Why reading the HTML does not rescue it 10:14 - Sold out is not text, it is a rendered zero 11:33 - The browser is a rendering engine, pixels are the truth 12:53 - Navigator, and writing JavaScript when that is faster 15:51 - Are computer use models actually good enough 17:20 - Latency and cost per task 18:28 - Another layer of mess that we will call an API --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/Ki980nV0__0-computer-use-models-will-agentify-the-web-not-apis Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/Ki980nV0__0' markdown: 'https://liquidgold.cc/watch/Ki980nV0__0.md' llms: 'https://liquidgold.cc/watch/Ki980nV0__0/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T05:24:37.027Z' language: null children: [] videoId: Ki980nV0__0 durationSeconds: 1260 publishedAt: '2026-08-14T14:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 23 endSeconds: 213 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=23' heldReason: rights - ordinal: 1 text: null startSeconds: 23 endSeconds: 213 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=23' heldReason: rights - ordinal: 2 text: null startSeconds: 23 endSeconds: 213 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=23' heldReason: rights - ordinal: 3 text: null startSeconds: 223 endSeconds: 335 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=223' heldReason: rights - ordinal: 4 text: null startSeconds: 346 endSeconds: 465 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=346' heldReason: rights - ordinal: 5 text: null startSeconds: 475 endSeconds: 531 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=475' heldReason: rights - ordinal: 6 text: null startSeconds: 541 endSeconds: 621 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=541' heldReason: rights - ordinal: 7 text: null startSeconds: 632 endSeconds: 693 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=632' heldReason: rights - ordinal: 8 text: null startSeconds: 704 endSeconds: 782 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=704' heldReason: rights - ordinal: 9 text: null startSeconds: 704 endSeconds: 782 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=704' heldReason: rights - ordinal: 10 text: null startSeconds: 794 endSeconds: 960 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=794' heldReason: rights - ordinal: 11 text: null startSeconds: 794 endSeconds: 960 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=794' heldReason: rights - ordinal: 12 text: null startSeconds: 794 endSeconds: 960 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=794' heldReason: rights - ordinal: 13 text: null startSeconds: 971 endSeconds: 1040 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=971' heldReason: rights - ordinal: 14 text: null startSeconds: 1051 endSeconds: 1116 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=1051' heldReason: rights - ordinal: 15 text: null startSeconds: 1127 endSeconds: 1260 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=1127' heldReason: rights - ordinal: 16 text: null startSeconds: 1127 endSeconds: 1260 watch: 'https://liquidgold.cc/watch/Ki980nV0__0?t=1127' heldReason: rights --- # Computer-use models will agentify the web, not APIs — Dhruv Batra, Yutori To learn what a US school district is buying, you file a Freedom of Information Act request. Someone scans the email you sent, puts the scan on Google Drive, and attaches the relevant PDFs. Dhruv Batra's question is whether anyone seriously expects that office to publish an MCP server. He grants the popular claim that agents will drive most of the action on the web, then rejects its usual next step, that the web will meet them with APIs. The head of the distribution might. The long tail, some 200 million active sites where infrastructure changes over decades, will not. Reading the HTML instead does not save you, because much of what you see was never written down anywhere. A basketball score is missing from the page that first loads and arrives later as JSON. A product page contains no text reading sold out, only a quantity of zero and a script that grays the option out. State is calculated and rendered rather than stored, which makes the browser closer to a game engine than a document and makes pixels the source of truth. He calls this the bitter lesson for web agents: scaffolding built per site does not generalize, and the general solution is the one the web was actually built for. Their Navigator model runs screenshot in and clicks out, now writes JavaScript when that is quicker, and checks the result on screen. It misses 8 of 300 trajectories on a benchmark he thinks should be retired. Speaker info: - https://x.com/DhruvBatra\_ - https://www.linkedin.com/in/dhruv-batra-dbatra/ - https://dhruvbatra.com Timestamps: 0:00 - The argument, and the part of it that is wrong 3:23 - Restaurant menus on easy, medium, and hard mode 5:31 - School district procurement, up to a FOIA request 7:38 - 200 million active sites that change slowly 8:42 - Why reading the HTML does not rescue it 10:14 - Sold out is not text, it is a rendered zero 11:33 - The browser is a rendering engine, pixels are the truth 12:53 - Navigator, and writing JavaScript when that is faster 15:51 - Are computer use models actually good enough 17:20 - Latency and cost per task 18:28 - Another layer of mess that we will call an API --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/Ki980nV0__0-computer-use-models-will-agentify-the-web-not-apis Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/Ki980nV0__0 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.