--- schema: 1 type: video title: 'Total Recall: Agent Memory and Harness Engineering — Ignacio Martinez, Oracle' description: >- Ignacio Martinez registered the domain for this workshop on the Saturday before he gave it, and then asked the room not to request the biggest machines on offer, because he was paying for everyone's GitHub Codespace out of his own pocket. That improvisation sits oddly beside the argument underneath it, which is one of the more carefully built cases for harness engineering here. A language model, he says, is the frozen part of the reasoning. The weights do not change, you rent them, and you have no say in what you are offered. Everything you control lives in the layer around it. So an agent is a model plus a harness, and harness engineering is the work of turning a nondeterministic core into repeatable output. He builds that harness in seven layers, and the sharpest arguments are about where memory physically lives. Files match a model's instincts and cost nothing to append to, but they have no transactional consistency, which is why, when he asks the room how parallel agents avoid trampling each other, the answer comes back immediately: work trees. Databases solved that a long time ago, he notes, and everyone forgot. His semantic layer borrows Umwelt from the biologist Jakob von Uexküll, whose claim was that every organism perceives reality only through the lens it happens to have. An agent's lens is whatever institutional knowledge you remembered to write down. Two ideas stay with you. A hysteresis variable encoding how much patience the harness extends to the model before it gives up on a task. And skill promotion, where a workflow that cost you four hours is distilled into a better version and the original retired. Speaker info: - https://x.com/nacho_martinez Timestamps: 0:00 - Getting into the workshop repo 3:22 - What you should leave with 4:02 - The frozen part of the reasoning 6:09 - Five layers of the agent stack 10:50 - Reasoning you rent, memory you control 13:43 - The seven layers of a harness 16:26 - Files versus databases 18:30 - The room answers: work trees 23:26 - The data sprawl a RAG pipeline creates 25:46 - What agent memory means 28:48 - Context rot and quadratic attention 34:23 - Umwelt, and an agent's perceptual lens 37:24 - Context engineering and the toolbox pattern 39:31 - Skill promotion and workflow promotion 41:50 - Workshop setup and the app book 48:13 - Nineteen to dos, starting with a bare model call 51:58 - Question: what if you have thousands of tools? 56:41 - A hysteresis variable for patience --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/xs-ob87TTzg-youtube-c443726c1140f80fdf76 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/xs-ob87TTzg' markdown: 'https://liquidgold.cc/watch/xs-ob87TTzg.md' llms: 'https://liquidgold.cc/watch/xs-ob87TTzg/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T03:24:51.515Z' language: null children: [] videoId: xs-ob87TTzg durationSeconds: 3647 publishedAt: '2026-09-18T16:00:11.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 212 endSeconds: 246 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=212' heldReason: rights - ordinal: 1 text: null startSeconds: 382 endSeconds: 658 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=382' heldReason: rights - ordinal: 2 text: null startSeconds: 382 endSeconds: 658 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=382' heldReason: rights - ordinal: 3 text: null startSeconds: 382 endSeconds: 658 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=382' heldReason: rights - ordinal: 4 text: null startSeconds: 671 endSeconds: 829 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=671' heldReason: rights - ordinal: 5 text: null startSeconds: 671 endSeconds: 829 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=671' heldReason: rights - ordinal: 6 text: null startSeconds: 840 endSeconds: 994 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=840' heldReason: rights - ordinal: 7 text: null startSeconds: 840 endSeconds: 994 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=840' heldReason: rights - ordinal: 8 text: null startSeconds: 1004 endSeconds: 1112 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1004' heldReason: rights - ordinal: 9 text: null startSeconds: 1004 endSeconds: 1112 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1004' heldReason: rights - ordinal: 10 text: null startSeconds: 1112 endSeconds: 1123 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1112' heldReason: rights - ordinal: 11 text: null startSeconds: 1123 endSeconds: 1227 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1123' heldReason: rights - ordinal: 12 text: null startSeconds: 1237 endSeconds: 1348 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1237' heldReason: rights - ordinal: 13 text: null startSeconds: 1237 endSeconds: 1348 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1237' heldReason: rights - ordinal: 14 text: null startSeconds: 1359 endSeconds: 1413 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1359' heldReason: rights - ordinal: 15 text: null startSeconds: 1424 endSeconds: 1557 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1424' heldReason: rights - ordinal: 16 text: null startSeconds: 1557 endSeconds: 1568 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1557' heldReason: rights - ordinal: 17 text: null startSeconds: 1568 endSeconds: 1733 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1568' heldReason: rights - ordinal: 18 text: null startSeconds: 1568 endSeconds: 1733 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1568' heldReason: rights - ordinal: 19 text: null startSeconds: 1733 endSeconds: 1745 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1733' heldReason: rights - ordinal: 20 text: null startSeconds: 1745 endSeconds: 1837 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1745' heldReason: rights - ordinal: 21 text: null startSeconds: 1849 endSeconds: 1966 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1849' heldReason: rights - ordinal: 22 text: null startSeconds: 1966 endSeconds: 1977 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=1966' heldReason: rights - ordinal: 23 text: null startSeconds: 2067 endSeconds: 2079 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=2067' heldReason: rights - ordinal: 24 text: null startSeconds: 2079 endSeconds: 2244 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=2079' heldReason: rights - ordinal: 25 text: null startSeconds: 2079 endSeconds: 2244 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=2079' heldReason: rights - ordinal: 26 text: null startSeconds: 2255 endSeconds: 2381 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=2255' heldReason: rights - ordinal: 27 text: null startSeconds: 2255 endSeconds: 2381 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=2255' heldReason: rights - ordinal: 28 text: null startSeconds: 2392 endSeconds: 2514 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=2392' heldReason: rights - ordinal: 29 text: null startSeconds: 3129 endSeconds: 3409 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=3129' heldReason: rights - ordinal: 30 text: null startSeconds: 3129 endSeconds: 3409 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=3129' heldReason: rights - ordinal: 31 text: null startSeconds: 3129 endSeconds: 3409 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=3129' heldReason: rights - ordinal: 32 text: null startSeconds: 3421 endSeconds: 3647 watch: 'https://liquidgold.cc/watch/xs-ob87TTzg?t=3421' heldReason: rights --- # Total Recall: Agent Memory and Harness Engineering — Ignacio Martinez, Oracle Ignacio Martinez registered the domain for this workshop on the Saturday before he gave it, and then asked the room not to request the biggest machines on offer, because he was paying for everyone's GitHub Codespace out of his own pocket. That improvisation sits oddly beside the argument underneath it, which is one of the more carefully built cases for harness engineering here. A language model, he says, is the frozen part of the reasoning. The weights do not change, you rent them, and you have no say in what you are offered. Everything you control lives in the layer around it. So an agent is a model plus a harness, and harness engineering is the work of turning a nondeterministic core into repeatable output. He builds that harness in seven layers, and the sharpest arguments are about where memory physically lives. Files match a model's instincts and cost nothing to append to, but they have no transactional consistency, which is why, when he asks the room how parallel agents avoid trampling each other, the answer comes back immediately: work trees. Databases solved that a long time ago, he notes, and everyone forgot. His semantic layer borrows Umwelt from the biologist Jakob von Uexküll, whose claim was that every organism perceives reality only through the lens it happens to have. An agent's lens is whatever institutional knowledge you remembered to write down. Two ideas stay with you. A hysteresis variable encoding how much patience the harness extends to the model before it gives up on a task. And skill promotion, where a workflow that cost you four hours is distilled into a better version and the original retired. Speaker info: - https://x.com/nacho_martinez Timestamps: 0:00 - Getting into the workshop repo 3:22 - What you should leave with 4:02 - The frozen part of the reasoning 6:09 - Five layers of the agent stack 10:50 - Reasoning you rent, memory you control 13:43 - The seven layers of a harness 16:26 - Files versus databases 18:30 - The room answers: work trees 23:26 - The data sprawl a RAG pipeline creates 25:46 - What agent memory means 28:48 - Context rot and quadratic attention 34:23 - Umwelt, and an agent's perceptual lens 37:24 - Context engineering and the toolbox pattern 39:31 - Skill promotion and workflow promotion 41:50 - Workshop setup and the app book 48:13 - Nineteen to dos, starting with a bare model call 51:58 - Question: what if you have thousands of tools? 56:41 - A hysteresis variable for patience --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/xs-ob87TTzg-youtube-c443726c1140f80fdf76 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/xs-ob87TTzg · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.