--- schema: 1 type: video title: 'The Spatial Harness: Bringing Agents to the Canvas — Max Drake, tldraw' description: >- One of Max Drake's colleagues turned a whiteboard app into a window manager, then used it to play Pong with real desktop windows. Drake is a product engineer at tldraw, the London company behind the free infinite canvas app and, more importantly to him, the SDK powering other people's canvases, including Replit's. tldraw exists because people building canvas apps got stuck on the canvas itself, on selection and resizing and matrix math, and never reached their actual product. His talk is about the same trap one layer up. Coding agents work well because code is text in and text out, the medium they were trained on. Ask one to align something in two dimensions and it falls apart. Teaching agents to understand space takes real engineering. The progression starts with getting a model to read a canvas at all, from a screenshot plus the underlying JSON. Told in one shot to make a mouse blow out a candle, it produces wind and then smoke, positioned correctly, out of ordinary shapes. That becomes the agent starter kit, MIT licensed, where an agent sets its own todos and moves its viewport to go looking for things. Then come fairies, agents rendered as visible characters you can throw around and recolor. The whimsy is load bearing: with ten agents running you read their state by glancing at them rather than reading a chat log. Select several and you get a group chat, one of them acting as orchestrator, assigning work and waiting to review it. Last is a dependency graph where every node is a coding agent, and a desktop build that lets any agent script the editor directly. Speaker info: - https://x.com/max__drake - https://www.linkedin.com/in/maxdrake1/ - https://maxdrake.md Timestamps: 0:00 - A demo kicked off before the talk begins 1:08 - tldraw, the app and the SDK behind other canvases 4:14 - Why coding agents work and canvas agents struggle 5:08 - Teaching a model to read and act on the canvas 7:01 - The agent starter kit goes agentic 8:26 - Fairies, multiplayer agents you can see working 9:49 - A group chat of agents with an orchestrator 12:06 - One node per coding agent, in a dependency graph 15:48 - Code mode, a window manager, and Pong 18:05 - The canvas as a place to work with agents --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/XWcXwnysmpY-youtube-1fd63ad9c4bb8eb6c691 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/XWcXwnysmpY' markdown: 'https://liquidgold.cc/watch/XWcXwnysmpY.md' llms: 'https://liquidgold.cc/watch/XWcXwnysmpY/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T04:05:12.964Z' language: null children: [] videoId: XWcXwnysmpY durationSeconds: 1132 publishedAt: '2026-09-10T14:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 23 endSeconds: 78 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=23' heldReason: rights - ordinal: 1 text: null startSeconds: 90 endSeconds: 254 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=90' heldReason: rights - ordinal: 2 text: null startSeconds: 90 endSeconds: 254 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=90' heldReason: rights - ordinal: 3 text: null startSeconds: 264 endSeconds: 318 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=264' heldReason: rights - ordinal: 4 text: null startSeconds: 318 endSeconds: 328 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=318' heldReason: rights - ordinal: 5 text: null startSeconds: 328 endSeconds: 430 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=328' heldReason: rights - ordinal: 6 text: null startSeconds: 440 endSeconds: 506 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=440' heldReason: rights - ordinal: 7 text: null startSeconds: 517 endSeconds: 595 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=517' heldReason: rights - ordinal: 8 text: null startSeconds: 605 endSeconds: 726 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=605' heldReason: rights - ordinal: 9 text: null startSeconds: 605 endSeconds: 726 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=605' heldReason: rights - ordinal: 10 text: null startSeconds: 738 endSeconds: 957 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=738' heldReason: rights - ordinal: 11 text: null startSeconds: 738 endSeconds: 957 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=738' heldReason: rights - ordinal: 12 text: null startSeconds: 738 endSeconds: 957 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=738' heldReason: rights - ordinal: 13 text: null startSeconds: 957 endSeconds: 969 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=957' heldReason: rights - ordinal: 14 text: null startSeconds: 969 endSeconds: 1093 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=969' heldReason: rights - ordinal: 15 text: null startSeconds: 969 endSeconds: 1093 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=969' heldReason: rights - ordinal: 16 text: null startSeconds: 1093 endSeconds: 1104 watch: 'https://liquidgold.cc/watch/XWcXwnysmpY?t=1093' heldReason: rights --- # The Spatial Harness: Bringing Agents to the Canvas — Max Drake, tldraw One of Max Drake's colleagues turned a whiteboard app into a window manager, then used it to play Pong with real desktop windows. Drake is a product engineer at tldraw, the London company behind the free infinite canvas app and, more importantly to him, the SDK powering other people's canvases, including Replit's. tldraw exists because people building canvas apps got stuck on the canvas itself, on selection and resizing and matrix math, and never reached their actual product. His talk is about the same trap one layer up. Coding agents work well because code is text in and text out, the medium they were trained on. Ask one to align something in two dimensions and it falls apart. Teaching agents to understand space takes real engineering. The progression starts with getting a model to read a canvas at all, from a screenshot plus the underlying JSON. Told in one shot to make a mouse blow out a candle, it produces wind and then smoke, positioned correctly, out of ordinary shapes. That becomes the agent starter kit, MIT licensed, where an agent sets its own todos and moves its viewport to go looking for things. Then come fairies, agents rendered as visible characters you can throw around and recolor. The whimsy is load bearing: with ten agents running you read their state by glancing at them rather than reading a chat log. Select several and you get a group chat, one of them acting as orchestrator, assigning work and waiting to review it. Last is a dependency graph where every node is a coding agent, and a desktop build that lets any agent script the editor directly. Speaker info: - https://x.com/max__drake - https://www.linkedin.com/in/maxdrake1/ - https://maxdrake.md Timestamps: 0:00 - A demo kicked off before the talk begins 1:08 - tldraw, the app and the SDK behind other canvases 4:14 - Why coding agents work and canvas agents struggle 5:08 - Teaching a model to read and act on the canvas 7:01 - The agent starter kit goes agentic 8:26 - Fairies, multiplayer agents you can see working 9:49 - A group chat of agents with an orchestrator 12:06 - One node per coding agent, in a dependency graph 15:48 - Code mode, a window manager, and Pong 18:05 - The canvas as a place to work with agents --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/XWcXwnysmpY-youtube-1fd63ad9c4bb8eb6c691 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/XWcXwnysmpY · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.