--- schema: 1 type: video title: 'Tell the Robot What You Want — Sandhya Subramani, AWS' description: >- Sandhya Subramani asks her rover how many people it can see, a question nobody ever trained it to answer. It thinks for a second, checks its front camera, and reports two, one near the speaker monitor and one further right. Scout is a small four legged robot running a Raspberry Pi, reaching the internet over a SIM card and a 4G connection, and its whole personality comes from an agent layer sitting above the movement policies it already had. The demo goes exactly as live robot demos go. It falls over, gets coaxed upright, announces a signature performance and then just turns its headlights off. The idea underneath is neat and portable. If you give an agent software tools, you can also hand it a hardware tool and let it choose which preset policy to fire, which turns a robot trained for a fixed task list into something you can address in plain language. She wires it up in about five lines using an open source AWS framework that already covers 40 or so robots. Scout actually runs three agents at once, one thinking about the environment, one talking to her over Telegram, and a voice agent she disabled so it would stop interrupting her on stage. Her framing is that the agent decides what to do while the trained policy decides how, and the robot doubles as a rig for collecting the next round of training data. Speaker info: - https://www.linkedin.com/in/sandhyasubramani/ Timestamps: 0:00 - Meet Scout, running on a Raspberry Pi over 4G 1:27 - Answering a question nobody trained it for 3:46 - Giving an agent a hardware tool 5:02 - Five lines to hand a robot to an agent 6:09 - Three agents running at once 7:17 - The four layers, and how observations travel up 9:35 - Splitting work between cloud and edge 10:45 - Where policies go if they scale like language models 11:53 - Asking it to do something complex 14:12 - What is actually under the hood 15:21 - Mirror mirror, over Telegram --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/S6aSoQ6_u5A-youtube-bfcd4e02445b077a68f3 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/tell-the-robot-what-you-want-sandhya-subramani-aws-11230bd8f192 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/tell-the-robot-what-you-want-sandhya-subramani-aws-11230bd8f192.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T04:17:05.393Z' language: null children: [] videoId: S6aSoQ6_u5A durationSeconds: 1043 publishedAt: '2026-08-29T18:30:17.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 98 endSeconds: 226 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=98' heldReason: rights - ordinal: 1 text: null startSeconds: 98 endSeconds: 226 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=98' heldReason: rights - ordinal: 2 text: null startSeconds: 98 endSeconds: 226 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=98' heldReason: rights - ordinal: 3 text: null startSeconds: 238 endSeconds: 308 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=238' heldReason: rights - ordinal: 4 text: null startSeconds: 238 endSeconds: 308 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=238' heldReason: rights - ordinal: 5 text: null startSeconds: 308 endSeconds: 320 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=308' heldReason: rights - ordinal: 6 text: null startSeconds: 320 endSeconds: 375 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=320' heldReason: rights - ordinal: 7 text: null startSeconds: 387 endSeconds: 442 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=387' heldReason: rights - ordinal: 8 text: null startSeconds: 387 endSeconds: 442 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=387' heldReason: rights - ordinal: 9 text: null startSeconds: 453 endSeconds: 581 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=453' heldReason: rights - ordinal: 10 text: null startSeconds: 453 endSeconds: 581 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=453' heldReason: rights - ordinal: 11 text: null startSeconds: 581 endSeconds: 592 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=581' heldReason: rights - ordinal: 12 text: null startSeconds: 592 endSeconds: 650 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=592' heldReason: rights - ordinal: 13 text: null startSeconds: 662 endSeconds: 719 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=662' heldReason: rights - ordinal: 14 text: null startSeconds: 662 endSeconds: 719 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=662' heldReason: rights - ordinal: 15 text: null startSeconds: 731 endSeconds: 854 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=731' heldReason: rights - ordinal: 16 text: null startSeconds: 854 endSeconds: 865 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=854' heldReason: rights - ordinal: 17 text: null startSeconds: 865 endSeconds: 923 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=865' heldReason: rights - ordinal: 18 text: null startSeconds: 865 endSeconds: 923 watch: 'https://liquidgold.cc/watch/S6aSoQ6_u5A?t=865' heldReason: rights --- # Tell the Robot What You Want — Sandhya Subramani, AWS Sandhya Subramani asks her rover how many people it can see, a question nobody ever trained it to answer. It thinks for a second, checks its front camera, and reports two, one near the speaker monitor and one further right. Scout is a small four legged robot running a Raspberry Pi, reaching the internet over a SIM card and a 4G connection, and its whole personality comes from an agent layer sitting above the movement policies it already had. The demo goes exactly as live robot demos go. It falls over, gets coaxed upright, announces a signature performance and then just turns its headlights off. The idea underneath is neat and portable. If you give an agent software tools, you can also hand it a hardware tool and let it choose which preset policy to fire, which turns a robot trained for a fixed task list into something you can address in plain language. She wires it up in about five lines using an open source AWS framework that already covers 40 or so robots. Scout actually runs three agents at once, one thinking about the environment, one talking to her over Telegram, and a voice agent she disabled so it would stop interrupting her on stage. Her framing is that the agent decides what to do while the trained policy decides how, and the robot doubles as a rig for collecting the next round of training data. Speaker info: - https://www.linkedin.com/in/sandhyasubramani/ Timestamps: 0:00 - Meet Scout, running on a Raspberry Pi over 4G 1:27 - Answering a question nobody trained it for 3:46 - Giving an agent a hardware tool 5:02 - Five lines to hand a robot to an agent 6:09 - Three agents running at once 7:17 - The four layers, and how observations travel up 9:35 - Splitting work between cloud and edge 10:45 - Where policies go if they scale like language models 11:53 - Asking it to do something complex 14:12 - What is actually under the hood 15:21 - Mirror mirror, over Telegram --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/S6aSoQ6_u5A-youtube-bfcd4e02445b077a68f3 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/tell-the-robot-what-you-want-sandhya-subramani-aws-11230bd8f192 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks --- #### Chunk 0: Document excerpt (0:13 to 1:27) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 1: Document excerpt (1:27 to 3:46) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 2: Document excerpt (3:46 to 5:20) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 3: Document excerpt (5:20 to 6:27) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 4: Document excerpt (6:27 to 7:33) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 5: Document excerpt (7:33 to 9:41) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 6: Document excerpt (9:41 to 11:02) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 7: Document excerpt (11:02 to 11:59) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 8: Document excerpt (11:59 to 14:14) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 9: Document excerpt (14:14 to 15:23) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 10: Document excerpt (15:23 to 17:23) > Held: transcript rights do not permit this chunk's text.