--- schema: 1 type: video title: 'Build with Perception Agents — Emile Baizel & Shruti Arora, Amazon AGI Lab' description: >- A coding agent says it added a duck icon to a podcast page; a browser verification pass checks whether the change actually appeared. That audience demo captures the loop Emile Baizel and Shruti Arora build around two perception agent primitives. Visual annotation lets a person select an element, draw on the page or drop a pin, then export structured feedback for a coding agent. Visual verification checks the rendered site against design expectations and user flows. The annotation step is deterministic and produces JSON; the verification step gives the agent a way to inspect the result of its work. Arora starts from a podcast landing page whose heading, colors and buttons need changes, then connects the browser extension to a coding agent through a local bridge. Participants modify the extension's permissions and add an Apply Changes button, turning exported feedback into a workflow that edits the code, runs verification and produces a report. Easy and medium paths let them follow the wiring directly or solve more of the integration themselves. A guest demonstration uses a simulated Bee conversation as another source of design feedback, feeding the same edit and verification loop. Audience examples extend the idea to spoken feature discussions, project planning and recorded UI feedback, with several integrations still at the concept stage. The closing discussion keeps the focus on making collaboration visual and checking the rendered result, rather than accepting a coding agent's claim that the change is done. Speaker info: - https://www.linkedin.com/in/emilebaizel - https://www.linkedin.com/in/shruti-arora-0730 - https://github.com/amazon-agi-labs/perception-agents-workshop Timestamps: 0:00 - Meet Emile Baizel and Shruti Arora 1:50 - Collaboration, perception and reliability 3:40 - Browser workflows with Amazon Nova Act 5:43 - Understanding screens beyond clicking 7:43 - Deterministic visual annotation 8:25 - Verify designs and user flows 9:23 - The frontend feedback scenario 12:52 - Draw, select an element or drop a pin 13:46 - Generate design specs and verification reports 15:26 - Wire the two primitives together 18:29 - Find the workshop repository 19:26 - The podcast landing page challenge 22:15 - Easy and medium workshop paths 23:22 - Nova Act and UI verification skills 24:47 - Walk through the solution 25:25 - Connect the extension to the coding agent 26:21 - Extension permissions and button wiring 27:45 - Apply a page change 29:24 - Logs, verification and the report 31:35 - Slow Wi-Fi during the demo 32:02 - Bee conversations as design feedback 36:52 - A duck podcast and verified changes 40:00 - A conversation to Jira backlog idea 41:37 - Sidekick and project planning 46:59 - A recorded UI feedback integration idea 49:54 - Audience ideas and voting 54:04 - Visual human and agent collaboration --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/ZGzthw4BZiA-youtube-31bae02ce8abb14c7964 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/build-with-perception-agents-emile-baizel-shruti-ar-3c255a2d8c8f markdown: >- https://liquidgold.cc/channel/aidotengineer/video/build-with-perception-agents-emile-baizel-shruti-ar-3c255a2d8c8f.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:19:08.655Z' language: null children: [] videoId: ZGzthw4BZiA durationSeconds: 3343 publishedAt: '2026-10-11T20:00:38.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 122 endSeconds: 228 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=122' heldReason: rights - ordinal: 1 text: null startSeconds: 122 endSeconds: 228 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=122' heldReason: rights - ordinal: 2 text: null startSeconds: 122 endSeconds: 228 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=122' heldReason: rights - ordinal: 3 text: null startSeconds: 122 endSeconds: 228 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=122' heldReason: rights - ordinal: 4 text: null startSeconds: 228 endSeconds: 240 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=228' heldReason: rights - ordinal: 5 text: null startSeconds: 240 endSeconds: 345 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=240' heldReason: rights - ordinal: 6 text: null startSeconds: 240 endSeconds: 345 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=240' heldReason: rights - ordinal: 7 text: null startSeconds: 240 endSeconds: 345 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=240' heldReason: rights - ordinal: 8 text: null startSeconds: 240 endSeconds: 345 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=240' heldReason: rights - ordinal: 9 text: null startSeconds: 345 endSeconds: 356 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=345' heldReason: rights - ordinal: 10 text: null startSeconds: 356 endSeconds: 468 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=356' heldReason: rights - ordinal: 11 text: null startSeconds: 356 endSeconds: 468 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=356' heldReason: rights - ordinal: 12 text: null startSeconds: 356 endSeconds: 468 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=356' heldReason: rights - ordinal: 13 text: null startSeconds: 468 endSeconds: 479 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=468' heldReason: rights - ordinal: 14 text: null startSeconds: 479 endSeconds: 513 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=479' heldReason: rights - ordinal: 15 text: null startSeconds: 479 endSeconds: 513 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=479' heldReason: rights - ordinal: 16 text: null startSeconds: 513 endSeconds: 523 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=513' heldReason: rights - ordinal: 17 text: null startSeconds: 523 endSeconds: 567 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=523' heldReason: rights - ordinal: 18 text: null startSeconds: 577 endSeconds: 777 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=577' heldReason: rights - ordinal: 19 text: null startSeconds: 777 endSeconds: 789 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=777' heldReason: rights - ordinal: 20 text: null startSeconds: 789 endSeconds: 836 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=789' heldReason: rights - ordinal: 21 text: null startSeconds: 846 endSeconds: 926 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=846' heldReason: rights - ordinal: 22 text: null startSeconds: 846 endSeconds: 926 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=846' heldReason: rights - ordinal: 23 text: null startSeconds: 1110 endSeconds: 1122 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1110' heldReason: rights - ordinal: 24 text: null startSeconds: 1171 endSeconds: 1183 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1171' heldReason: rights - ordinal: 25 text: null startSeconds: 1183 endSeconds: 1341 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1183' heldReason: rights - ordinal: 26 text: null startSeconds: 1353 endSeconds: 1407 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1353' heldReason: rights - ordinal: 27 text: null startSeconds: 1531 endSeconds: 1542 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1531' heldReason: rights - ordinal: 28 text: null startSeconds: 1597 endSeconds: 1674 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1597' heldReason: rights - ordinal: 29 text: null startSeconds: 1764 endSeconds: 1776 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1764' heldReason: rights - ordinal: 30 text: null startSeconds: 1931 endSeconds: 1941 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1931' heldReason: rights - ordinal: 31 text: null startSeconds: 1931 endSeconds: 1941 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=1931' heldReason: rights - ordinal: 32 text: null startSeconds: 2411 endSeconds: 2503 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=2411' heldReason: rights - ordinal: 33 text: null startSeconds: 2614 endSeconds: 2704 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=2614' heldReason: rights - ordinal: 34 text: null startSeconds: 2850 endSeconds: 2994 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=2850' heldReason: rights - ordinal: 35 text: null startSeconds: 3262 endSeconds: 3343 watch: 'https://liquidgold.cc/watch/ZGzthw4BZiA?t=3262' heldReason: rights --- # Build with Perception Agents — Emile Baizel & Shruti Arora, Amazon AGI Lab A coding agent says it added a duck icon to a podcast page; a browser verification pass checks whether the change actually appeared. That audience demo captures the loop Emile Baizel and Shruti Arora build around two perception agent primitives. Visual annotation lets a person select an element, draw on the page or drop a pin, then export structured feedback for a coding agent. Visual verification checks the rendered site against design expectations and user flows. The annotation step is deterministic and produces JSON; the verification step gives the agent a way to inspect the result of its work. Arora starts from a podcast landing page whose heading, colors and buttons need changes, then connects the browser extension to a coding agent through a local bridge. Participants modify the extension's permissions and add an Apply Changes button, turning exported feedback into a workflow that edits the code, runs verification and produces a report. Easy and medium paths let them follow the wiring directly or solve more of the integration themselves. A guest demonstration uses a simulated Bee conversation as another source of design feedback, feeding the same edit and verification loop. Audience examples extend the idea to spoken feature discussions, project planning and recorded UI feedback, with several integrations still at the concept stage. The closing discussion keeps the focus on making collaboration visual and checking the rendered result, rather than accepting a coding agent's claim that the change is done. Speaker info: - https://www.linkedin.com/in/emilebaizel - https://www.linkedin.com/in/shruti-arora-0730 - https://github.com/amazon-agi-labs/perception-agents-workshop Timestamps: 0:00 - Meet Emile Baizel and Shruti Arora 1:50 - Collaboration, perception and reliability 3:40 - Browser workflows with Amazon Nova Act 5:43 - Understanding screens beyond clicking 7:43 - Deterministic visual annotation 8:25 - Verify designs and user flows 9:23 - The frontend feedback scenario 12:52 - Draw, select an element or drop a pin 13:46 - Generate design specs and verification reports 15:26 - Wire the two primitives together 18:29 - Find the workshop repository 19:26 - The podcast landing page challenge 22:15 - Easy and medium workshop paths 23:22 - Nova Act and UI verification skills 24:47 - Walk through the solution 25:25 - Connect the extension to the coding agent 26:21 - Extension permissions and button wiring 27:45 - Apply a page change 29:24 - Logs, verification and the report 31:35 - Slow Wi-Fi during the demo 32:02 - Bee conversations as design feedback 36:52 - A duck podcast and verified changes 40:00 - A conversation to Jira backlog idea 41:37 - Sidekick and project planning 46:59 - A recorded UI feedback integration idea 49:54 - Audience ideas and voting 54:04 - Visual human and agent collaboration --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/ZGzthw4BZiA-youtube-31bae02ce8abb14c7964 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/build-with-perception-agents-emile-baizel-shruti-ar-3c255a2d8c8f · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.