---
schema: 1
type: video
title: 'HTML Is All Agents Need — James Russo, HeyGen'
description: >-
LLMs are great at writing code. So the question we kept asking was: can they
write code that produces a video? We thought it would be easy. The reality was
a year of trying. We started with massive prompts to get very mediocre output.
We made it more agentic to iterate and improve its output. This worked okay
but wasn't production-ready. Eventually we tried Remotion. It got us
deterministic video, but the React framework kept boxing the agent in. The
more guardrails we added, the safer and more boring the outputs got. When we
utilized plain HTML, CSS, and JavaScript, the creativity came back to the
output. So we set out to build a video rendering framework on top of HTML. But
it needed to work with Gemini Flash. Why? Because one tell that a framework is
fighting an agent is needing the biggest model just to get usable output. So
from there we shaped the framework around what small models could reliably
author. That left one real engineering question: can we keep the freedom of
HTML and still render a deterministic MP4? Browsers don't want to do that.
Image decoders, font loaders, and animation clocks all run async on their own
schedule. Great for performance. Terrible for "render the same pixels every
time." Throughout, we iterated constantly with agentic loops and
self-improving evals to test out the framework, find issues in our renderer,
and shape a set of skills that gave the agents Taste instead of guardrails.
This talk is what it took to get there. Speaker: James Russo — Software
Engineer, HeyGen Engineering lead for HyperFrames. Currently at HeyGen
building the future of video storytelling, Previously at Brex X:
https://x.com/Rames_Jusso LinkedIn:
https://www.linkedin.com/in/james-russo-56026897/ Website:
https://boredhacking.com/ Timeline: 0:00 Introduction and the HeyGen mission
0:58 The challenge of creating launch videos 1:27 The importance of A-roll,
B-roll, and composition 2:13 Why HTML, CSS, and JavaScript are the native
languages of LLMs 3:06 Comparing HTML to other frameworks like Remotion 5:24
Designing the Hyperframes framework with Gemini Flash 6:54 How Hyperframes
works in the browser 8:56 Leveraging browser-native technologies like Three.js
and WebGL 9:25 Using Skills to teach agents video taste 10:56 Crafting videos:
the human-in-the-loop workflow 12:09 Keyframes integration 12:35 Scaling and
performance metrics 13:28 Future goals: Code-to-Video benchmarking Quotes:
"Why not let the LLMs and agents talk in their native tongue when creating
videos?" (2:58) "One tell that a framework is fighting an agent is needing the
biggest model just to get usable output." (5:24) "We don't have to teach them
the language. We just teach them how to create good videos." (9:46) "Agents
have made building incredibly easy. Launching is still quite hard." (14:31)
--- Watch along with the corrected transcript, summary, timestamps and
resources on this talk's official AIE page:
https://ai.engineer/talks/Cz4v1WHVyZc-html-is-all-agents-need Join our
upcoming AI Engineer conferences (see https://ai.engineer landing page): -
AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and
engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference
returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS
2026 in Sydney ---
canonical: 'https://liquidgold.cc/watch/Cz4v1WHVyZc'
markdown: 'https://liquidgold.cc/watch/Cz4v1WHVyZc.md'
llms: 'https://liquidgold.cc/watch/Cz4v1WHVyZc/llms.txt'
parent: 'https://liquidgold.cc/channel/aidotengineer.md'
channel: aidotengineer
updated: '2026-10-24T07:07:54.804Z'
language: null
children: []
videoId: Cz4v1WHVyZc
durationSeconds: 913
publishedAt: '2026-07-21T18:54:01.000Z'
searchChunks:
- ordinal: 0
text: null
startSeconds: 146
endSeconds: 192
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=146'
heldReason: rights
- ordinal: 1
text: null
startSeconds: 101
endSeconds: 136
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=101'
heldReason: rights
- ordinal: 2
text: null
startSeconds: 203
endSeconds: 330
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=203'
heldReason: rights
- ordinal: 3
text: null
startSeconds: 342
endSeconds: 423
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=342'
heldReason: rights
- ordinal: 4
text: null
startSeconds: 342
endSeconds: 423
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=342'
heldReason: rights
- ordinal: 5
text: null
startSeconds: 434
endSeconds: 536
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=434'
heldReason: rights
- ordinal: 6
text: null
startSeconds: 434
endSeconds: 536
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=434'
heldReason: rights
- ordinal: 7
text: null
startSeconds: 536
endSeconds: 548
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=536'
heldReason: rights
- ordinal: 8
text: null
startSeconds: 570
endSeconds: 583
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=570'
heldReason: rights
- ordinal: 9
text: null
startSeconds: 583
endSeconds: 665
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=583'
heldReason: rights
- ordinal: 10
text: null
startSeconds: 676
endSeconds: 739
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=676'
heldReason: rights
- ordinal: 11
text: null
startSeconds: 739
endSeconds: 749
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=739'
heldReason: rights
- ordinal: 12
text: null
startSeconds: 826
endSeconds: 913
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=826'
heldReason: rights
- ordinal: 13
text: null
startSeconds: 203
endSeconds: 330
watch: 'https://liquidgold.cc/watch/Cz4v1WHVyZc?t=203'
heldReason: rights
---
# HTML Is All Agents Need — James Russo, HeyGen
LLMs are great at writing code. So the question we kept asking was: can they write code that produces a video? We thought it would be easy. The reality was a year of trying. We started with massive prompts to get very mediocre output. We made it more agentic to iterate and improve its output. This worked okay but wasn't production-ready. Eventually we tried Remotion. It got us deterministic video, but the React framework kept boxing the agent in. The more guardrails we added, the safer and more boring the outputs got. When we utilized plain HTML, CSS, and JavaScript, the creativity came back to the output. So we set out to build a video rendering framework on top of HTML. But it needed to work with Gemini Flash. Why? Because one tell that a framework is fighting an agent is needing the biggest model just to get usable output. So from there we shaped the framework around what small models could reliably author. That left one real engineering question: can we keep the freedom of HTML and still render a deterministic MP4? Browsers don't want to do that. Image decoders, font loaders, and animation clocks all run async on their own schedule. Great for performance. Terrible for "render the same pixels every time." Throughout, we iterated constantly with agentic loops and self-improving evals to test out the framework, find issues in our renderer, and shape a set of skills that gave the agents Taste instead of guardrails. This talk is what it took to get there. Speaker: James Russo — Software Engineer, HeyGen Engineering lead for HyperFrames. Currently at HeyGen building the future of video storytelling, Previously at Brex X: https://x.com/Rames_Jusso LinkedIn: https://www.linkedin.com/in/james-russo-56026897/ Website: https://boredhacking.com/ Timeline: 0:00 Introduction and the HeyGen mission 0:58 The challenge of creating launch videos 1:27 The importance of A-roll, B-roll, and composition 2:13 Why HTML, CSS, and JavaScript are the native languages of LLMs 3:06 Comparing HTML to other frameworks like Remotion 5:24 Designing the Hyperframes framework with Gemini Flash 6:54 How Hyperframes works in the browser 8:56 Leveraging browser-native technologies like Three.js and WebGL 9:25 Using Skills to teach agents video taste 10:56 Crafting videos: the human-in-the-loop workflow 12:09 Keyframes integration 12:35 Scaling and performance metrics 13:28 Future goals: Code-to-Video benchmarking Quotes: "Why not let the LLMs and agents talk in their native tongue when creating videos?" (2:58) "One tell that a framework is fighting an agent is needing the biggest model just to get usable output." (5:24) "We don't have to teach them the language. We just teach them how to create good videos." (9:46) "Agents have made building incredibly easy. Launching is still quite hard." (14:31) --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/Cz4v1WHVyZc-html-is-all-agents-need Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney ---
Canonical: https://liquidgold.cc/watch/Cz4v1WHVyZc · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md)
> Search chunk text is held until public-source provenance is verified.
## Document chunks
---
#### Chunk 0: Document excerpt (0:36 to 0:58)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 1: Document excerpt (0:58 to 1:30)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 2: Document excerpt (1:30 to 2:16)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 3: Document excerpt (2:16 to 3:23)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 4: Document excerpt (3:23 to 5:30)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 5: Document excerpt (5:30 to 7:03)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 6: Document excerpt (7:03 to 8:56)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 7: Document excerpt (8:56 to 9:30)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 8: Document excerpt (9:30 to 11:05)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 9: Document excerpt (11:05 to 12:19)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 10: Document excerpt (12:19 to 12:39)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 11: Document excerpt (12:39 to 13:33)
> Held: transcript rights do not permit this chunk's text.
---
#### Chunk 12: Document excerpt (13:33 to 15:13)
> Held: transcript rights do not permit this chunk's text.