---
schema: 1
type: video
title: 'Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal'
description: >-
  A frontier scale checkpoint is around 500 GB, so shipping one to a rollout
  fleet in another region takes minutes to hours and kills any hope of weight
  updates landing in seconds. Nan Jiang's claim is that you can send roughly 500
  MB instead and have the rollout engine reconstruct a bitwise identical weights
  version. Fewer than 1% of rollout visible weights actually change between
  consecutive versions, and the reason is not that gradients are sparse.
  Gradients are dense, about 99% of parameters get a nonzero gradient and the
  FP32 master update is dense too. It is just small. The mechanism is a small
  Adam step meeting finite precision. The rollout engine serves a BF16 view
  whose rounding boundary sits near theta over 256, about 0.0039 for a weight
  around 1, while a typical Adam step at RL post training learning rates runs
  around 3 millionths, more than a thousand times too small to cross it. The
  master weights move and the served value does not, which he calls Adam
  absorption. Lower precision serving makes it sharper still: an internal run
  serving GLM 4.7 Air in FP8 saw 0.15% of weights change on the first step and
  settle near 0.05%. Once a lossless patch is the unit of synchronization
  instead of a checkpoint, the rollout fleet stops needing to live in the
  trainer's cluster. Training keeps its all reduce and its fast fabric, rollout
  islands scatter across whatever regions and providers have GPUs right now, a
  sidecar makes any engine version aware, and scattered inference capacity
  becomes one elastic rollout fleet. Modal's implementation is called Stitch.
  Speaker info: - https://x.com/nanjiangwill -
  https://www.linkedin.com/in/nanjiangwill/ - https://www.nanjiangwill.com/ -
  https://github.com/nanjiangwill Timestamps: 0:00 - Where the GPUs actually are
  1:29 - The standard RL post training loop 2:06 - The cathedral and the bazaar
  2:43 - RL wants four things at once 3:22 - What can leave the cluster and what
  cannot 3:59 - The rollout serving island as the movable unit 5:16 - Why a full
  checkpoint is the wrong unit of sync 6:33 - The bet: under 1% of served
  weights change 7:10 - Ingredient one, the precision floor 8:30 - Ingredient
  two, the size of an Adam step 9:48 - Adam absorption, visualized 11:06 -
  Shipping a lossless patch, not a delta 12:21 - What the measurements show
  12:58 - Why this is not gradient sparsity 13:35 - FP8, NVFP4, and group scaled
  formats 14:54 - An internal run on GLM 4.7 Air 15:33 - The bulletin board
  architecture 16:48 - The sidecar that makes engines version aware 17:26 - 500
  GB down to 500 MB 18:02 - Stitch 18:43 - Open questions: Muon, fully async RL,
  and beyond
canonical: 'https://liquidgold.cc/watch/maRzp4kImJ4'
markdown: 'https://liquidgold.cc/watch/maRzp4kImJ4.md'
llms: 'https://liquidgold.cc/watch/maRzp4kImJ4/llms.txt'
parent: 'https://liquidgold.cc/channel/aidotengineer.md'
channel: aidotengineer
updated: '2026-10-24T05:35:46.362Z'
language: null
children: []
videoId: maRzp4kImJ4
durationSeconds: 1190
publishedAt: '2026-08-10T17:30:30.000Z'
searchChunks:
  - ordinal: 0
    text: null
    startSeconds: 23
    endSeconds: 98
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=23'
    heldReason: rights
  - ordinal: 1
    text: null
    startSeconds: 23
    endSeconds: 98
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=23'
    heldReason: rights
  - ordinal: 2
    text: null
    startSeconds: 98
    endSeconds: 109
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=98'
    heldReason: rights
  - ordinal: 3
    text: null
    startSeconds: 109
    endSeconds: 131
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=109'
    heldReason: rights
  - ordinal: 4
    text: null
    startSeconds: 131
    endSeconds: 143
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=131'
    heldReason: rights
  - ordinal: 5
    text: null
    startSeconds: 143
    endSeconds: 168
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=143'
    heldReason: rights
  - ordinal: 6
    text: null
    startSeconds: 179
    endSeconds: 210
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=179'
    heldReason: rights
  - ordinal: 7
    text: null
    startSeconds: 210
    endSeconds: 222
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=210'
    heldReason: rights
  - ordinal: 8
    text: null
    startSeconds: 222
    endSeconds: 245
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=222'
    heldReason: rights
  - ordinal: 9
    text: null
    startSeconds: 245
    endSeconds: 256
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=245'
    heldReason: rights
  - ordinal: 10
    text: null
    startSeconds: 256
    endSeconds: 323
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=256'
    heldReason: rights
  - ordinal: 11
    text: null
    startSeconds: 323
    endSeconds: 334
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=323'
    heldReason: rights
  - ordinal: 12
    text: null
    startSeconds: 334
    endSeconds: 399
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=334'
    heldReason: rights
  - ordinal: 13
    text: null
    startSeconds: 399
    endSeconds: 412
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=399'
    heldReason: rights
  - ordinal: 14
    text: null
    startSeconds: 443
    endSeconds: 510
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=443'
    heldReason: rights
  - ordinal: 15
    text: null
    startSeconds: 510
    endSeconds: 522
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=510'
    heldReason: rights
  - ordinal: 16
    text: null
    startSeconds: 592
    endSeconds: 604
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=592'
    heldReason: rights
  - ordinal: 17
    text: null
    startSeconds: 668
    endSeconds: 680
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=668'
    heldReason: rights
  - ordinal: 18
    text: null
    startSeconds: 748
    endSeconds: 758
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=748'
    heldReason: rights
  - ordinal: 19
    text: null
    startSeconds: 758
    endSeconds: 781
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=758'
    heldReason: rights
  - ordinal: 20
    text: null
    startSeconds: 794
    endSeconds: 818
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=794'
    heldReason: rights
  - ordinal: 21
    text: null
    startSeconds: 829
    endSeconds: 894
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=829'
    heldReason: rights
  - ordinal: 22
    text: null
    startSeconds: 894
    endSeconds: 907
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=894'
    heldReason: rights
  - ordinal: 23
    text: null
    startSeconds: 907
    endSeconds: 940
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=907'
    heldReason: rights
  - ordinal: 24
    text: null
    startSeconds: 951
    endSeconds: 1008
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=951'
    heldReason: rights
  - ordinal: 25
    text: null
    startSeconds: 1008
    endSeconds: 1018
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=1008'
    heldReason: rights
  - ordinal: 26
    text: null
    startSeconds: 1018
    endSeconds: 1051
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=1018'
    heldReason: rights
  - ordinal: 27
    text: null
    startSeconds: 1051
    endSeconds: 1063
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=1051'
    heldReason: rights
  - ordinal: 28
    text: null
    startSeconds: 1085
    endSeconds: 1096
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=1085'
    heldReason: rights
  - ordinal: 29
    text: null
    startSeconds: 1131
    endSeconds: 1142
    watch: 'https://liquidgold.cc/watch/maRzp4kImJ4?t=1131'
    heldReason: rights
---


# Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal

A frontier scale checkpoint is around 500 GB, so shipping one to a rollout fleet in another region takes minutes to hours and kills any hope of weight updates landing in seconds. Nan Jiang's claim is that you can send roughly 500 MB instead and have the rollout engine reconstruct a bitwise identical weights version. Fewer than 1% of rollout visible weights actually change between consecutive versions, and the reason is not that gradients are sparse. Gradients are dense, about 99% of parameters get a nonzero gradient and the FP32 master update is dense too. It is just small. The mechanism is a small Adam step meeting finite precision. The rollout engine serves a BF16 view whose rounding boundary sits near theta over 256, about 0.0039 for a weight around 1, while a typical Adam step at RL post training learning rates runs around 3 millionths, more than a thousand times too small to cross it. The master weights move and the served value does not, which he calls Adam absorption. Lower precision serving makes it sharper still: an internal run serving GLM 4.7 Air in FP8 saw 0.15% of weights change on the first step and settle near 0.05%. Once a lossless patch is the unit of synchronization instead of a checkpoint, the rollout fleet stops needing to live in the trainer's cluster. Training keeps its all reduce and its fast fabric, rollout islands scatter across whatever regions and providers have GPUs right now, a sidecar makes any engine version aware, and scattered inference capacity becomes one elastic rollout fleet. Modal's implementation is called Stitch. Speaker info: - https://x.com/nanjiangwill - https://www.linkedin.com/in/nanjiangwill/ - https://www.nanjiangwill.com/ - https://github.com/nanjiangwill Timestamps: 0:00 - Where the GPUs actually are 1:29 - The standard RL post training loop 2:06 - The cathedral and the bazaar 2:43 - RL wants four things at once 3:22 - What can leave the cluster and what cannot 3:59 - The rollout serving island as the movable unit 5:16 - Why a full checkpoint is the wrong unit of sync 6:33 - The bet: under 1% of served weights change 7:10 - Ingredient one, the precision floor 8:30 - Ingredient two, the size of an Adam step 9:48 - Adam absorption, visualized 11:06 - Shipping a lossless patch, not a delta 12:21 - What the measurements show 12:58 - Why this is not gradient sparsity 13:35 - FP8, NVFP4, and group scaled formats 14:54 - An internal run on GLM 4.7 Air 15:33 - The bulletin board architecture 16:48 - The sidecar that makes engines version aware 17:26 - 500 GB down to 500 MB 18:02 - Stitch 18:43 - Open questions: Muon, fully async RL, and beyond

Canonical: https://liquidgold.cc/watch/maRzp4kImJ4 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md)

> Search chunk text is held until public-source provenance is verified.

## Document chunks

> Missing: no stored document chunks are available.