--- schema: 1 type: video title: 'Scaling to Long Horizons — Ross Taylor & Chengxi Taylor, General Reasoning' description: >- Ross Taylor opens with some history: back in 2022 he worked on Galactica, an early large model for science that briefly crossed the Rubicon on curated high quality data and intermediate reasoning tokens before the reaction overshadowed the work. That obsession, optimizing what happens between the question and the answer, is where this talk on long horizon reinforcement learning picks up. He and Chengxi Taylor of General Reasoning treat long horizon less as a benchmark and more as a mindset: if you want agents that stay coherent over hours, you have to be patient about signal and deliberate about how you spend tokens. The mechanics they walk through are the ones that make long rollouts trainable. Value models reduce variance and help with credit assignment, bootstrapping pulls signal out of sparse rewards, and the real constraint becomes the tradeoff between off policy staleness and GPU utilization as sequences get longer. They make it concrete with a task where frontier models were handed real money to trade football matches and did poorly, exposing how little the environment was actually simulated. The takeaway is that scaling to long horizons demands better environments and simulation, not just bigger context windows, and they point listeners to openreward.ai to go deeper. Speaker info: - Ross Taylor (General Reasoning): - https://x.com/rosstaylor90 - https://www.linkedin.com/in/rosstaylor90/ - https://rossjtaylor.com - Chengxi Taylor (General Reasoning): - https://x.com/chengxitaylor - https://www.linkedin.com/in/chengxi-taylor/ - https://www.chengxitaylor.com/ Timestamps: 0:00 - Introduction and a look back 1:57 - The Galactica story 5:15 - Curated data and thinking tokens 8:09 - What got RL cooking 9:12 - Long horizon as a mindset 10:16 - Why value models help 11:08 - Credit assignment and bootstrapping 12:38 - Trading football matches for real money 13:44 - Why models struggled 14:36 - Off policy staleness versus GPU use 16:18 - openreward.ai and what's next canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/scaling-to-long-horizons-ross-taylor-chengxi-taylor-931887d4117c markdown: >- https://liquidgold.cc/channel/aidotengineer/video/scaling-to-long-horizons-ross-taylor-chengxi-taylor-931887d4117c.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T06:29:20.527Z' language: null children: [] videoId: 2bvtay8wGYI durationSeconds: 1087 publishedAt: '2026-07-31T21:30:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 134 endSeconds: 323 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=134' heldReason: rights - ordinal: 1 text: null startSeconds: 134 endSeconds: 323 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=134' heldReason: rights - ordinal: 2 text: null startSeconds: 134 endSeconds: 323 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=134' heldReason: rights - ordinal: 3 text: null startSeconds: 334 endSeconds: 492 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=334' heldReason: rights - ordinal: 4 text: null startSeconds: 334 endSeconds: 492 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=334' heldReason: rights - ordinal: 5 text: null startSeconds: 334 endSeconds: 492 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=334' heldReason: rights - ordinal: 6 text: null startSeconds: 503 endSeconds: 560 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=503' heldReason: rights - ordinal: 7 text: null startSeconds: 571 endSeconds: 625 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=571' heldReason: rights - ordinal: 8 text: null startSeconds: 635 endSeconds: 668 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=635' heldReason: rights - ordinal: 9 text: null startSeconds: 668 endSeconds: 679 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=668' heldReason: rights - ordinal: 10 text: null startSeconds: 679 endSeconds: 768 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=679' heldReason: rights - ordinal: 11 text: null startSeconds: 679 endSeconds: 768 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=679' heldReason: rights - ordinal: 12 text: null startSeconds: 768 endSeconds: 778 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=768' heldReason: rights - ordinal: 13 text: null startSeconds: 826 endSeconds: 836 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=826' heldReason: rights - ordinal: 14 text: null startSeconds: 896 endSeconds: 984 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=896' heldReason: rights - ordinal: 15 text: null startSeconds: 896 endSeconds: 984 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=896' heldReason: rights - ordinal: 16 text: null startSeconds: 996 endSeconds: 1087 watch: 'https://liquidgold.cc/watch/2bvtay8wGYI?t=996' heldReason: rights --- # Scaling to Long Horizons — Ross Taylor & Chengxi Taylor, General Reasoning Ross Taylor opens with some history: back in 2022 he worked on Galactica, an early large model for science that briefly crossed the Rubicon on curated high quality data and intermediate reasoning tokens before the reaction overshadowed the work. That obsession, optimizing what happens between the question and the answer, is where this talk on long horizon reinforcement learning picks up. He and Chengxi Taylor of General Reasoning treat long horizon less as a benchmark and more as a mindset: if you want agents that stay coherent over hours, you have to be patient about signal and deliberate about how you spend tokens. The mechanics they walk through are the ones that make long rollouts trainable. Value models reduce variance and help with credit assignment, bootstrapping pulls signal out of sparse rewards, and the real constraint becomes the tradeoff between off policy staleness and GPU utilization as sequences get longer. They make it concrete with a task where frontier models were handed real money to trade football matches and did poorly, exposing how little the environment was actually simulated. The takeaway is that scaling to long horizons demands better environments and simulation, not just bigger context windows, and they point listeners to openreward.ai to go deeper. Speaker info: - Ross Taylor (General Reasoning): - https://x.com/rosstaylor90 - https://www.linkedin.com/in/rosstaylor90/ - https://rossjtaylor.com - Chengxi Taylor (General Reasoning): - https://x.com/chengxitaylor - https://www.linkedin.com/in/chengxi-taylor/ - https://www.chengxitaylor.com/ Timestamps: 0:00 - Introduction and a look back 1:57 - The Galactica story 5:15 - Curated data and thinking tokens 8:09 - What got RL cooking 9:12 - Long horizon as a mindset 10:16 - Why value models help 11:08 - Credit assignment and bootstrapping 12:38 - Trading football matches for real money 13:44 - Why models struggled 14:36 - Off policy staleness versus GPU use 16:18 - openreward.ai and what's next Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/scaling-to-long-horizons-ross-taylor-chengxi-taylor-931887d4117c · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.