---
schema: 1
type: video
title: 'Training an LLM from Scratch, Locally — Angelos Perivolaropoulos, ElevenLabs'
description: >-
  Training an LLM from scratch on a local machine sounds unreasonable, until it
  isn't. In this workshop, Angelos Perivolaropoulos from ElevenLabs walks
  through what it actually takes to train a language model locally, with a
  practical focus on the tooling, constraints, and engineering tradeoffs
  involved. If you want a hands-on look at small-scale LLM training beyond the
  cloud-heavy default, this is a useful deep dive. Speaker info: -
  https://www.linkedin.com/in/angelos-perivolaropoulos/ -
  https://github.com/angelos-p Timestamp: 0:00 Introduction and background of
  the speaker 1:21 Overview of the workshop objectives 3:12 Inspiration from
  Andre Karpathy's NanoGPT 4:37 The four fundamental building blocks of an LLM
  7:08 Prerequisites and setup tools (UV, Python, hardware requirements) 9:06
  Part 1: The Tokenizer (character-level tokenization explained) 24:29 Model
  architecture and parameters (vocab size, layers, embeddings) 30:13 The GPT
  class structure and transformer blocks 37:52 Parameter count and model sizing
  40:54 The training loop: objectives and next-token prediction 44:44
  Optimization and learning rate strategies (warm-up and cosine decay) 47:56
  Validation and monitoring loss 53:07 Part 3: Text generation and inference
  strategies 56:30 Putting it all together (project file structure) 58:46
  Monitoring training and debugging common issues 1:00:27 Workshop challenge and
  competition details 1:05:24 Q&A: Differences between base models and reasoning
  models 1:11:31 Q&A: Applying these concepts to audio and multimodal models ---
  Watch along with the corrected transcript, summary, timestamps and resources
  on this talk's official AIE page:
  https://ai.engineer/talks/UsB70Tf5zcE-training-an-llm-from-scratch-locally
  Join our upcoming AI Engineer conferences (see https://ai.engineer landing
  page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs
  and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference
  returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS
  2026 in Sydney ---
canonical: 'https://liquidgold.cc/watch/UsB70Tf5zcE'
markdown: 'https://liquidgold.cc/watch/UsB70Tf5zcE.md'
llms: 'https://liquidgold.cc/watch/UsB70Tf5zcE/llms.txt'
parent: 'https://liquidgold.cc/channel/aidotengineer.md'
channel: aidotengineer
updated: '2026-10-24T14:14:51.956Z'
language: null
children: []
videoId: UsB70Tf5zcE
durationSeconds: 4886
publishedAt: '2026-05-04T18:30:06.000Z'
searchChunks:
  - ordinal: 0
    text: null
    startSeconds: 14
    endSeconds: 87
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=14'
    heldReason: rights
  - ordinal: 1
    text: null
    startSeconds: 98
    endSeconds: 200
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=98'
    heldReason: rights
  - ordinal: 2
    text: null
    startSeconds: 211
    endSeconds: 281
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=211'
    heldReason: rights
  - ordinal: 3
    text: null
    startSeconds: 557
    endSeconds: 568
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=557'
    heldReason: rights
  - ordinal: 4
    text: null
    startSeconds: 696
    endSeconds: 807
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=696'
    heldReason: rights
  - ordinal: 5
    text: null
    startSeconds: 819
    endSeconds: 897
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=819'
    heldReason: rights
  - ordinal: 6
    text: null
    startSeconds: 908
    endSeconds: 1039
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=908'
    heldReason: rights
  - ordinal: 7
    text: null
    startSeconds: 1050
    endSeconds: 1183
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=1050'
    heldReason: rights
  - ordinal: 8
    text: null
    startSeconds: 1305
    endSeconds: 1416
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=1305'
    heldReason: rights
  - ordinal: 9
    text: null
    startSeconds: 1474
    endSeconds: 1485
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=1474'
    heldReason: rights
  - ordinal: 10
    text: null
    startSeconds: 1832
    endSeconds: 1931
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=1832'
    heldReason: rights
  - ordinal: 11
    text: null
    startSeconds: 1941
    endSeconds: 2028
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=1941'
    heldReason: rights
  - ordinal: 12
    text: null
    startSeconds: 2287
    endSeconds: 2460
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=2287'
    heldReason: rights
  - ordinal: 13
    text: null
    startSeconds: 2470
    endSeconds: 2684
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=2470'
    heldReason: rights
  - ordinal: 14
    text: null
    startSeconds: 2695
    endSeconds: 2884
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=2695'
    heldReason: rights
  - ordinal: 15
    text: null
    startSeconds: 2895
    endSeconds: 2994
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=2895'
    heldReason: rights
  - ordinal: 16
    text: null
    startSeconds: 3006
    endSeconds: 3106
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=3006'
    heldReason: rights
  - ordinal: 17
    text: null
    startSeconds: 3204
    endSeconds: 3395
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=3204'
    heldReason: rights
  - ordinal: 18
    text: null
    startSeconds: 3546
    endSeconds: 3632
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=3546'
    heldReason: rights
  - ordinal: 19
    text: null
    startSeconds: 3643
    endSeconds: 3926
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=3643'
    heldReason: rights
  - ordinal: 20
    text: null
    startSeconds: 4023
    endSeconds: 4033
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=4023'
    heldReason: rights
  - ordinal: 21
    text: null
    startSeconds: 4033
    endSeconds: 4135
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=4033'
    heldReason: rights
  - ordinal: 22
    text: null
    startSeconds: 4146
    endSeconds: 4291
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=4146'
    heldReason: rights
  - ordinal: 23
    text: null
    startSeconds: 4302
    endSeconds: 4393
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=4302'
    heldReason: rights
  - ordinal: 24
    text: null
    startSeconds: 4405
    endSeconds: 4542
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=4405'
    heldReason: rights
  - ordinal: 25
    text: null
    startSeconds: 4405
    endSeconds: 4542
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=4405'
    heldReason: rights
  - ordinal: 26
    text: null
    startSeconds: 4405
    endSeconds: 4542
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=4405'
    heldReason: rights
  - ordinal: 27
    text: null
    startSeconds: 4652
    endSeconds: 4751
    watch: 'https://liquidgold.cc/watch/UsB70Tf5zcE?t=4652'
    heldReason: rights
---


# Training an LLM from Scratch, Locally — Angelos Perivolaropoulos, ElevenLabs

Training an LLM from scratch on a local machine sounds unreasonable, until it isn't. In this workshop, Angelos Perivolaropoulos from ElevenLabs walks through what it actually takes to train a language model locally, with a practical focus on the tooling, constraints, and engineering tradeoffs involved. If you want a hands-on look at small-scale LLM training beyond the cloud-heavy default, this is a useful deep dive. Speaker info: - https://www.linkedin.com/in/angelos-perivolaropoulos/ - https://github.com/angelos-p Timestamp: 0:00 Introduction and background of the speaker 1:21 Overview of the workshop objectives 3:12 Inspiration from Andre Karpathy's NanoGPT 4:37 The four fundamental building blocks of an LLM 7:08 Prerequisites and setup tools (UV, Python, hardware requirements) 9:06 Part 1: The Tokenizer (character-level tokenization explained) 24:29 Model architecture and parameters (vocab size, layers, embeddings) 30:13 The GPT class structure and transformer blocks 37:52 Parameter count and model sizing 40:54 The training loop: objectives and next-token prediction 44:44 Optimization and learning rate strategies (warm-up and cosine decay) 47:56 Validation and monitoring loss 53:07 Part 3: Text generation and inference strategies 56:30 Putting it all together (project file structure) 58:46 Monitoring training and debugging common issues 1:00:27 Workshop challenge and competition details 1:05:24 Q\&A: Differences between base models and reasoning models 1:11:31 Q\&A: Applying these concepts to audio and multimodal models --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/UsB70Tf5zcE-training-an-llm-from-scratch-locally Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney ---

Canonical: https://liquidgold.cc/watch/UsB70Tf5zcE · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md)

> Search chunk text is held until public-source provenance is verified.

## Document chunks

> Missing: no stored document chunks are available.