---
schema: 1
type: video
title: >-
  The spelled-out intro to neural networks and backpropagation: building
  micrograd
description: >-
  This is the most step-by-step spelled-out explanation of backpropagation and
  training of neural networks. It only assumes basic knowledge of Python and a
  vague recollection of calculus from high school. Links: - micrograd on github:
  https://github.com/karpathy/micrograd - jupyter notebooks I built in this
  video:
  https://github.com/karpathy/nn-zero-to-hero/tree/master/lectures/micrograd -
  my website: https://karpathy.ai - my twitter: https://twitter.com/karpathy -
  "discussion forum": nvm, use youtube comments below for now :) - (new) Neural
  Networks: Zero to Hero series Discord channel: https://discord.gg/3zy8kqD9Cp ,
  for people who'd like to chat more and go beyond youtube comments Exercises:
  you should now be able to complete the following google collab, good luck!:
  https://colab.research.google.com/drive/1FPTx1RXtBfc4MaTkf7viZZD4U2F9gtKN?usp=sharing
  Chapters: 00:00:00 intro 00:00:25 micrograd overview 00:08:08 derivative of a
  simple function with one input 00:14:12 derivative of a function with multiple
  inputs 00:19:09 starting the core Value object of micrograd and its
  visualization 00:32:10 manual backpropagation example #1: simple expression
  00:51:10 preview of a single optimization step 00:52:52 manual backpropagation
  example #2: a neuron 01:09:02 implementing the backward function for each
  operation 01:17:32 implementing the backward function for a whole expression
  graph 01:22:28 fixing a backprop bug when one node is used multiple times
  01:27:05 breaking up a tanh, exercising with more operations 01:39:31 doing
  the same thing but in PyTorch: comparison 01:43:55 building out a neural net
  library (multi-layer perceptron) in micrograd 01:51:04 creating a tiny
  dataset, writing the loss function 01:57:56 collecting all of the parameters
  of the neural net 02:01:12 doing gradient descent optimization manually,
  training the network 02:14:03 summary of what we learned, how to go towards
  modern neural nets 02:16:46 walkthrough of the full code of micrograd on
  github 02:21:10 real stuff: diving into PyTorch, finding their backward pass
  for tanh 02:24:39 conclusion 02:25:20 outtakes :)
canonical: >-
  https://liquidgold.cc/channel/andrejkarpathy/explore/video/the-spelled-out-intro-to-neural-networks-and-backpr-a7c3ec5e215b
markdown: >-
  https://liquidgold.cc/channel/andrejkarpathy/video/the-spelled-out-intro-to-neural-networks-and-backpr-a7c3ec5e215b.md
llms: null
parent: 'https://liquidgold.cc/channel/andrejkarpathy.md'
channel: andrejkarpathy
updated: '2026-10-25T03:13:13.626Z'
language: null
children: []
videoId: VMj-3S1tku0
durationSeconds: 8752
publishedAt: '2022-08-16T22:44:26.000Z'
searchChunks:
  - ordinal: 0
    text: null
    startSeconds: 44
    endSeconds: 133
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=44'
    heldReason: rights
  - ordinal: 1
    text: null
    startSeconds: 144
    endSeconds: 258
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=144'
    heldReason: rights
  - ordinal: 2
    text: null
    startSeconds: 269
    endSeconds: 358
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=269'
    heldReason: rights
  - ordinal: 3
    text: null
    startSeconds: 269
    endSeconds: 358
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=269'
    heldReason: rights
  - ordinal: 4
    text: null
    startSeconds: 369
    endSeconds: 493
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=369'
    heldReason: rights
  - ordinal: 5
    text: null
    startSeconds: 652
    endSeconds: 790
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=652'
    heldReason: rights
  - ordinal: 6
    text: null
    startSeconds: 802
    endSeconds: 858
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=802'
    heldReason: rights
  - ordinal: 7
    text: null
    startSeconds: 868
    endSeconds: 1157
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=868'
    heldReason: rights
  - ordinal: 8
    text: null
    startSeconds: 1170
    endSeconds: 1262
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=1170'
    heldReason: rights
  - ordinal: 9
    text: null
    startSeconds: 1374
    endSeconds: 1506
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=1374'
    heldReason: rights
  - ordinal: 10
    text: null
    startSeconds: 1516
    endSeconds: 1597
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=1516'
    heldReason: rights
  - ordinal: 11
    text: null
    startSeconds: 1812
    endSeconds: 1936
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=1812'
    heldReason: rights
  - ordinal: 12
    text: null
    startSeconds: 2174
    endSeconds: 2286
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=2174'
    heldReason: rights
  - ordinal: 13
    text: null
    startSeconds: 2554
    endSeconds: 2667
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=2554'
    heldReason: rights
  - ordinal: 14
    text: null
    startSeconds: 2680
    endSeconds: 2759
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=2680'
    heldReason: rights
  - ordinal: 15
    text: null
    startSeconds: 2886
    endSeconds: 3020
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=2886'
    heldReason: rights
  - ordinal: 16
    text: null
    startSeconds: 3086
    endSeconds: 3178
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=3086'
    heldReason: rights
  - ordinal: 17
    text: null
    startSeconds: 3188
    endSeconds: 3287
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=3188'
    heldReason: rights
  - ordinal: 18
    text: null
    startSeconds: 3287
    endSeconds: 3297
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=3287'
    heldReason: rights
  - ordinal: 19
    text: null
    startSeconds: 3488
    endSeconds: 3499
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=3488'
    heldReason: rights
  - ordinal: 20
    text: null
    startSeconds: 3773
    endSeconds: 3882
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=3773'
    heldReason: rights
  - ordinal: 21
    text: null
    startSeconds: 4664
    endSeconds: 4952
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=4664'
    heldReason: rights
  - ordinal: 22
    text: null
    startSeconds: 4963
    endSeconds: 5230
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=4963'
    heldReason: rights
  - ordinal: 23
    text: null
    startSeconds: 5355
    endSeconds: 5485
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=5355'
    heldReason: rights
  - ordinal: 24
    text: null
    startSeconds: 5496
    endSeconds: 5600
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=5496'
    heldReason: rights
  - ordinal: 25
    text: null
    startSeconds: 5713
    endSeconds: 5725
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=5713'
    heldReason: rights
  - ordinal: 26
    text: null
    startSeconds: 5815
    endSeconds: 5894
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=5815'
    heldReason: rights
  - ordinal: 27
    text: null
    startSeconds: 5992
    endSeconds: 6242
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=5992'
    heldReason: rights
  - ordinal: 28
    text: null
    startSeconds: 6254
    endSeconds: 6390
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=6254'
    heldReason: rights
  - ordinal: 29
    text: null
    startSeconds: 6533
    endSeconds: 6666
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=6533'
    heldReason: rights
  - ordinal: 30
    text: null
    startSeconds: 6791
    endSeconds: 6802
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=6791'
    heldReason: rights
  - ordinal: 31
    text: null
    startSeconds: 7090
    endSeconds: 7282
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=7090'
    heldReason: rights
  - ordinal: 32
    text: null
    startSeconds: 7380
    endSeconds: 7390
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=7380'
    heldReason: rights
  - ordinal: 33
    text: null
    startSeconds: 7594
    endSeconds: 7607
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=7594'
    heldReason: rights
  - ordinal: 34
    text: null
    startSeconds: 7916
    endSeconds: 7927
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=7916'
    heldReason: rights
  - ordinal: 35
    text: null
    startSeconds: 8223
    endSeconds: 8474
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=8223'
    heldReason: rights
  - ordinal: 36
    text: null
    startSeconds: 8474
    endSeconds: 8485
    watch: 'https://liquidgold.cc/watch/VMj-3S1tku0?t=8474'
    heldReason: rights
---


# The spelled-out intro to neural networks and backpropagation: building micrograd

This is the most step-by-step spelled-out explanation of backpropagation and training of neural networks. It only assumes basic knowledge of Python and a vague recollection of calculus from high school. Links: - micrograd on github: https://github.com/karpathy/micrograd - jupyter notebooks I built in this video: https://github.com/karpathy/nn-zero-to-hero/tree/master/lectures/micrograd - my website: https://karpathy.ai - my twitter: https://twitter.com/karpathy - "discussion forum": nvm, use youtube comments below for now :) - (new) Neural Networks: Zero to Hero series Discord channel: https://discord.gg/3zy8kqD9Cp , for people who'd like to chat more and go beyond youtube comments Exercises: you should now be able to complete the following google collab, good luck!: https://colab.research.google.com/drive/1FPTx1RXtBfc4MaTkf7viZZD4U2F9gtKN?usp=sharing Chapters: 00:00:00 intro 00:00:25 micrograd overview 00:08:08 derivative of a simple function with one input 00:14:12 derivative of a function with multiple inputs 00:19:09 starting the core Value object of micrograd and its visualization 00:32:10 manual backpropagation example #1: simple expression 00:51:10 preview of a single optimization step 00:52:52 manual backpropagation example #2: a neuron 01:09:02 implementing the backward function for each operation 01:17:32 implementing the backward function for a whole expression graph 01:22:28 fixing a backprop bug when one node is used multiple times 01:27:05 breaking up a tanh, exercising with more operations 01:39:31 doing the same thing but in PyTorch: comparison 01:43:55 building out a neural net library (multi-layer perceptron) in micrograd 01:51:04 creating a tiny dataset, writing the loss function 01:57:56 collecting all of the parameters of the neural net 02:01:12 doing gradient descent optimization manually, training the network 02:14:03 summary of what we learned, how to go towards modern neural nets 02:16:46 walkthrough of the full code of micrograd on github 02:21:10 real stuff: diving into PyTorch, finding their backward pass for tanh 02:24:39 conclusion 02:25:20 outtakes :)

Canonical: https://liquidgold.cc/channel/andrejkarpathy/explore/video/the-spelled-out-intro-to-neural-networks-and-backpr-a7c3ec5e215b · Channel: [Andrej Karpathy](https://liquidgold.cc/channel/andrejkarpathy.md)

> Search chunk text is held until public-source provenance is verified.

## Document chunks

> Missing: no stored document chunks are available.