---
schema: 1
type: term
title: '**next-token prediction**'
description: >-
  **Next-token prediction** is the way a language model generates text: given
  everything that has come so far, the model produces the next piece of text —
  the next token — one step at a time, drawing from a probability distribution
  over possible continuations. In the channel's material it is connected to the
  question `what happens next` that stories also ask, and it explains why output
  depends heavily on the surrounding context.
canonical: >-
  https://liquidgold.cc/channel/everyinc/explore/term/next-token-prediction-feea38fd3f29
markdown: >-
  https://liquidgold.cc/channel/everyinc/explore/term/next-token-prediction-feea38fd3f29.md
llms: null
parent: 'https://liquidgold.cc/channel/everyinc/llms.txt'
channel: everyinc
updated: '2026-10-23T22:12:35.346Z'
language: null
children: []
---


# \*\*next-token prediction\*\*

Canonical: https://liquidgold.cc/channel/everyinc/explore/term/next-token-prediction-feea38fd3f29

Parent: [Channel index](https://liquidgold.cc/channel/everyinc/llms.txt)

## Definition

**Next-token prediction** is the way a language model generates text: given everything that has come so far, the model produces the next piece of text — the next token — one step at a time, drawing from a probability distribution over possible continuations. In the channel's material it is connected to the question `what happens next` that stories also ask, and it explains why output depends heavily on the surrounding context.

In this channel's material, **next-token prediction** is the mechanism behind how language models produce text: the model reads what has come so far and generates the next word — the next token — one step at a time. It is the reason the conversation with novelist Robin Sloan frames books and language models as sharing a core question, `what happens next`: stories move forward by asking what comes next, and a language model answers that same question mechanically, token by token, drawing from a probability distribution over possible continuations.Two ideas in the captured material connect directly to next-token prediction. First, great writing is described as sitting **at the edge of a probability distribution** — the model's next-token machinery makes the most likely continuations available, and the most *interesting* writing lives just beyond the predictable ones. Second, when a model acts as a _summarizer_ — as ChatGPT is characterized in the discussion of the human intellect — it is still running the same next-token engine, choosing each output token as the best continuation of the source text it has absorbed.For writers using AI — the throughline of the related material on drafting, editing, and using ChatGPT as a thinking partner — understanding next-token prediction explains *why* prompting matters: everything you put in context shapes which next tokens become likely. The model does not retrieve finished prose; it predicts continuations, so the surrounding context steers the prediction.- A language model generates text by repeatedly predicting the **next token** given everything before it.- The channel links this to storytelling: both books and models are organized around the question of `what happens next`.- **Great writing sits at the edge of a probability distribution** — beyond the most statistically predictable next tokens.- The same next-token mechanism underlies behaviors like summarizing, which is why context shapes output so strongly.