An LLM performs one operation over and over: given the text so far, predict what comes next.
That description is simplified and narrow, but close. Predicting the next piece of text requires the model to account for grammar, meaning, style, facts, and the relationships between words spread across its context. The operation is simple to state because the complexity has been compressed into the model.
Understanding that loop explains both how an LLM produces an answer and why the answer can surprise us.
Text enters as tokens
A language model does not receive sentences as sentences. It receives tokens, which are small pieces of text. A token might be a whole short word, part of a longer word, punctuation, or whitespace. The exact split depends on the model's tokenizer.
Consider this unfinished sentence:
The cat sat on the
The model turns that text into tokens and processes them through many layers of matrix calculations. Those calculations use numerical weights learned during training. Attention allows each token's representation to draw information from relevant earlier tokens, while the other operations transform and combine that information.
The result is a score for every token the model could produce next. Those scores become a probability distribution. A simplified version might look like this:
mat 42%
floor 21%
chair 12%
roof 3%
all other tokens combined: 22%The model selects one token and adds it to the text. It then runs the prediction again with the longer context:
"The cat sat on the" → predict "mat"
"The cat sat on the mat" → predict what follows "mat"Every generated token becomes input for the next step. The model builds an answer one piece at a time.
Training builds the prediction machinery
During training, the model sees enormous amounts of text and repeatedly tries to predict the next token. Each wrong prediction produces a small adjustment to the model's weights. Repeated across many examples, those adjustments encode useful regularities in language.
The weights do not store a clean library of sentences and facts. They shape how the model transforms a given context into next-token probabilities. Patterns about spelling and syntax live in the same machinery as patterns about source code, historical events, recipes, and how an explanation usually develops.
This is why fluent language and factual accuracy can differ. The model is optimized to continue text plausibly. When its learned patterns support the right answer, plausibility and accuracy align. When the context is weak or the patterns point in the wrong direction, it can be confidently wrong. Each token can fit the preceding text even when the completed claim does not fit reality.
The calculation and the choice are separate
For a particular model and a particular sequence of tokens, the next-token probabilities are determined by the weights and the calculation. Run the same model under the same conditions and it will calculate the same distribution.
The next step depends on how the system chooses from it. Greedy decoding always takes the highest-probability token. Sampling allows lower-probability tokens to be chosen according to their scores. Settings such as temperature control how concentrated or varied that choice can be.
Sampling introduces explicit randomness, but randomness is only part of the reason outputs are difficult to predict. Even with greedy decoding, a person cannot inspect a prompt and work out the result of billions of numerical operations in their head. The only practical way to discover the next token is to run the model.
Then the chosen token changes the context. That produces a new distribution, which produces another token, which changes the context again. A small difference near the beginning can redirect everything that follows.
Determinism is about rules
A process is deterministic when its current state and rules determine its next state. Predictability is about whether an observer can know the result in advance.
Those properties are easy to confuse because simple deterministic systems are usually predictable. A calculator follows fixed rules, and its output is easy to check without watching every internal operation. An LLM also follows mathematical rules, but the scale and feedback of generation remove that easy shortcut.
Knowing the rules does not give us the completed answer. We can describe the process, inspect the probability of a particular next token, and reproduce a run under controlled conditions. We still have to let the sequence unfold to learn exactly where it goes.
Computational irreducibility is a useful name for this limit. A calculation can be decomposed into smaller steps, but decomposition does not necessarily provide a shortcut to its result. At some point the remaining work is the computation itself. For an LLM, predicting the completed answer generally requires letting the model produce it.
Why the output feels like thought
The gap between fixed rules and an unpredictable result explains part of why model output can feel like thought. From the outside, human reasoning and model generation share an awkward property: we see language emerge from an internal process we cannot directly observe.
The resemblance does not establish that the processes are equivalent. Human thought is shaped by a body, persistent memory, motives, perception, and lived experience. An LLM transforms the tokens available in its context using learned weights. The mechanisms differ even when both produce a sentence that surprises the listener.
What they share is opacity to the observer. Complex internal processes are hard to predict from their inputs alone. Surprise tells us that we could not foresee the result. It does not tell us how the result was made.
That is enough to understand the useful part of the comparison. An LLM is a prediction system whose own outputs become its next inputs. Its rules may be fixed, its calculations may be reproducible, and its final answer may remain unknown until the last token is generated.
The one-line version
I have determined the outcome to be unpredictable.