How does an LLM work?
A serving-oriented tour of transformer inference: attention, layers and the autoregressive loop.
Draft
Foundations
Draft page
This page is an outline. It describes what will be covered and is not yet complete technical documentation.
You do not need to be able to derive backpropagation to serve a model well, but you do need an accurate picture of what the hardware is doing during each forward pass. This page provides that picture, at the level of detail that serving decisions actually depend on.
The emphasis is on where the time and the memory go, rather than on model quality or training dynamics.
What you will learn
- The shape of a transformer forward pass, layer by layer.
- Why attention cost grows with sequence length.
- How autoregressive generation turns one request into many forward passes.
- Which parts of the computation dominate time and memory.
Recommended outline
This page is an outline. The subsections below are the planned structure; they are filled in as the handbook is written.