Foundations
The core vocabulary and mental models behind LLM inference.
The core vocabulary and mental models behind LLM inference.
The pages in this section are listed below in the recommended reading order.
- What is LLM inference? Define LLM inference and distinguish the prefill and decode phases that shape every serving decision.
- Training vs. inference Compare the two workloads and explain why infrastructure tuned for training rarely suits serving.
- How does an LLM work? A serving-oriented tour of transformer inference: attention, layers and the autoregressive loop.
- Where is LLM inference run? Survey the deployment targets for inference — cloud, on-premises, hybrid and edge — and their trade-offs.
- Key metrics for LLM inference Define TTFT, TPOT, end-to-end latency, throughput and goodput, and explain what each one hides.