GPU architecture fundamentals
The GPU execution and memory model that kernel performance depends on.
Draft
Kernel optimization
Draft page
This page is an outline. It describes what will be covered and is not yet complete technical documentation.
Reasoning about kernel performance requires an accurate model of the hardware: how work is scheduled onto execution units, and how the memory hierarchy behaves under different access patterns.
This page covers the architectural fundamentals that inference performance depends on.
What you will learn
- How GPU work is organised into threads, warps and blocks.
- The memory hierarchy and the cost of each level.
- Why coalesced memory access matters so much.
- How to reason about occupancy and arithmetic intensity.
Recommended outline
This page is an outline. The subsections below are the planned structure; they are filled in as the handbook is written.