Kernel optimization
Optimizing the GPU kernels that inference workloads spend their time in.
Optimizing the GPU kernels that inference workloads spend their time in.
The pages in this section are listed below in the recommended reading order.
- Kernel optimization for LLM inference Understand when kernel-level work is warranted and what it involves.
- GPU architecture fundamentals The GPU execution and memory model that kernel performance depends on.
- Choosing the right kernel optimization tool Compare the approaches available for writing and tuning inference kernels.
- FlashAttention Understand how FlashAttention reduces attention memory traffic through tiling and recomputation.