https://ait.bsc.es/handbook/model-preparation/ Recent content in Model preparation on AI Tools Hugo en https://ait.bsc.es/handbook/model-preparation/llm-fine-tuning/ Mon, 01 Jan 0001 00:00:00 +0000 https://ait.bsc.es/handbook/model-preparation/llm-fine-tuning/ <p>Fine-tuning adapts a pretrained model to a narrower task or style. It is frequently reached for when prompt engineering or retrieval would have solved the problem more cheaply, and it carries serving consequences that are easy to overlook.</p> <p>This page covers when fine-tuning earns its cost and what it changes downstream.</p> <h2 id="what-you-will-learn">What you will learn</h2> <ul> <li>When fine-tuning is the appropriate tool and when it is not.</li> <li>How full fine-tuning and parameter-efficient methods differ operationally.</li> <li>What serving a fine-tuned or adapter-based model requires.</li> <li>How to evaluate whether a fine-tune actually improved your task.</li> </ul> <h2 id="recommended-outline">Recommended outline</h2> <p>This page is an outline. The subsections below are the planned structure; they are filled in as the handbook is written.</p> https://ait.bsc.es/handbook/model-preparation/llm-quantization/ Mon, 01 Jan 0001 00:00:00 +0000 https://ait.bsc.es/handbook/model-preparation/llm-quantization/ <p>Quantization stores and computes with lower-precision numbers, reducing memory footprint and often increasing throughput. The saving is real and frequently large; the accuracy cost is real too, and workload-dependent.</p> <p>This page explains the main approaches and how to decide whether a given quantization is acceptable for your task.</p> <h2 id="what-you-will-learn">What you will learn</h2> <ul> <li>What quantization changes in the memory budget and the compute path.</li> <li>How post-training quantization differs from quantization-aware training.</li> <li>Which parts of a model are most sensitive to reduced precision.</li> <li>How to measure quality loss on your own task rather than trusting a general claim.</li> </ul> <h2 id="recommended-outline">Recommended outline</h2> <p>This page is an outline. The subsections below are the planned structure; they are filled in as the handbook is written.</p> https://ait.bsc.es/handbook/model-preparation/llm-distillation/ Mon, 01 Jan 0001 00:00:00 +0000 https://ait.bsc.es/handbook/model-preparation/llm-distillation/ <p>Distillation transfers the behaviour of a large teacher model into a smaller student. Where it works, it changes the serving economics substantially, because the deployed model is genuinely smaller rather than merely compressed.</p> <p>This page covers the approach, its requirements and its limits.</p> <h2 id="what-you-will-learn">What you will learn</h2> <ul> <li>What distillation is and how it differs from quantization and pruning.</li> <li>What data and compute a distillation run requires.</li> <li>Which task types distil well and which resist it.</li> <li>How to validate that the student preserves the behaviour you care about.</li> </ul> <h2 id="recommended-outline">Recommended outline</h2> <p>This page is an outline. The subsections below are the planned structure; they are filled in as the handbook is written.</p>