Installation
Load the llm-inference module on MareNostrum 5 and verify it works.
Draft page
This page is an outline. It describes what will be covered and is not yet complete technical documentation.
llm-inference is already installed on MareNostrum 5. You do not build or install anything —
you load it as a module.
Requirements
- A MareNostrum 5 account with access to the GPU partition (
acc). - A valid Slurm billing account.
Load the module
module load EB/apps
module use /apps/ACC/LLM-INFERENCE/modulefiles
module load llm-inference/0.0.1-GCCcore-13.2.0Verify
llm-inference --version
llm-inference --helpIf llm-inference is not found, the module use line did not take effect — check the path and
re-run the three commands in order.
Make it permanent
Avoid retyping this every session
Add the first two lines to your ~/.bashrc:
echo 'module load EB/apps' >> ~/.bashrc
echo 'module use /apps/ACC/LLM-INFERENCE/modulefiles' >> ~/.bashrc
The module load llm-inference/... line is deliberately left out, so you stay in control of
which version you use.
Confirm your model is available
Before launching anything, check the allow list:
llm-inference model listThe table shows, per model: ID, family, data type, allowed GPU range, multi-node support, and
whether the deployed vLLM can load it right now. The SUPPORTED column must read yes.
Warning
A model outside the allow list, or a --gpus value outside its allowed range, is rejected with
model_rejected before any GPU is reserved.
Next
Run your first job: Quickstart.