vllm serve meta-llama/Llama-3.1-8B-Instruct --gpu-memory-utilization 0.9

Lets vLLM consume up to 90 percent of GPU memory for the KV cache, the buffer that stores conversation context, instead of the conservative default. Raise it on dedicated inference boxes to fit longer contexts; lower it when the GPU also runs other workloads. Combined with --max-model-len it is the main VRAM tuning knob.