vllm serve meta-llama/Llama-3.1-8B-Instruct --chat-template ./chat_template.jinja

Category: AI/ML Tooling

vllm serve meta-llama/Llama-3.1-8B-Instruct --chat-template ./chat_template.jinja

Serve with a custom chat template

Uses the Jinja2 template in chat_template.jinja to format chat messages into the model's expected prompt shape, overriding the template bundled with the tokenizer. Custom templates fix garbled multi-turn output for models with unusual chat formats. Test with one /v1/chat/completions call after starting.
Looking for more? Search all 7,657 commands — works offline, in English or Spanish, and fixes typos.