lm-eval run --model hf --model_args pretrained=meta-llama/Llama-3.1-8B-Instruct --tasks hellaswag --limit 100
Smoke-test an evaluation quickly
Evaluates only the first 100 hellaswag examples, cutting a long run down to minutes so you can verify the setup before committing to the full benchmark. The limit accepts a fraction too, like 0.1. Remove --limit for the real, complete evaluation.
Looking for more? Search all 7,657 commands — works offline, in English or Spanish, and fixes typos.