lm-eval run --config my_eval_config.yaml --tasks mmlu
Run evaluation from a YAML config
Loads model, batch, and generation settings from my_eval_config.yaml, keeping long flag lists out of the command line and making runs reproducible. Settings given directly on the command line override the file. Great for re-running the same benchmark after model changes.
Looking for more? Search all 7,657 commands — works offline, in English or Spanish, and fixes typos.