Checks that the hellaswag task definition loads and its few-shot prompt construction works, catching YAML or data errors before a long evaluation run starts. It is the harness's self-test for task configs. Custom tasks can be added with --include_path pointing at their directory.
Looking for more? Search all 7,657 commands — works offline, in English or Spanish, and fixes typos.