Evals · Part 4Evaluation & Reliability
Prompt and Harness Sensitivity: Same Model, Same Items, New Score
Format, few-shot examples, option order, scoring method and answer extraction all move MMLU scores. Measured on four open models, Haiku 5.5 and Sonnet 5.5.