Evals · Part 1Evaluation & Reliability
Why Measuring LLMs Is Hard: Validity, Saturation and Goodhart
What a benchmark score claims and what it can't: validity, reliability, saturation and Goodhart's law, with measured probes on ARC, MMLU and MMLU-Pro.