A benchmark score looks like a measurement. It is closer to a poll. Ask a different few hundred questions and the same model scores a point or two higher or lower, with nothing about the model changed. When two models are that close, the leaderboard order is partly luck.
The fix is not a better model. It is more questions, and the patience to check the interval before you believe the gap. Four times the questions buys half the interval.
What is real here. Five models with fixed true pass rates each answer N fresh questions per run, from a seeded generator. Every bar, whisker (Wilson 95% interval), and histogram count falls out of those draws. Nothing is scripted.
One noise source only: a finite set of questions. Real evals also have prompt wording, sampling temperature, grader error, and test questions that leaked into training data. This sim isolates the one you can fix by counting.