Regression to the Mean: Why Extreme Results Lie About What Comes Next

Your sales team has its best quarter in company history. Revenue is up 40%. The CEO attributes it to the new CRM system implemented three months ago. The sales director credits the revised commission structure. The head of marketing points to the rebrand. Everyone has a causal story. And at the next quarterly review, when revenue drops back toward the historical average, the CRM is questioned, the commission structure is reviewed, and the marketing spend is scrutinised. The stories reverse direction as fluidly as they formed.

the record quarter was, in large part, a statistical outlier. Performance fluctuates. Some quarters are high, some are low, most cluster around a mean. An extreme result in one direction will, by mathematical necessity, tend to be followed by a result closer to the average — regardless of what management does in between. The claim does not concern any specific quarter. It is a property of how variable systems behave over time.

The research

Francis Galton first described regression to the mean in 1886, in a paper published in the Journal of the Anthropological Institute. Studying the heights of parents and their children, he observed that exceptionally tall parents tended to have children who were tall but less exceptionally so, and exceptionally short parents tended to have children who were short but closer to average. He called this “regression towards mediocrity.” The effect wasn’t biological in the way Galton initially supposed — it was statistical. Whenever a measurement includes a component of random variation, extreme values on one measurement will tend to be followed by less extreme values on the next, simply because the random component is unlikely to be extreme twice in succession.

Daniel Kahneman described what he considers one of the most important lessons of his career in Thinking, Fast and Slow (2011). Early in his work with the Israeli Air Force, flight instructors told him that praise didn’t work: when they praised a cadet for an exceptional flight, the next flight was usually worse. Punishment, they said, was more effective — when they criticised a poor performance, the next flight improved. Kahneman realised the instructors were observing pure regression to the mean and constructing a causal story around it. After an exceptionally good performance (a statistical outlier), the next performance would regress toward average regardless of feedback. After an exceptionally bad performance, the same regression would occur. The instructors were attributing to their feedback what was actually a property of statistical variation.

Horace Secrist published an entire book in 1933, The Triumph of Mediocrity in Business, documenting how businesses with extreme performance — either very high or very low — tended to converge toward industry averages over time. Secrist interpreted this as a troubling economic finding. The statistician Harold Hotelling pointed out in a review that Secrist had spent an entire book documenting regression to the mean and mistaking it for a substantive discovery about business. The mathematical guarantee had been dressed up as economic insight.

The mechanism

The mechanism is straightforward once understood, but profoundly counterintuitive in practice. Any measured outcome is a function of two components: a stable component (true underlying performance, skill, structural conditions) and a variable component (luck, timing, random fluctuation, measurement noise). When an outcome is extreme — much higher or lower than average — the variable component has almost certainly contributed disproportionately. On the next measurement, the variable component will take a new, independent value, which is most likely to be closer to zero. The result regresses toward the mean.

Adrian Barnett, Jolieke van der Pols, and Annette Dobson, writing in the International Journal of Epidemiology in 2005, identified regression to the mean as one of the most common sources of error in medical and social science research. Patients selected for a study because their symptoms are severe will tend to improve, regardless of treatment — because severe symptoms represent an extreme measurement that will naturally regress. Without a control group, any intervention applied to an extreme group will appear effective.

Thomas Schall and Gary Smith demonstrated the same principle in sports in a 2000 paper in The American Statistician. Baseball players who had exceptional seasons — batting averages well above their career norms — reliably regressed toward their career averages the following year. The regression wasn’t caused by complacency, aging, or strategy changes. It was caused by the mathematics of variable performance.

The critical implication for decision-making is that interventions applied after extreme results will almost always appear to work. Fire a manager after the worst quarter? The next quarter will probably be better — because of regression, not because of the firing. Implement a new process after a record month? The next month will probably be worse — because of regression, not because the process failed. Without understanding this mechanism, every management intervention is evaluated against a baseline that guarantees it will look effective or ineffective for reasons that have nothing to do with the intervention itself.

The most dangerous thing about regression to the mean is that it makes every coincidental intervention look like a cause. Whatever you changed after an extreme result will get the credit or the blame — and you’ll never know it didn’t deserve it.

The practical implications

Wait for a second data point before changing strategy. One extreme result — positive or negative — is not a pattern. It’s a data point that is statistically likely to be followed by a more average result regardless of action. Before overhauling a strategy, a team, or a process in response to a single outlier, wait for at least one more measurement. If the extreme persists, the signal may be real. If it regresses, you’ve avoided an unnecessary and potentially harmful intervention.

Establish baselines so you can distinguish regression from real change. Without knowing what “average” looks like for your system — your team’s typical output, your market’s normal variation, your personal usual performance — you can’t distinguish a meaningful shift from a statistical fluctuation. A baseline doesn’t need to be sophisticated: the average of the last six measurements, with a rough sense of the range, is enough to flag when a result is likely an outlier rather than a new normal.

Be especially sceptical of causal stories attached to extreme results. When a record quarter is attributed to a new strategy, or a terrible month is blamed on a recent change, ask: would this result have occurred anyway, given normal variation? The causal story will always feel more compelling than the statistical explanation — because stories are concrete and vivid, while regression is abstract and boring. But the boring explanation is correct far more often than the compelling one.

The bigger picture

Regression to the mean is one of the most important statistical phenomena in the world and one of the least intuitively understood. Its consequences ripple through every domain where variable performance is measured and managed: sports, medicine, education, business, investing. In each domain, the pattern is the same: extreme results trigger interventions, the results regress to the mean, and the intervention claims credit for a statistical inevitability.

The cost is not just wasted effort. It’s corrupted learning. When you attribute regression to your intervention, you build a false model of what works — a model that will eventually fail when applied in conditions where it needs to actually produce results rather than ride the statistical tide. You learn to repeat whatever you did before a regression-driven improvement, even though your action had no causal connection to the outcome.

Understanding regression doesn’t mean doing nothing in the face of poor results. It means distinguishing between a signal that warrants action and noise that warrants patience. It means building the baseline awareness required to tell the difference. And it means accepting that the most intellectually honest response to an extreme result is often the least satisfying one: wait, measure again, and resist the urge to construct a story before the data tells you whether one is warranted.