Base Rate Neglect: Why Your Brain Ignores Statistics and Trusts Stories

Your friend started a restaurant last year and it’s thriving. The food is exceptional, the location is perfect, the reviews are glowing. You’ve been thinking about opening your own place for years. Watching their success, the path feels not just possible but obvious. You start looking at properties. You draft a business plan. The excitement builds. At no point do you consult the statistic that should anchor every restaurant decision you make: roughly 60% of restaurants fail within the first year, and nearly 80% close within five years.

You know this statistic. You’ve heard it before. But it doesn’t feel relevant — because your restaurant won’t be like those restaurants. You have a better concept, a better location, a better team. You have a vivid, detailed mental model of success, anchored by a friend’s experience. Against that rich narrative, a dry percentage doesn’t stand a chance.

The research

Daniel Kahneman and Amos Tversky identified this pattern in a 1973 paper in Psychological Review that became one of the most cited works in the history of behavioural science. They called it base rate neglect — the systematic tendency to ignore general statistical information (the base rate) in favour of specific, vivid, case-level details. In a series of experiments, they showed that when people were given both a statistical base rate and a descriptive personality sketch, the personality sketch overwhelmed the statistics in shaping their predictions, even when the base rate was far more diagnostic.

In one classic experiment, participants were told that a group consisted of 70 engineers and 30 lawyers (or vice versa). They were then given personality descriptions and asked to estimate the likelihood that each person was an engineer or lawyer. The descriptions dominated the estimates regardless of the base rate. Even when told the proportion was 70/30 in one direction, a description that sounded “lawyerly” produced estimates of 80-90% for lawyer. The statistical prior — the single most informative piece of evidence — was functionally ignored.

Kahneman and Tversky elaborated on the practical implications in a 1977 report for DARPA, distinguishing between the “inside view” and the “outside view.” The inside view takes the specifics of a situation and builds a forecast from them — your particular plan, your particular team, your particular market. The outside view asks a different question: what typically happens in situations like this one? The inside view produces vivid, confident predictions. The outside view produces boring, accurate ones. People almost always default to the inside view.

Bent Flyvbjerg, a professor of major programme management at Oxford, applied this framework to large-scale infrastructure and technology projects in a 2006 paper in the Project Management Journal. He found that cost overruns, schedule delays, and benefit shortfalls were not occasional misfortunes — they were the norm. Rail projects exceeded budgets by an average of 45%. Technology projects overran by an average of 27%. The consistent pattern across decades and continents wasn’t bad planning. It was base rate neglect: every project team believed their project would be the exception.

The mechanism

The mechanism is rooted in how the brain constructs predictions. When you evaluate a specific situation — your restaurant, your project, your investment — the brain assembles a causal model: a story of how the pieces fit together to produce the expected outcome. This model feels detailed, specific, and therefore reliable. What it lacks is an appreciation for all the ways the same pieces have failed to produce that outcome in the past.

Roger Buehler, Dale Griffin, and Michael Ross demonstrated this in a 1994 study on the planning fallacy, published in the Journal of Personality and Social Psychology. They asked university students to predict when they would complete their thesis projects. Students generated estimates based on their specific plans and circumstances — the inside view. The average prediction was 33.9 days. The actual average completion time was 55.5 days. When the same students were asked to recall how long similar projects had taken them in the past — the outside view — their estimates were far more accurate. They had access to the base rate. They just didn’t use it when making their own predictions.

Kahneman explained the underlying psychology in Thinking, Fast and Slow (2011). The brain operates two systems: System 1, which generates quick, intuitive judgements based on pattern matching and narrative coherence, and System 2, which handles deliberate, effortful analysis. Base rates are System 2 information — abstract, statistical, and cognitively expensive to integrate. Vivid case details are System 1 fuel — concrete, story-shaped, and instantly compelling. When the two conflict, System 1 wins by default unless System 2 is deliberately engaged.

This is why the friend’s successful restaurant is more psychologically powerful than the failure statistic. The restaurant is a story: specific, sensory, emotionally resonant. The base rate is a number: abstract, impersonal, easy to dismiss. Your brain doesn’t reject the statistic. It simply cannot compete for attention against a narrative that feels personally relevant.

Base rates don’t tell you what will happen. They tell you what usually happens — and the gap between those two predictions is where most forecasting errors live.

The practical implications

Start with the base rate, then adjust for specifics. This is the core of what Kahneman called “reference class forecasting.” Before you evaluate the unique features of your situation, identify the relevant reference class — projects like this, businesses like this, decisions like this — and find out what typically happens. Then ask: what specific, concrete evidence do I have that my situation will differ from the base rate? The evidence needs to be specific and verifiable, not “we have a great team” or “we’re more committed.”

The planning fallacy is the most practically damaging form of base rate neglect. Every project, every timeline, every budget you set is a prediction. If you anchor that prediction on your specific plan rather than on how long similar projects have actually taken, you will be systematically overconfident. The fix is simple and uncomfortable: look at your track record. How long did the last three projects like this actually take? That number — not your optimistic new plan — is your best starting estimate.

The base rate is a starting position, not a prison. Starting with “most restaurants fail” doesn’t mean you shouldn’t open a restaurant. It means your business plan needs to explicitly address why you have specific, evidence-based reasons to expect a different outcome. “I’m passionate” is not a reason. “I’ve run two profitable food businesses before and secured a lease in a high-footfall location with below-market rent” might be. The base rate creates a benchmark that your specific evidence must clear.

The bigger picture

We live in a culture that celebrates the exception. Success stories are told and retold precisely because they deviate from the base rate. The entrepreneur who beat the odds, the project that came in under budget, the startup that defied the statistics — these narratives are compelling because they’re rare. But their vividness creates a systematic distortion: the exceptions feel representative, and the base rate — the vastly more common outcome — fades into the background.

This is not a call for pessimism. Base rates are not destiny. They are the accumulated evidence of what happens when ordinary plans meet ordinary reality. Extraordinary outcomes are possible. They’re just less likely than your inside view suggests, and the gap between your confidence and the statistical reality is where the most expensive mistakes are made.

The discipline of starting with “in cases like this, the usual outcome is…” doesn’t limit ambition. It grounds it. The base rate tells you the size of the challenge you’re actually taking on, stripped of narrative gloss and optimistic projection. What you do with that information is still your choice. But at least it will be an informed one.