Every forecast starts with a question: *What is the probability that X will happen?*
Most people answer that question by telling a story — about why this company is different, why this market cycle is unique, why *this time* the pattern will not hold. It is a very human way to think. It is also one of the most reliable paths to a poorly calibrated forecast.
The alternative has a name: **base rates**. Understanding them may be the single highest-leverage improvement available to anyone who takes financial forecasting, probabilistic reasoning, or decision-making under uncertainty seriously.
This article is not trading advice. It is an exploration of a core idea from probability, statistics, and behavioral economics — one that belongs in the analytical toolkit of anyone working with uncertain outcomes.
---
What Is a Base Rate?
A base rate is the historical frequency of an event within a defined reference class.
Before you assess how likely *this* company is to grow its revenue by 30% next year, ask: across comparable companies — similar sector, similar stage, similar macro conditions — what fraction have historically achieved that growth rate? That fraction is the base rate. It is your anchor before you add case-specific evidence.
Base rates come from statistics and actuarial science. In Bayesian terms, they are *prior probabilities* — the probability you assign to an outcome before incorporating new, situation-specific information.
The practical formula is straightforward:
**P(event) ≈ base rate + adjustment for specific evidence**
In practice, most forecasters invert this. They start with the specific story and adjust only weakly — if at all — for historical frequency. That inversion has a name: **base rate neglect**. And it is nearly universal.
---
The Inside View vs. the Outside View
The psychologist Daniel Kahneman, working with Amos Tversky, formalized a distinction that every serious forecaster should internalize: the **inside view** versus the **outside view**.
The **inside view** is the natural mode of human thinking. You examine the specific case in front of you — its details, its narrative, its apparently unique features — and you build a forecast from those particulars. A CFO projecting next year's revenue is using the inside view. So is an analyst who believes a stock will recover because management is changing. So is almost everyone, almost all of the time.
The **outside view** deliberately steps back and asks: *What happened across all similar cases?* It treats your specific situation as one instance of a broader class of events and anchors on the historical base rate for that class before adding case-specific adjustments.
The outside view is, on average, more accurate — especially over large samples. It is also almost always uncomfortable, because it forces you to treat a carefully analyzed situation as statistically ordinary, which feels wrong precisely because you have invested effort in understanding the particulars.
Philip Tetlock's landmark research on superforecasting confirmed this pattern. The forecasters who consistently outperformed domain experts were those who started with the outside view and adjusted carefully, rather than those who relied primarily on narrative expertise. The combination of outside-view anchoring and disciplined, incremental updating defined the superforecasting approach.
---
Base Rate Neglect: Why We Ignore the Most Useful Data
If base rates are so valuable, why do almost all of us underuse them? Several well-documented mechanisms are at work.
**Availability bias**: Recent, vivid, specific information is mentally louder than statistical abstractions. The story of why *this* trade is compelling overrides the dry fact that a majority of similar trades have not worked.
**Narrative superiority**: Humans process information through stories, not frequency tables. A compelling narrative about a company or market feels more real and more actionable than a base rate drawn from a dataset of historical outcomes.
**The uniqueness illusion**: We routinely believe our situation is more unique than the evidence supports. Every founder believes their startup is different from the base rate. Every acquirer believes their merger will outperform the well-documented majority failure rate. Every market participant believes this particular cycle has genuinely novel features that make historical comparisons less relevant.
**Effort justification**: The more analytical work you have done on a specific case, the more confident you tend to feel — irrespective of whether that work produced better probabilistic calibration. Effort and accuracy are not the same thing, but they feel related.
None of this reflects irrationality in a clinical sense. These are cognitive adaptations that served humans well in many environments. In the context of forecasting market and financial outcomes, however, they introduce systematic, measurable, and ultimately costly bias.
---
Reference Class Forecasting: A Practical Framework
The structured antidote to base rate neglect is **reference class forecasting**, developed by Kahneman and the urban planning researcher Bent Flyvbjerg — who applied it extensively to infrastructure project cost overruns, one of the most persistent and well-documented areas of base rate neglect in any field.
The steps:
1. **Identify a reference class** — a set of past situations genuinely comparable to the one you are forecasting. This step requires discipline: the class must be broad enough to be statistically meaningful, but specific enough to be relevant to your case.
2. **Establish the base rate** — what was the distribution of outcomes in that reference class? Not just the mean, but the variance, the range, and the shape of the tail outcomes.
3. **Place your case within the distribution** — where does your specific situation sit relative to the reference class? What specific evidence, if any, warrants moving from the center of the distribution?
4. **Adjust carefully and conservatively** — the research is consistent here: most forecasters over-adjust. The base rate should move, but rarely dramatically. Specific evidence needs to be strong and directly relevant to justify a large departure.
5. **Track your forecasts** — calibration improves only through honest feedback. Record your probability estimates and compare them to outcomes over time. Without this step, the framework is incomplete.
Applying this consistently is harder than it sounds. Choosing the right reference class involves real judgment, and the remaining steps require resisting strong cognitive pulls toward the inside view at every stage.
---
Base Rates in Financial Markets
Markets are rich with base rate data that is routinely underused in practice.
Consider the kinds of questions base rates can anchor:
- What fraction of companies that miss earnings by a given magnitude recover to prior price levels within six months, across comparable sectors and market conditions? - Across documented drawdowns of a certain depth, what is the historical distribution of recovery timelines? - How often do analyst price targets at a given magnitude above current price get reached within 12 months? - How frequently do currencies that depreciate sharply within a defined window continue depreciating versus mean-reverting?
None of these questions produces a trading signal. They produce a *prior* — a calibration anchor before specific analysis is layered on. The specific analysis then either earns a departure from that prior, or it does not. The size of any warranted departure is itself a judgment the analyst must make explicitly.
Markets also have a structural feature that makes base rates particularly important: tail events are more frequent than standard distributional assumptions suggest. Base rates drawn from long historical windows tend to be more honest about this than forward projections built from shorter, recent data — which systematically underweight low-frequency, high-impact outcomes.
---
How AI Changes the Base Rate Game
One of the genuine advantages well-designed AI systems bring to analytical work is the capacity to work with base rates at scale and without the emotional resistance that causes human analysts to underweight them.
A thoughtfully built system can rapidly construct reference classes across large datasets, identify which classes are statistically relevant for a given situation, track calibration over time as outcomes accumulate, and surface base rate anchors as visible, interrogable parts of the analytical output — rather than allowing them to be suppressed by narrative.
This does not remove judgment. It relocates it. The analyst still decides which reference class is appropriate, whether specific evidence justifies a departure from the base rate, and what to do with the output. But the mechanical work of retrieving and presenting the historical frequency can be systematized in a way that human cognitive architecture makes difficult to sustain without assistance.
Transparency matters here more than it might initially seem. An analytical tool that shows you its reference class, its historical data, and the basis for its probability estimates gives you the ability to interrogate and challenge the reasoning. A black box that outputs a number gives you nothing to check against. In probabilistic forecasting, interpretability is not a luxury — it is a precondition for trust, for learning, and for knowing when the system is wrong.
---
Building Better Forecasts: A Starting Framework
Whether you are analyzing markets, evaluating a business decision, or simply trying to think more clearly about uncertain outcomes, here is a minimal framework grounded in base rate reasoning:
**Step 1**: Before analyzing specifics, ask — *what is the reference class for this type of event, and what is the historical base rate?*
**Step 2**: Write down your base rate estimate before doing any detailed case-specific analysis.
**Step 3**: Conduct your detailed analysis. Then ask — *does this specific evidence genuinely warrant moving from the base rate? If so, by how much, and why?*
**Step 4**: State your final probability estimate explicitly. Avoid vague language like "likely" or "possible" where a number is possible. Forcing a number forces honest thinking.
**Step 5**: Record the forecast, the reasoning, and the eventual outcome. Calibration is a property that only emerges from honest, documented feedback loops over time.
This framework does not generate alpha. It reduces the most predictable forecasting errors — overconfidence, narrative bias, and base rate neglect — which is, paradoxically, where most of the durable edge in forecasting actually comes from.
---
The Discipline of the Outside View
Markets reward conviction. They punish miscalibrated conviction. The difference is almost always invisible in the short run and very visible across long periods and many decisions.
Base rates are not a constraint on thinking. They are a foundation for thinking more honestly. The outside view does not replace analysis of the specific case — it makes that analysis more rigorous by forcing an acknowledgment of where the case sits within a broader historical pattern.
The forecasters who consistently outperform are not always those with access to the most information. They are frequently those who have learned to be honestly calibrated about uncertainty — who start with the reference class, adjust carefully, and track their own accuracy over time.
Base rate neglect is a solvable problem. The solution requires no proprietary data and no special technology. It requires only a decision to look at the reference class first, every time, before the story takes over.