Forecasting

Attribute Contributions

Prerequisites

Overview

Forecasting is the practice of making explicit, probabilistic predictions about uncertain future events and evaluating the accuracy of those predictions against outcomes. It is distinct from ordinary prediction or speculation in two essential ways: predictions are expressed as probability estimates (60% chance of X, not just 'probably X'), and accuracy is tracked systematically against outcomes over time. This feedback loop — making explicit predictions, waiting for resolution, and comparing prediction to outcome — is the mechanism through which calibration improves and systematic biases become visible.

The forecasting research community, most prominently represented by Philip Tetlock's Good Judgment Project and subsequent superforecasting research, has identified that forecasting skill is real, trainable, and highly variable across individuals. The characteristics of good forecasters — probabilistic thinking, active search for disconfirming evidence, frequent belief updating, comfort with uncertainty, and intellectual humility — can be developed through deliberate practice. Superforecasters outperform domain experts, intelligence analysts, and most benchmarks by significant margins, demonstrating that forecasting skill is learnable rather than innate.

Getting Started

The first discipline is expressing predictions as probability estimates rather than verbal categories. "Probably," "likely," and "unlikely" are poorly defined and do not accumulate into feedback. Forcing explicit numerical probabilities — 75%, 40%, 90% — makes predictions specific enough to score against outcomes and reveals calibration errors that verbal categories conceal. Starting with this discipline, even where the probabilities feel arbitrary, begins the process of developing quantitative intuition.

Calibration is the relationship between stated confidence and actual accuracy. A person who is 70% confident in their predictions should be right about 70% of the time. Overconfident forecasters are right less often than their stated confidence implies; underconfident forecasters are right more often. Measuring calibration requires a sufficient number of resolved predictions — at least fifty, preferably more — to detect systematic bias. Prediction tracking platforms (Metaculus, PredictionBook, Manifold Markets) provide infrastructure for this, recording predictions with timestamps and probability estimates and scoring them automatically when outcomes are known.

Base rates — how often events of this class occur historically — are the most underused input in forecasting. Before reasoning from specific features of a situation, asking how often situations like this one produce the outcome in question provides a prior estimate that specific evidence then adjusts. Neglecting base rates in favor of specific narrative reasoning is the representative heuristic bias that produces some of the largest systematic forecasting errors.

Common Pitfalls

Updating too slowly when new evidence arrives — anchoring on initial estimates and resisting significant revision — produces forecasts that fail to incorporate information that would change a well-calibrated estimate. Bayesian updating (adjusting probabilities in proportion to the strength of new evidence) is the normative standard; practitioner forecasters who update frequently and substantially when evidence warrants perform better than those who adjust incrementally.

Making predictions too vague to score — predictions that can be rationalized as correct regardless of outcome — provides the feeling of forecasting without the feedback that produces learning. The resolution conditions for a prediction must be specified in advance: what exact outcome, by what date, verified by what source. Vagueness is a form of avoiding accountability for predictions.

Confusing forecasting skill with domain expertise is a common error. Deep domain knowledge helps with specific predictions about that domain but does not automatically produce calibration. Political scientists are not reliably better political forecasters than intelligent generalists. The skills of forecasting — probabilistic thinking, calibration discipline, and belief updating — are somewhat independent of domain knowledge.

Milestones

Making and tracking fifty resolved predictions with explicit probability estimates, and calculating a calibration score that shows awareness of systematic bias, marks the foundational data collection milestone. Improving calibration score over a set of at least one hundred predictions relative to an initial baseline marks measurable skill development. Being identified as a top-quintile forecaster on a public prediction platform over at least twelve months marks competitively demonstrated forecasting skill.

Advanced forecasting involves participation in formal forecasting tournaments, development of team forecasting processes, and application to professional decision-making contexts.

Where to Specialize

Superforecasting develops the full toolkit of the Good Judgment Project methodology for geopolitical and current events prediction. Economic forecasting applies forecasting discipline to macroeconomic and financial indicators. Sports analytics applies probabilistic forecasting to game outcomes and player performance. Weather and environmental forecasting develops the specific statistical tools used in scientific prediction systems. Scenario planning applies forecasting thinking to organizational strategic planning under uncertainty.

Tips for Success

• Express predictions as percentages, not words — 'probably' means nothing scorable; 75% can be evaluated against outcomes.
• Track predictions and score them — calibration only improves through feedback, which only exists when predictions are recorded.
• Start with base rates before reasoning from specific details — how often events like this occur is the most underused forecasting input.
• Update significantly when strong evidence arrives — anchoring on initial estimates and adjusting minimally is one of the largest calibration errors.
• Specify resolution conditions before making predictions — vague predictions that can be rationalized as correct provide no useful feedback.
• Seek disconfirming evidence for your predictions — the information that would change your mind is the most valuable information to find.
• Distinguish domain expertise from forecasting skill — knowing a lot about a field does not automatically produce calibrated probability estimates.

Practice Quests

Suggested activities for building your Forecasting skill at different intensities.

Notable Practitioners

Learning Resources

Ready to start tracking Forecasting?