Superforecasting is a leaf non-fiction book about forecasting and predictions.
- Good forecasters aggregate perspectives
- During periods of extreme volatility (randomness), forecasts should regress to the mean.
- Hedge hogs and foxes # Introduction
Forecasting should have much more measurement than it currently does. A reasonable model is to forecast-measure-revise, but rarely are forecaster subjected to it. Author is an optimistic-skeptic, skeptical because chaos theory bounds forecasting, but optimistic because so much actually is predictable. Author feels that we likely see and need computer based forecasting blended with subjective judgment. Computer based forecasting will help overcome cognitive limitations and biases. Forecasting often has competing goals and biases besides predicting the future (partisan biases, entertainment, financial incentives, etc.).
Psychology
- Chapter two can safely be skipped if you are familiar or have read Thinking, Fast and Slow.
The Wisdom of Crowds
The crowd aggregates their individual pieces of information. All valid information points in one direction, invalid information cancels each other out in net. This is the power of aggregation.
IARPA Project and Finding Superforecasters
Reiterates much of intro and the formation of the IARPA tournament and the good judgment project.
Intelligence and Superforecasters
- Superforecasters are in the top 80% of the general population, with forecasters (self-selectors to the program) being in the top 70%.
- Superforecasters start with the âoutside viewâ, i.e. the one not specific to the particular case. In most cases this means starting with base rate: how common something is in the broader class.
- Aggregate several perspectives:
- outside view
- inside view
- others outside/inside views
- your own second opinions (by tweaking the question)
Order of Magnitude Estimation (Fermi)
Numeracy and Superforecasters
- Superforecasters are typically highly numerate
- Most people think in terms of: âgoing to happenâ, ânot going to happenâ, âmaybeâ
- In reality, there are few if any certainties (in either direction), and therefore most questions lie in the âmaybeâ region and require probabilistic thinking
- Granularity predicts accuracy in forecasting
Reaction to new information
Under and over reaction are defined by commitment to your forecast. Underreacting to new information is often caused by stickiness to your original idea or forecast. Over reaction lack commitment to their ideas and are swayed too easily by potentially irrelevant information
- Superforecasters update their forecasts frequently over times
Counter Arguments
Tetlock brings up the Black Swan Event and the originator Nassim Taleb who would posit that these critical, history altering events are exceedingly rare and also not predictable. The criticism here is that Superforecasters are good at predicting the mundane, short range events, but those donât really matter in the context of black swan events rarity and importance. Tetlockâs response to this is two fold:
- Black swan events are defined by the acute event but also to the subsequent events. e.g. 9/11 was a dramatic acute event, but had Afghanistan handed over Bin Laden before US invasion, would the 2000s have been defined by the two wars of Iraq and Afghanistan? These trailing events are often the types of questions focused on by the forecasts.
- Black swan events are not as rare or as unpredictable as they seem, e.g. 9/11 was anticipated and several similar plots were foiled in the past