The Process of Probabilistic Thinking
In “The Signal and The Noise,” data scientist Nate Silver uses the tales of a professional gambler and a poker champion to illustrate a skill deemed critical to their respective successes: the ability to simultaneously hold in mind multiple hypotheses about the possible outcomes of sporting events and poker hands and update their beliefs accordingly as new data are revealed. Silver writes, “Successful gamblers—and successful forecasters of any kind—do not think of the future in terms of no-lose bets, unimpeachable theories, and infinitely precise measurements. Successful gamblers, instead, think of the future as speckles of probability, flickering upward and downward like a stock market ticker to every new jolt of information.”
Similarly, in “Superforecasting: The Art and Science of Predicting,” Philip Tetlock writes about the strategies of an unlikely but elite group of forecasters whose accuracy greatly exceeded that of supposed experts and states, “[this book is] about how to be accurate, and the superforecasters show that probabilistic thinking is essential for that.”
This skill of probabilistic thinking is crucial in virtually any context in which we need to reason about an uncertain future. Humans are notoriously prone to cognitive biases that, once we have a pet theory in mind, lead us to discount information that conflicts with that theory and more heavily weight information that supports it (see Nobel prize winner Daniel Kahneman’s “Thinking Fast and Slow” for more on this fascinating topic).
This article describes a simple process for making and updating your forecasts in a principled, consistent manner that makes maximum use of the available evidence and helps avoid harmful cognitive biases.
Important note: the following example is completely fictional and shouldn’t be construed as reflecting anything but my own imagination. Nothing about the number of scenarios (I chose four, but there could be from two to two million – it doesn’t matter), the probabilities (as long as they sum to 1.0 you’re okay), nor likelihoods shown (be sensible within your own context) has any basis in reality. They are just there to illustrate the process. When applying this to your own situation, as in most of life, the “Garbage In, Garbage Out” rule applies.
Step 1: Identify The Possible Outcomes
For a given event that you’re trying to forecast, identify a set of mutually exclusive, collectively exhaustive (MECE) outcomes or scenarios. For example, when speculating about the likely response of a competitor to some strategic thrust of your own, you might speculate that they would:
| Scenario | Prior | Likelihood | Posterior |
|---|---|---|---|
| Acquire a smaller, complementary firm | |||
| Develop in-house capabilities | |||
| Partner with a third-party | |||
| Do nothing |
Step 2: Assign Probabilities
For each of the scenarios, establish your best guess as to the relative probability. We’ll refer to these as the “prior probabilities.” Note that the probabilities over this set must sum to 1.0.
Continuing the example, you might assign the following probabilities:
| Scenario | Prior | Likelihood | Posterior |
|---|---|---|---|
| Acquire a smaller, complementary firm | 0.10 | ||
| Develop in-house capabilities | 0.40 | ||
| Partner with a third-party | 0.40 | ||
| Do nothing | 0.10 |
Step 3: Evaluate New Evidence As It Arrives
As new evidence becomes available, evaluate each scenario in terms of the likelihood of seeing that evidence assuming that scenario were true. In other words, how plausible do you believe it is that you’d see that evidence under the given scenario? For example, suppose you heard that your competitor just recruited an executive vice president from another firm in that complementary industry.
First ask yourself: “If my competitor was going to acquire a smaller, complementary firm, how likely is it they would have hired this person?” and assign your best-guess probability. Perhaps that’s 20%.
Next consider, “If my competitor was going to develop in-house capabilities, how likely is it they would have hired this person?” Let’s suppose you estimate that at 60%.
“If my competitor was going to partner with a third-party, how likely is it they would have hired this person?” Maybe 30%?
“If my competitor was going to do nothing…” Fairly unlikely, let’s say only 10%.
| Scenario | Prior | Likelihood | Posterior |
|---|---|---|---|
| Acquire a smaller, complementary firm | 0.10 | 0.20 | |
| Develop in-house capabilities | 0.40 | 0.60 | |
| Partner with a third-party | 0.40 | 0.30 | |
| Do nothing | 0.10 | 0.10 |
Note: unlike probabilities, these likelihood estimates do not have to sum to 1.0. Also, math-nerd pro-tip: never use a likelihood of zero, because once you do, no amount of subsequent evidence to the contrary can remove that (zero times anything remains zero). Use a really, really small number instead, like 0.0001
Step 4: Normalize To Probabilities
Now for each scenario, first multiply each prior probability by the likelihood of the new evidence, then normalize these updated probabilities by dividing each one by the sum of all of them. This ensures that they once again sum to 1.0 as is required for probabilities over a MECE set of discrete events.
So for this example, the intermediate products are 0.02, 0.24, 0.12 and 0.01 and their sum is 0.39, so dividing each scenario’s intermediate value by 0.39 yields posterior probabilities as follows:
| Scenario | Prior | Likelihood | Posterior |
|---|---|---|---|
| Acquire a smaller, complementary firm | 0.10 | 0.20 | 0.05 |
| Develop in-house capabilities | 0.40 | 0.60 | 0.61 |
| Partner with a third-party | 0.40 | 0.30 | 0.31 |
| Do nothing | 0.10 | 0.10 | 0.03 |
We see that our initial best guesses that the competitor would “develop in-house capabilities” compared to “partner with a third-party,” which we originally judged equally likely at 40% probability, have diverged significantly given the new information with the former now twice as probable as the latter.
Step 5: Repeat
As each new piece of data arrives, you can repeat the above calculation (“Bayesian updating”, for my fellow math nerds) treating the former posterior probabilities as the new prior probabilities and updating them by the likelihood of the datum just received. For example, suppose now you hear that the new EVP has been having long meetings with an M&A firm. Just like before, you can make an assessment of the likelihood of this happening under the assumption that the respective hypotheses are true, multiply the prior probabilities by the likelihoods, then renormalize so all the posterior probabilities add up to 1.0
Summary
The ability to consider multiple scenarios simultaneously and adjust one’s beliefs about their relative plausibility as new evidence comes in is one of the most valuable skills a person can develop, whether playing poker for fun and profit or strategizing about more serious concerns. All it requires is some thoughtful consideration of the likelihood of the different scenarios and evidence and a little simple algebra. Next time you’re facing a forecasting problem and want a consistent, principled way to integrate new evidence, give this approach a try.
Give this a try on a problem of your own and let me know how it went in the comments below.