Forecasts & Uncertainty

A trio of models

Friends,

I hope that all is well with you and yours.

Finally, we are free from Covid (at least for the time being). To be blunt, the last weeks have been a fucking pain in the hole; a family struck down completely by illness, limited to a kitchen, a bathroom and a bedroom, with an ever-growing list of jobs-to-be-done constantly reminding everyone of their presence through unopened cans of paint stacked high. Finding time to write was, and still is, exceedingly difficult.

But even if I can do nothing else, at least I can make an effort. So, let us cut to the chase.

Today, we are going to start looking at various bodies of work that claim to deal with strategic forecasting. To reiterate a previous point, I will not dig into the finer mathematical nuances of predictive modeling; I am not a mathematician, nor will I ever claim to be one. Fortunately, this is not necessarily overly limiting to the present context. Forecasting, as we shall see, has to go beyond algorithms anyway.

Admittedly, that view is not necessarily universal. Laplace’s demon is very much alive and kicking; the consensus appears to be that a better look at the future is inevitable so long as we get a clearer view of the past. This, as it happens, is not true. While making sense of the present is a foundational exercise that can allow us to draw conclusions about the future (as we saw during our Cynefin analysis), some things will inherently be impossible to foresee.

An illustration is provided by the the integrated framework suggested by van Dorsser, Taneja, Walker, and Marchau:

Beginning on the left, on “uncertainty level 1”, we find clear (ordered) contexts. Cause and effect relationships will be obvious and solutions to problems more or less self-evident. Consequently, it stand to reason that we can predict the future with accuracy.

If we take a step to the right, we move to a more complicated (ordered) context. While causal relationships exist, not everyone will be able to see them. Expertise is required; with enough relevant knowledge, one will be able to identify a number of identifiable approaches, all equally valid, that may solve the problem. Put differently, the sample space can be defined – and probabilities can thus be calculated.

Once we get to the rightmost half, things become significantly more difficult. On uncertainty level 3, we may do scenario planning and list plausible alternatives, but not establish how probable any of them are. Level 4 takes us to the kind of complexity described by Stuart Kauffman when it comes to, for example, the becoming of the economy. Here, we find radical emergence, and it is not that we do not know what might happen; a priori, we cannot know what might happen.

Each level demands its own approach:

The Futures Pyramid

Another way of expressing the various levels of uncertainty and the forecast techniques that might be applied accordingly is as a Futures Pyramid. Its central premise is that the higher the place in the hierarchy, the higher the level of associated uncertainty.

To make the framework easier to apply in practice, van Dorsser et. al. elaborate on what each level entails:

Deterministic Forecasting aims to provide a single reliable forecast of the future state of the system based on trend extrapolation and expert judgment. Deterministic forecasts are intended to provide, with a high level of certainty, a single most probable estimate of detailed attributes of the system under consideration for a relatively short time period ahead (e.g., by forecasting the expected output in the next few periods: the next few days, weeks, or years, depending on the time scale in which the historical data are gathered). Deterministic forecasts are often accompanied by a sensitivity analysis that indicates their sensitivity to changes in the most relevant drivers.

Probabilistic Forecasting aims to provide a clear view of the possible outcomes of a system for which the variation in the main parameters is expected to be known with reasonable certainty. The art of Probabilistic Forecasting is fully explored in Bayesian Forecasting, which is founded on the premise that all uncertainties should be presented and measured by probabilities. Probabilistic projections can be applied to both short- and long-term issues, though projections for longer term issues can be obtained only at a higher level of aggregation, since the farther out the time horizon, the less detail that can be taken into account if one aims to remain within a reasonable bandwidth.

Foresight aims to develop a coherent forward view of plausible futures without the need to be explicit on the probabilities with which these futures occur, since at this level of uncertainty it is no longer possible to assign probabilities. The word ‘coherent’ refers to the fact that the forward view is intended to cover the full expected range of plausible futures that lay within the realm of normal expectations, not including the more extreme ones that are addressed by the Futures discipline.

Futures aims to provide a systematic view of possible futures (including ‘known unknowns’ and ‘wildcard’ surprises). The methods for dealing with the ‘known unknowns’ and ‘wildcard’ surprises are associated with Level 4 uncertainty, which is often referred to as ‘deep uncertainty’.

In other words, depending on the level of uncertainty, a particular discipline will apply. So far, so good.

The flaw(s)

It would appear that the pyramid might help strategists classify projections and highlight what techniques might be useful when, in turn improving key decision-making and overall organizational predictability. However, there are two fundamental flaws that keep it from being anything more than an illustrative tool.

Firstly, the pyramid is easily misconstrued. When moving up the design, what is known becomes “smaller” and the insights into the future become less certain; this is reflected by design in the narrowing of the pyramid at the top. Yet my experience tells me that this design also implies a smaller volume, as if the instances where one cannot know what lies ahead were fewer. In a strategic context, the opposite is generally true – and true to the framework, collecting more data does not solve the issue.

Indeed, limiting uncertainty is not as straightforward a matter as just gathering information. For one, collection requires effort, time, and money; eventually one has to decide where to draw the line and define how accurate is accurate enough. But perhaps more to the point, a complex context will not allow such collection unless through probing. This is not something that can be done beforehand and, so to speak, set aside under the assumption that things will remain unchanged.

Secondly, known unknowns rarely bite you in the ass the way that unknown unknowns do, and these do not fit into the pyramid. While there are methods for creating hypotheticals (e.g., Klein’s Pre-Mortems or Kerwin’s Time Machine), we will never be able to know that we have listed them all. Consequently, we still need to develop the ability to adapt to unforeseen events and Black Swans; if we rely solely on the pyramid, there is a significant risk that we end up in hot water in both brisk and brusque fashion.

Given the above, it should not come as a surprise that companies and individuals alike, despite advanced technology and frameworks such as that discussed here today, are notoriously inept at judging the likelihood of uncertain events. However, evidence suggests that there might be a superior approach. In three weeks time, we will take a long look at it.

But first, as always, a Christmas gift special and a wrap up of the year that has been.

Until then, have the loveliest of weekends.

Onwards and upwards,
JP

This newsletter continues below with an additional market analysis exclusive to premium subscribers. To unlock it, an e-book, and a number of lovely perks, merely click the button. If you would rather try the free version first, click here instead.