Forecasting and Judgement – The Differences between Foxes and Hedgehogs

As a futurist, I’m always interested in forecasts and how many people, especially time-poor senior managers, rely on the opinions of experts, who are often not accurate in their outlooks.  So I was very interested when I read about the work of Philip Tetlock on the Long Now site and an excellent article by Paul Monk on it.

Tetlock is a psychologist who has made assessments of the predictions of experts.  Basically, he has demonstrated that increased expertise does not correlate with improved forecasting accuracy.  Who experts were and what they thought did not matter.  What matters is how they think – their style of reasoning – their judgement. 

This is equated with the analogy of the fox (who are sceptical and circumspect) and hedgehogs (who have one grand theory and courage that it will be pertinent in many domains).  Foxes avoid many of the big mistakes of hedgehogs and are somewhat more accurate in shorter term forecasts.  Hedgehogs are prone to hubris and closed-mindedness, of dismissing dissonant possibilities too quickly.  But foxes don’t get off scot-free; they fall into the danger of the cognitive chaos of excessive open-mindedness. 

I liked his view of scenario planning, the next depth stage beyond forecasting for futurists.  He notes that scenario exercises often fail to open the minds of the hedgehogs but succeed in confusing the already open-minded foxes!  It may be that we are better off to acknowledge the uncertainty and hedge against it rather than map it into detailed scenarios. 

A key aspect then to overcome fox and hedgehog limiations is good judgement.  This requires high cognitive skills and the cultivation of the art of overhearing – eavesdropping on the mental conversations we have with ourselves. This is incredibly difficult and it needs a commitment to thinking about how we think – of examining our worldview and rethinking our assumptions. 

Finally, both the Monk and Brand articles stress the importance of keeping score, of using the scientific method to test the results and see what went wrong and learning from it.  It is simply practising double loop learning.  But it rarely gets done. 

The work of Tetlock also reminds me of the work of Scott Armstrong who I heard present in Scotland in 2004.  He noted that forecasts are more accurate if the people making them used simulated interactions (works better with experts) or structured analogies (works better without experts).  His conclusion was that structured methods like these work far better than focus groups or simply gaining expert opinions.

The critical thing from this for me is to give permission to listen to that inner questioning voice and to seek out others to question my assumptions.  One of my major learnings was from a class that gave permission for us to learn from each other, to highlight shortcomings and freely question assumptions.  The trusted conversation was rich and very empowering. 

Advertisements

4 Responses to Forecasting and Judgement – The Differences between Foxes and Hedgehogs

  1. Han van Loon says:

    Hi Luke, the term you may have been seeking is ‘Metacognition’ – or thinking about thinking. It is one of the twelve thinking modes I describe in my book: Reach for the STARS. The ‘Think’ stage in the STARS personal theme explores these modes and how to use them in the appropriate situation.
    regards
    Han van Loon

  2. Thanks Han
    Pual Monk’s piece makes specific mention of judgement as a meta-cognitive skill – this aspect of self-overhearing. It is a key part of futures thinking – of transcending the situation and seeing the way that the environment is changing as a result of your actions and how that way that it is changing (or not changing) you. Of course, beyond metacognition, there is meta-metacognition!

  3. Maree Conway says:

    Hi Luke

    I think too that you need to consider the time frame of the forecasts – if it’s one year out, then any old expert can probably get it right, no matter what their thinking style. If it’s 50 years out, that’s more problematic, and no one should claim expertise over that period of time, given the uncertainty you also mention.

    The examining of worldviews is probably at the core of much futures work – if folks don’t challenge their assumptions about how they think and see the world, not much will be gained.

    Cheers

    Maree

  4. Thanks for that Maree
    I think what the articles were saying was that hedgehogs tend to try and predict out into the future, beyond the capabilities of their expertise in certain timeframes and assume that present trends (and worldviews) will continue.
    You are of course quite right about how worldviews examiniation form the basis of futures work – it’s what makes it so much fun and challenging!
    Thanks for the visit and comment and hope to catch up sometime
    Luke

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: