"Here's something to think about: How come you never see a headline like 'Psychic Wins Lottery'?" – Jay Leno
What if you knew exactly how everything was going to happen? How would you feel? Probably bored out of your mind. We need certainties (food, water, shelter) but we also need surprise - uncertainty. To deal with life’s uncertainties, we have established the long tradition of prediction and have put a lot of faith (and money) in it.
Predictions are not about divining the truth. Far from it, predictions or forecasts are about tying a change in current conditions to a possible outcome in the future. In other words, if done well a prediction is an educated guess and may come close to what actually happens. There are no sure things in the world of predicting. Clearly, your mileage may vary with the forecaster and his/her forecast. We follow other people’s predictions, notably the weather (and until recently, that housing prices would continue to rise), and very rarely do we stop to think about exactly what they are doing. There is no value-added superstition behind forecasting, no mumbo-jumbo. But chances are, there is a mathematical model behind the scenes. Even so, there is no certainty involved, only possibility. And mathematics. And an imperfect understanding of reality. And that is where things can start to go awry.
The economic models of the last ten years have predicted untold wealth in the real estate and mortgage markets – so much so that we all came to believe it, and therefore it became our reality. But the models had one flaw. Okay, more than one, but this one is a doozy. These economic models never included the possibility of housing prices decreasing. Now that seems a pretty obvious factor to take into consideration, doesn’t it? Intuitively and practically we usually think that things that go up have to come down at some point or at least level off. So how did this get missed? To answer that question, let’s paint a picture and revisit a flood.
Imagine a friend asks you to paint their portrait. You would find a canvas and paints, situate your subject in a suitable pose and begin. Okay, if you have no artistic talent imagine it’s modern art and you can fake it. At the end of your work, you have a portrait. Great! Now, imagine that the person asked you to make the portrait speak. “Yeah, I want it to say something witty every time anyone walks past it”, they say. This isn’t the same thing as that plastic fish hanging on the wall singing “Take me to the river”, after all. How can a flat, two-dimensional representation hope to bear close resemblance to real-life? But this is precisely the conundrum that forecasters find themselves in when they want to make a prediction: they need to model the real world, using something that isn’t the real world (paper, computer etc). And the real world changes, unpredictably. No, really – it does.
Twelve years ago, forecasters at the North Central River Forecast Center in Chanhassen thought the Red River would just top the 50-foot mark in Grand Forks. The weather service has remarked that its 49-foot crest prediction was never meant to be perfect, nor was it something that residents could completely rely on. But most did. The forecast was off by 3.8 feet, a small error relative to the size of the crest but one that that forced most of the 60,000 people in Grand Forks and neighboring East Grand Forks, Minn., out of their homes. The total cost of damages ran to about $3.5 billion. Viewed in that way, it was a not-so-small prediction error.
If you calculate the total volume of water in a landscape and plot it against the depth of that water collecting together is flows down a river, you typically get a nice smooth arc that looks like about half a rainbow. And that's how the Red River seemed to work for most of recorded history.
But it turns out that as the volume of water peaks in very flat rivers, like the Red, odd things start to happen. Unexpected things. The flooding gets higher, even though the total amount of water is decreasing. If you're plotting it on a graph, the curve actually loops up at the end, like a whip snap, before it goes down; an elegant and poignant detail that was not part of the general flood model. The weather service now figures this effect, known as "looping", accounted for about two feet of the 1997 flooding in Grand Forks – more than half of their miscalculation. Looping is caused by a complex phenomenon of friction and velocity that are not easily modeled but is easily verifiable through Geological Survey measurements. However, modeling real life in real time has proven to be very difficult. But that’s where it counts.
The National Weather Service has changed its forecasting methods since the 1997 flood disaster - not only for Grand Forks but also for disasters such as Hurricane Katrina. "Our science, modeling and communication are better," said Dan Luna, hydrologist in charge at the National Weather Service office in Chanhassen, Minn. "We learned a lot in 1997."
Some of what they learned involves telling the public what forecasters really don't know.
Forecasters now stress the uncertainties of their predictions, with a range of worst-case possibilities. A prediction now represents multiple possible future outcomes. The national Weather Service calls it "hedging." That seems to be a better term for what remains an educated guess about possibilities. The models have improved, but they are not perfect – the mathematics cannot yet mirror real-life. Not even for the Red River, where the National Weather Service has some hard-won experience.
Interesting stuff, but what are the conclusions? Over-reliance on forecasting models can cost money and lives. We are still learning the dynamics of how our world works. Sometimes, we are so excited about predicted possibilities we don’t take into account the most obvious of factors. Over-reliance on models is part of the mortgage crisis. There is more than appears to the eye at work in the economy; in general people aren’t able to actually make the chain between cause and effect. If mathematics can’t guide your every decision, what can go beyond models?
Our inability to draw a perfectly straight line between cause, truth and effect leave us with a variety of possibilities in between. Aside from knowing and acknowledging the limitations of the models that can predict where that straight line may fall, we can also try to account for the many options in between. This is strategy. Rather than blindly trusting in models, we look at the range of possibilities, the consequences of each and try to plan around those outcomes. In the case of the Red River, it means building the walls of sandbags higher than the possible “loop” of the river’s crest. In the case of the mortgage crisis, it means looking at the possibility of house price declines building in some element of risk to buyers that performance is not guaranteed. And in the case of Jay Leno’ psychic, it means not giving up your day job just yet.
"Giving up the illusion that you can predict the future is a very liberating moment. All you can do is give yourself the capacity to respond. The creation of that capacity is the purpose of strategy." -- Lord John Brown, British Petroleum.
