THE SOUTH AFRICAN INDEX INVESTOR
“Greater rewards, lower costs”
Developing Better Foresight
By The Index Investor (www.indexinvestor.com)
One of the most frequently heard comments about the crash of 2008 is, “I didn’t see it coming.” This raises a critical question: How can you improve the accuracy of your financial forecasts, or, more broadly, the quality of your foresight?
We believe the answer to this question begins with understanding the nature of the system whose behaviour we are trying to predict. At one extreme, physical systems are characterized by relationships defined by the laws of physics and chemistry that are stable over time. It should therefore be possible to use a single model to forecast the behaviour of such a system with a high level of confidence over both short and long time horizons. Moreover, knowledge of this system’s past behaviour can be used to accurately specify the values for the variables used to model its future behaviour.
At the other extreme, social systems – like financial markets -- are populated by thinking, feeling, and socially interacting agents who adapt their behaviour and goals as events unfold, causing the underlying relationships that drive system behaviour to be both complex (e.g., multiple causes for an effect, positive feedback loops and non-linear relationships between causes and effects, and wide time separation between causes and effects) and unstable over time. This system presents forecasters with a far more difficult challenge.
First, because of the system’s complexity, there is an irreducible level of uncertainty associated with the identification of the variables to include in a forecasting model, and the specification of the relationships between them. Second, once one has developed a forecasting model, accurately estimating the future values of the included variables and relationships presents a further challenge – because the system constantly evolves, knowledge of historical values may provide a poor guide to what lies ahead, particularly as the forecast time horizon lengthens. Third, it is often the case that forecasting models and their users are themselves part of the process that drives the evolution of a complex adaptive system. For example, a model that accurately forecasts the price of an asset can be discovered by others, whose subsequent use of the model changes the underlying relationships and competes away its ability to generate profitable predictions.
Beyond understanding the nature of the underlying system, there is the equally challenging issue of the nature of the forecasters themselves. To varying degrees, all human beings are affected by factors that reduce the accuracy of the forecasts they make. Perhaps the most important of these are the so-called “anchoring”, “confirmation” and “overconfidence” biases.
Anchoring refers to our tendency to insufficiently adjust our forecasts when we receive new information. Confirmation refers to the tendency to pay more attention, and give greater weight to information which supports our current forecast, and less to information which contradicts it. Other studies have repeatedly found that many forecasters are overconfident – when asked to provide a range that includes 80% or 95% of possible outcomes, most people provide answers that are too narrow compared to actual results. Put differently, we tend to underestimate volatility and variance, and how they compound over time. Finally, recent research in neurobiology has found that increased uncertainty triggers feelings of fear, as well as stronger desire to avoid social isolation. Put differently, when uncertainty rises, we have a natural tendency to follow the herd, and accept the conventional wisdom about what lies ahead. While that was undoubtedly advantageous eons ago when our ancestors were trying to survive on the East African savannah, it often works to our disadvantage when we are trying to survive and prosper in financial markets.
What can investors do to overcome the challenges they face, and improve the accuracy of their financial forecasts? We believe they should keep three important points in mind. First, they can align the focus and confidence level of their forecast with its time horizon. As we have repeatedly noted, when forecasting the behaviour of a complex adaptive system over a long period of time, an analyst should have more confidence in a “strategic warning” for “what” may happen and “why”, than in an operational warning about “how” something might occur, much less a tactical warning about “when, who and where” an event will take place. Over time, the number of tactical possibilities compounds much faster than the number of operational possibilities, which in turn grow faster than the number of possible strategic outcomes. For this reason, models with a short-term forecasting horizon can emphasize their fidelity to historical data as evidence of their likely accuracy. In contrast, for forecasts with longer term horizons a high fidelity to historical data indicates low robustness to uncertainty, which should cause an analyst to have less confidence in its predictions.
These conclusions are generally in line with what we observe in financial markets, where short term tactical trading models are often highly quantitative and based on recent investor behaviour, while long term asset class allocation models focus on fundamental valuation and economic considerations. In the middle lie security and sector investment selection models, which usually include a mix of variables related to fundamental valuation and investor behaviour.
Second, as studies have repeatedly shown, investors can increase the accuracy of their predictions (and overcome their confirmation bias) by actively seeking out and combining different forecasts. While there are many complex techniques for weighting different forecasts, researchers have found that simple averaging often works surprisingly well, provided that the forecasts are based on different underlying methodologies. This is a critical point, as multiple studies have found that professional forecasters have a tendency to herd The key benefit of forecast combination is that it tends to cancel out some of the model specification and parameter estimation errors in the individual methodologies. Studies showing the benefits of forecast combination are closely related to other research which has found that confidence in a prediction increases when forecasts based on different methodologies reach similar conclusions.
The third technique that can improve the quality of an investor’s foresight is to always ask these four questions of any forecast he or she makes or receives: What are the critical assumptions upon which it is based? Which of these are the most uncertain? What indicators will tell me they are not turning out as expected? And where should I look for them? The inescapable fact is that our ability to pay attention to information is limited by time and neurobiology, and is further challenged by the deluge of data that technology delivers to us each day. In today’s world, taking a passive approach to the allocation of your scarce attention is likely to reduce the quality of your foresight.
In sum, accurately forecasting the behaviour of a complex adaptive system like a financial market is an extremely difficult task, particularly as the time horizon grows longer. Yet it is still possible to improve one’s foresight, and to improve your ability to avoid the painful losses and regrets that so many investors have recently experienced.
Monthly Insight, July 2009.