Coping With “Unknown Unknowns”
Coverage of the 2010 Risk Management Conference
BY Philippe Jorion | October 18, 2010
Many financial institutions that experienced large losses during the credit crisis apparently employed sophisticated risk management systems. This has led to generalized criticism of the risk management profession. A headline in The New York Times, for example, declares: “The best Wall Street minds and their best risk-management tools failed to see the crash coming.”
This is a simplistic interpretation, however. A large loss is not evidence of a risk management failure because a large loss can happen even if risk management is flawless. To demonstrate this point, let us classify risks into three categories: “known knowns,” “known unknowns,” and “unknown unknowns.”
Let us start with a flawless risk measurement system, where all the risks are perfectly measured. This implies that the risk manager correctly identifies all the risk factors and properly measures their distribution as well as the exposures of the current portfolio, leading to an appropriate description of the distribution of total profits and losses. Top management then decides on a particular risk-return profile for the business. In this case, losses can still occur due to a combination of bad luck and the fact that management accepted too much exposure.
To deal with these known knowns, investors should measure risks across their entire portfolio to avoid surprises.
Risk management systems, however, can fail for a number of reasons. First, the risk manager could have ignored important known risk factors. Second, the distribution of risk factors, including volatilities and correlations, could be measured inaccurately. Third, the mapping process, which consists of replacing positions with exposures on the risk factors, could be incorrect. A notable example is the failure of credit rating agencies to generate proper credit ratings for asset-backed securities. As a result, many investors experienced large losses on AAA-rated tranches of securities backed by subprime mortgages. This is an example of a flawed risk management process.
To deal with these known unknowns, risk managers must have experience with markets and potential weaknesses in traditional risk measurement systems. Investors should also examine closely risk management practices. Recent regulatory reports demonstrate that risk managers had more expertise and influence in institutions that fared better during the crisis.
In the last category of risks are events totally outside the scope of most scenarios. This includes regulatory risks such as the sudden restrictions on short sales, which played havoc with hedging strategies, or structural changes such as the conversion of investment banks to commercial banks, which accelerated the deleveraging of the industry. In addition, the financial system is a complex network. When assessing counterparty risk, it is not enough to know your counterparty, you need to know your counterparty’s counterparty too. This explains why the Lehman bankruptcy created such financial contagion effects.
Risk management systems are not designed to capture this type of uncertainty. They can only capture measurable risks. In addition, this can only be done at the company level. Evaluating such unknown unknowns will always be a challenge but their existence must be acknowledged. This can lead to higher capital cushions than otherwise. Even so, financial institutions cannot possibly carry enough capital to withstand massive counterparty failures, or systemic risk. In such situations, the bank regulator becomes effectively the risk manager of last resort. Their attempt to create a better understanding of network effect also justifies recent efforts toward more transparency in financial markets.
Philippe Jorion is from the Paul Merage School of Business, University of California at Irvine and Pacific Alternative Asset Management Company (PAAMCO)