It was with some irony that the first Canadian Investment Review Risk Management Conference was held on the one-year anniversary of one of the most significant market crises of the century. On Aug 27, 1998, one year prior to the date of the conference, the TSE35 fell 7.4% and began a three-day sell off that totaled 11.5%. During that same three-day period the S&P500 lost 12.1% of its value and CBOE Volatility Index, a proxy for the market perception of implied option volatility, moved from 31% to 48%.

Despite unprecedented investment in sophisticated risk management systems, many hedge funds, investment dealers and commercial banks experienced extraordinary losses during that period. For example, Long Term Capital Management (LTCM) lost 44% of their capital in the month of August. Other financial institutions experienced similar losses. Table 1 presents the change in revenue from 2nd to 3rd quarter of 1998 for Long Term Capital Management, Salomon Smith Barney, J.P. Morgan and Merrill Lynch.

Table 1 illustrates the enormous magnitude of the losses experienced in the 3rd quarter of 1998. More shocking, however, is the size of the losses in comparison to the quantity of market risk on the books of these firms as indicated by their Value at Risk, or VaR statistics.

VaR is a widely applied measure of market risk. The VaR of a portfolio is the level of loss in market value that should not on average be exceeded with a given probability (typically 99%). In other words, the VaR is the dollar value loss that should not be exceeded 99% of the time.

LTCM put enormous reliance on VaR, as did other market participants. In fact, the Bank for International Settlements (BIS) requires that a version of VaR be used by all commercial banks to establish the amount of capital required to support their market risk taking operations. As a result of this requirement many commercial banks have made VaR the central component of their risk management system. In addition to commercial banks, investment dealers actively use VaR to measure, monitor and limit market risk taking.

LTCM and Merrill Lynch experienced losses approximately 40 times the size of their VaR, Salomon Smith Barney about 30 times their VaR and J.P. Morgan 25 times their VaR statistic. Table 1 reports the probability of incurring losses of this magnitude under the assumption that market returns follow a normal distribution.

The loss incurred by LTCM is statistically equivalent to consecutively flipping 768 heads in a fair coin toss, a feat that would take 4 hours and 16 minutes at a speed of 1 flip per 20 seconds. This can be contrasted to the experiment held during the conference in which approximately 40 participants each tossed a fair coin and recorded the number of consecutive heads. Of the of 40 participants, nobody tossed more than five consecutive heads, an outcome with a probability of approximately 3%, a total 150 times less than the 768 consecutive tosses implied by the LTCM 3rd quarter results.

In fact, these losses were less likely to occur, given their estimated VaR, than the probability of a chimpanzee sitting at a computer keyboard and randomly entering the following sequence of 64 key strokes: "there are three types of lies: lies, damn lies, and statistics"- an outcome that has a probability or likelihood of less than 1 in 1026.

There are four conclusions to be drawn from the trading results of these and other firms in 1998:

* The amount of market and credit risk on the balance sheet of these firms during the 3rd quarter of 1998 was significant.

* VaR did not provide an accurate measure of this market risk and of the potential losses for these firms.

* The investment made by these firms in risk management technology failed to prevent or control losses.

* The investment in risk management technology encouraged risk taking that was not properly measured or monitored and ultimately led to significant losses.Ironically, the root of the problem for these firms may have been the very VaR model that was designed to assist firms in avoiding the enormous losses they experienced in 1998. The problem is simple. The assumptions inherent in most VaR models are invalid in times of market crisis. More specifically:

* The assumption that returns follow a normal distribution does not properly reflect the probability of massive market moves, and

* The correlation among financial instruments that results in times of crisis in no way resemble the correlation among the same financial instruments that results in times of market calm.This, combined with the fact that the structural integrity of a portfolio is most tested in time of a crisis, implies that an overreliance on a VaR model as a risk management tool may lead to serious mis-specification of the risk inherent in a portfolio of risky assets. In fact, the firms listed in Table 1 had considerably more "Value at Risk" than indicated by their VaR model. The systematic under-estimation of VaR led to a substantial increase in the level of risk taking and ultimately to significant trading losses among hedge funds, investment dealers and commercial banks.

Risk Attribution
Methodologies such as VaR have distracted risk managers from developing suitable approaches to the management of risk. Risk management is a large-scale data problem. There are numerous risks that can potentially affect the performance of a portfolio. In order to reduce "risk" to a measurable quantity, we typically compute generalized statistics. VaR is such a statistic. However, it is important to keep in perspective what VaR is and VaR is not.

VaR is- an important statistic useful for summarizing certain properties of a portfolio, an important part of a comprehensive risk management system.

VaR is not- the only statistic required to summarize the properties of a portfolio, a substitute for a comprehensive risk management system.

Just as the medical profession does not rely solely on a single diagnostic such as body temperature to determine the health of human being, risk managers should not rely solely on VaR to determine the health of a portfolio of risky assets. Instead, risk managers should apply a variety of methods. Comprehensive risk management systems combine the use of statistical risk measures such as VaR with other techniques such as stress testing, scenario analysis and visualization.

There are two essential steps to determining the risk condition of a portfolio:

* The identification of the risk attributes and valuation attributes of the portfolio, and

* The quantification of the effect these risk attributes will have on the valuation attributes of the portfolio through summary statistics, scenario analysis and stress testing.Risk attribution is the process of identifying and quantifying the risk inherent in a transaction, portfolio or collection of portfolios. Generally the attributes of a transaction, portfolio or collection of portfolios are either risk attributes or valuation attributes.

Risk attributes are the variables that effect the valuation attributes of a transaction, portfolio or collection of portfolios. Interest rates, time to maturity, foreign exchange rates, implied volatility and inflation are examples of risk attributes.

Valuation attributes are the summary statistics that describe the valuation properties of a transaction, portfolio or collection of portfolios. Present value, expected value, VaR, cashflow, duration, convexity, carry and vega are examples of valuation attributes.

Once the risk attributes and valuation attributes are identified, scenario analysis and stress testing techniques are used to vary the risk attributes and study their effect on the valuation attributes. Risk attribution is similar to portfolio performance attribution with a simple distinction: performance attribution assigns past performance to risk attributes, whereas risk attribution assigns future risk to risk attributes. Risk attribution tests the effect risk attributes will have on future portfolio performance. The output of the stress test or scenario analysis is a large amount of data that must be evaluated. The most efficient approach to evaluating the data is through visualization. Using multi-dimensional visualization techniques risk managers can quickly assess the effect that certain risk attributes will have on the performance of their portfolio.

Risk attribution provides simple framework for analyzing the risk of a transaction or portfolio. To illustrate consider the following simple example.

Assume we have a fund with a simple 20-year liability. For years 1 through 10 the annual cashflow of our simple liability is $0. However, for years 11 through 20 the annual cashflow of our liability will be $5mm, indexed to the rate of inflation from the starting date of the liability to the cashflow date. To hedge this liability we form a portfolio of three hedge instruments: 1) a 10 year risk free par bond, 2) a 20 year risk free par bond and 3) a Real Return Bond paying a 4% coupon and maturing in 2021. The objective of the analysis is to determine whether a real return bond is a more effective hedge for an inflation indexed liability than a simple coupon bond.

There are two risk attributes that are important to analyze for this simple hedging problem: interest rates and inflation. Table 2 summarizes the sensitivities of the liabilities and hedge instruments to changes in the two risk attributes.

Obviously, the price of the coupon bonds is sensitive only to changes in interest rates and is not affected by changes in inflation. The liability and the RRB are affected by both interest rates and inflation. Notice that when both inflation and interest rates move together, the value of both the liability and the RRB are invariant to the changes.

Figure 1 provides visually the output of a scenario analysis of the residual risk for our liability hedged with only the 10-year and 20-year bonds. Interest rates and inflation rates were varied independently and the liability and hedge instruments revalued for each scenario. Notice that for changes in interest rates the hedge performs well, but for changes in the inflation rate the bonds provide no hedge at all.

Figure 2 provides visually the output of a scenario analysis of the residual risk for our liability hedged with the 10-year and 20-year bonds and the RRB. Again, interest rates and inflation rates were varied independently and the liability and hedge instruments revalued for each scenario. The hedge performs well for changes in both inflation and interest rates. Figure 3 overlays the bond only hedge and RRB hedge for comparison. For scenarios in which inflation and interest rates move in tandem (are positively correlated) the RRB hedge significantly outperforms the bond only hedge. Using very simple risk management techniques we have determined the usefulness of hedging inflation linked liabilities with real return bonds.

Conclusion
Methodologies such as VaR are but a small component of a comprehensive risk management process. The intense focus on VaR as a risk management tool has in many cases led to a mis-specification of market risk among hedge funds, investment dealers, commercial banks and other managers of financial risk. The result in many cases has been an unmonitored increase in risk taking, which played a significant role in the performance of many market participants in the 3rd quarter of 1998. As well, the obsession with VaR has distracted risk managers from developing more suitable approaches. Risk management is a large-scale data problem. Statistics such as VaR help reduce "risk" to a measurable quantity. But due to the limitations of the assumptions that underlie most VaR methodologies, risk managers should not rely solely on VaR. Instead, risk managers should apply a variety of methods to help determine the risks inherent in their portfolios. Comprehensive risk management systems combine the use of statistical risk measures such as VaR with other techniques such as stress testing, scenario analysis and visualization. Risk attribution provides a simple framework for applying these tools to risk management. Such an approach relies less on simplifying assumptions and provides a visual road map that helps describe the nature of the risk of our portfolios.

Michael Durland is deputy head of the capital markets group, Scotia Capital.