Current investment practices are all too often an outgrowth of internal politics and history, steeped in bureaucracy and resistant to change. Unwieldy committees of well-intended fiduciaries attempt to deal with rapidly changing markets by evaluating performance, with flawed methodologies, over too short a time horizon. Failing to realize their targets of first quartile performance and frustrated with disappointing results, funds quickly dispense with their current manager complement, in many cases seeking out last year's fad.

Clearly a new process is needed. The key is fiduciary communication. The myth of comparative analysis (universe comparison) must be stripped from the fiduciaries' repertoire and the limitations of quantitative analysis exposed, so that fiduciaries firmly embrace the precept of qualitative analysis, ratified by quantitative performance. Although greater resources, time and tools are necessary, the end result is clearly superior decision making, with lower pension impact costs.

Current Practice
Before putting forth a solution, let's begin with an analysis of current practice.To evaluate individual manager performance, most Canadian fund sponsors use a combination of quantitative analysis and intuition, over a four-year time horizon (the stated norm for evaluating plan and manager performance, according to the typical Canadian Pension Statement of Investment Policies and Procedures. In practice, however, most investment committees' pain threshold commences at about two years and action is generally taken well before the stated four-year evaluation period. But let's not quibble over time horizon; as we'll see, even four years is too short).

Why do fund sponsors use this approach? Quite frankly, it is the accepted norm, readily available, easy to use, naively appealing, and relatively inexpensive.

Everybody likes the idea of watching a performance horse race and witnessing the definitive winner emerge, based on a finite quantitative test. After all, it's far less complex than analyzing a multitude of qualitative and quantitative variables and painstakingly trying to interpret the best solution for your fund.

But there are two main flaws with the current evaluation process:
* Quantitative analysis is not predictive and is often inaccurate; and
* Group intuition is subject to several behavioural impediments and is not objective.

Quantitiative Analysis
One of the most common tools used in quantitative analysis is universe median comparisons, or relative performance measurement. Unfortunately universe median comparisons are poor benchmarks, as they are ambiguous, not specified in advance, not investable (can't be replicated), not reflective of current investment opinion, and inappropriate due to varying styles, asset mix and plan risk tolerance.1 In addition, they are subject to survivorship bias.2

The rest of the quantitative tools (benchmarks, scattergrams, tracking error, value added charts, information ratios and attribution analysis) are at least helpful, but are not predictive or consistent,3 and are unable to discern luck from skill without a great deal of data and time, both of which are often unavailable.

The latter point is particularly relevant, given the four-year time horizon generally used to assess managers. Is there any magic to four years? No. The concept probably evolved from the four-year presidential cycle, which most people assumed coincided with the business cycle and, therefore, capital market returns.

According to Statistics Canada, there have been six full Canadian business cycles between 1960 and 1998. The average business cycle lasted about 6.3 years, while the longest was almost nine years. In fact, the last two have each been over eight years. Only the shortest, at 3.75 years, is close to the evaluation period used by most investment plan sponsors.4

Given that four years is not a useful time frame, how long does it take to discern luck from skill on a pure quantitative basis?

The chart below shows the required time horizons, by asset class.

With a minimum of at least 16 years required to statistically determine if managers can add value due to skill, we see that what plan sponsors are measuring over a shorter time period is just white noise. This demonstrates the futility of using quantitative analysis over a four-year time horizon to evaluate manager performance. Is it any wonder that, on average, pension funds fail to add value to their passive composite benchmarks?5

Group Intuition, A Recipe for Disaster
The second major flaw associated with current investment plan practices is the group dynamics associated with investment committee decisions. The following observations reflect my personal experience, but are not meant to be all-encompassing, nor do they necessarily apply to all fiduciaries.

Firstly, high-level fiduciaries are well intentioned, but they usually have busy, full-time careers. Their services on behalf of the pension or endowment plan are often on a volunteer basis. Although in the majority of cases they perform their fiduciary functions in a conscientious manner, they are often time-sensitive and can be impatient to return to their pressing full-time endeavours.

Secondly, fiduciaries have varied backgrounds and in many cases are experts in their own field. While this is helpful in bringing different perspectives to the investment committee table, few have pension expertise and most reveal their bias for business and corporate profit. Issues such as time horizon are considerably different for pension and endowment funds, which are long-term in nature, and business enterprise, where four quarters is an eternity. Used to taking charge and making a short-term impact in the corporate sector, these captains of industry want simple answers to what in many cases are complex investment issues.

Another trait is their suspicion of the new and unknown. They constantly look for problems or the downside, which is only natural, given the potential for personal liability. However, this may impede fund progress. Success in many instances belongs to those who conduct good research, do proper due diligence and implement on a timely basis. Investment funds need to be one step ahead of mercurial markets and to keep pace with new investment vehicles such as derivatives, asset-backed securities and inflation-linked securities.

Finally, fiduciary turnover may be an issue for many investment committees. New members, who did not take part in the initial board in the existing policy and strategic decisions, must be shown their merits and properly indoctrinated. A good paper trail, such as a statement of investment beliefs, reviewed with the Chair and Sponsor, as well as formal education, would surely assist in this indoctrination process.

Group Dynamics
There are four significant behavioural impediments exhibited by Investment Committees:

* Overconfidence and Irrationality
* Portfolio Segregation
* Reference Dependence; and
* Personal Agenda.These arguments can best be summarized by Tversky. His analysis indicated that classical investment theory called for asset integration, risk aversion and rational expectations, but it is probably behavioural psychology which explains fiduciary practices of portfolio segregation, loss aversion, and biased expectations.6

Overconfidence and Irrationality
Studies have shown that most people are overconfident in their daily lives. In one study, 70% of males said they were in the top quartile in leadership qualities.7 This overconfidence often spills over into investment management and may lead to irrational or inappropriate decisions. This is generally manifested in two primary forms:

Biased fiduciary opinions, often based on one or several of the following:
* extrapolating last year's winning strategy into the future, without objective analysis;
* loss aversion and a failure to recognize their own (lower) risk tolerance as a public fiduciary, due to the pressures of higher visibility;
* their personal experience in managing a business on a quarterly profit basis, while failing to understand the long-term nature of the investment business; and
* subjective manager evaluation, usually determined by form (presentation skills), not substance (predictive ability to add value), often based on an annual, one hour, formal economic presentation from the investment firm.Lack of formal fiduciary pension training, resulting in:
* fiduciaries, in positions of authority, not having completed a comprehensive investment course;
* investment policies and strategies lacking accountability, conviction, commitment and an adequate paper trail;
* a failure to heed professional advice from staff and consultants, due to overconfidence in their own abilities (based on a gut feel for the market); and
* poor implementation/planning and a failure to understand its importance, effectiveness, cost efficiency or plan impact, which often leads to inadequate research, resources and staffing.

Portfolio Segregation
Portfolio segregation is the penchant for fiduciaries to dwell on their worst investment decisions, forgetting that the whole can be greater than the sum of its parts. In effect, they regret having invested in poorly performing elements, when in reality the investment decision was made in a sound portfolio, or asset class, context. Two examples of this would be:

* evaluating a manager's performance in isolation, without reviewing the rationale for the asset class strategy, objectives, structure and, in particular, the manager's role within that structure; and
* failing to focus on fund performance relative to the plan objectives.

Reference Dependence
Reference dependence is defined as a choice between two assets, where the investor's decision depends on their reference point (where they are now).8 Fiduciaries evaluate the performance of managers in terms of the gains and losses at a point in time, relative to some benchmark. They fail to recognize that benchmark outperformance comes sporadically, not in neat, quarterly value-added packets. Thus, for example, if a Canadian equity manager was hired to add 100 basis points (bps) or 1% value added over the TSE 300 per annum, the value-added component would probably not be derived by adding 25 bps to the benchmark each quarter. But it could appear by outperforming the TSE 300 by 1000 bps (10%) in a quarter, following five years of underperformance of 100 bps per annum.

Fiduciaries ignore managers with low recent returns and, therefore, fail to engage good managers whose styles may be currently out of favour.9 Yet hiring a good investment firm at the bottom of their style cycle is akin to buying at the market low, an excellent way to add value to the fund. While this is logical and increases the probability of success, few fiduciaries can overcome the trap of reference dependence.

Personal Agendas
Many investment committees terminate quality investment managers, currently underperforming on some periodic basis (two to four years), to alleviate the discomfort of reporting poor results to a Board of Trustees for several quarters in a row. The Investment Committee Chair wants to be viewed as implementing positive change, so falls prey to his own internal doubts and fires the manager. At the next Trustee meeting the results may still be bad, but the Chair can demonstrate action and probably buy himself two more years of breathing room. The professional staff and consultant do not oppose these irrational decisions by Trustees, because personal interest takes hold and they remain silent to limit career risk.

The Better Way - Asset Class Evaluation
Our opening premise was that "most Canadian plan sponsors use a combination of quantitative analysis and intuition, over a four-year time horizon, to evaluate individual manager performance." I trust by now you are seriously reconsidering this outdated approach. What, then, is a fund sponsor to do?

Plan sponsors initially attempt to engage managers who satisfy the four P's: people, philosophy, process, and performance. In other words, they perform due diligence. Performance, in the main, is a manifestation of the other qualitative factors; if you hire skilled people with a sound investment philosophy and who implement the process wisely, you will probably receive a fair value-added component over a full business cycle. A proper monitoring plan should therefore focus primarily on people, philosophy and process in its ongoing due diligence process. But in most plans it doesn't, because it requires additional resources, either internally or through external third party services, and this need is not widely recognized.

The consistency of the value-added return component is determined by how you combine the managers. An attempt is made to pick managers whose styles complement each other. Risk is diffused by the multi-manager configuration and its reasonable benchmark consistency based on capitalization, style, etc. However, in practice, the actual size of the fund, economies of scale, and depth of the market and managers may impose a limit to the number of managers that can be engaged cost effectively by the plan. These limitations will subsequently create gaps and biases in the asset class structure. The greater the gaps, the less controlled the risk and the greater the time horizon necessary to evaluate its success.

Fiduciary Communication
The key is to convey these asset class biases to the fiduciaries in an understandable, proactive manner. Consultants and professionals who deal in investment matters daily often use buzzwords such as beta, variance or standard deviation to describe risk. It is assumed that these terms are understood by the part-time fiduciaries. Often it is only later, at some critical juncture, that we recognize they were not. We need to provide fiduciaries with practical, understandable examples of these complex terms. Then, with a better comprehension, they can deal with the issues or decide to live with the risks.

To help explain risk and other technical terms, we often use actual strategies and results of our clients (on a no-name basis) as examples, to provide fiduciaries with a more relevant perspective. One such example used to describe manager volatility has been outlined below:

Client A has a moderately high risk tolerance, with a composite Canadian equity manager structure composed of:
* a large cap market-oriented, core manager representing 75% of the asset class structure;
* a midcap value manager of considerably higher risk for 12.5%; and
* a small cap growth manager of high risk for 12.5%.
* Client B has a much higher risk tolerance and lesser assets under management to diversify risk (greater structural bias), composed of:
* a large/midcap growth manager of high risk representing 50% of the structure; and
* a small/midcap market-oriented manager of high risk for the balance.To reinforce the verbal explanation, graphic mapping of the two clients' risk exposure is provided on a Canadian equity style matrix, as outlined in Exhibit I. The fiduciaries are then told that for the five-year period ended Dec. 31, 1996, both funds earned about 200 bps value added per annum over the TSE 300. (Achievement of this value-added target is usually considered "Nirvana" in the Canadian equity environment.)

Finally, we present what is referred to as the 'Investment Committee Stress Test,' as shown in Exhibit 2, which reveals how the value added was derived. It also shows the worst consecutive four, eight and 12-quarter periods of value lost by each of the individual managers and the respective client asset class performance. The concept of risk is readily visible and the following conclusions can be demonstrated:

* Attaining 200 bps value added in Canadian equities over five years is not easy;
* To achieve this target one must accept considerable underperformance risk, by individual managers and at the asset class level over extended periods;
* The manager structure of the portfolio is important and each manager should be evaluated within the context of that structure;
* The greater the manager structure diverges from the benchmark the more likely the opportunity for outperformance and the need to accept downside risk over the course of the investment cycle; and
* Assuming good managers and proper due diligence, time is often the best healer (due to mean reversion).All of the aforementioned are valuable educational points which can assist fiduciaries in gaining greater understanding of a complex issue.

This type of practical experience can also be easily applied. When selecting prospective new managers, it is quite simple to analyze historic return performance to evaluate the maximum (four, eight or 12-quarter) downside volatility, to determine if the fiduciaries could endure the roller coaster ride associated with the manager. The potential downside should be recorded as an indication of the expected volatility of future performance and the rationale for proceeding minuted. The minutes can then be resurrected when fiduciaries are too quick to pass judgment based solely on short-term, negative-return performance. After all, the worst case was contemplated when entering into the strategy, so why panic if that which was considered, actually occurs.

A Preferable Monitoring and Performance Evaluation Process
An effective monitoring process should be similarly easy to comprehend and explain. A process should be implemented that tracks the managers' portfolios individually and on an aggregate basis, relative to the respective asset class benchmark. This will identify gaps and assist in controlling risk. It is also important to regularly ensure the managers do not stray too far from their mandates and that their people, investment process and philosophy remain consistent and of high quality. Hence, a more preferable due diligence package will incorporate three broad elements, which address all of the factors noted. Exhibit 3 shows such a process.

Think of this performance measurement and monitoring system as a three-legged stool. Unfortunately most sponsors are currently balancing on a single leg. They feed their fiduciaries only core performance data, which with no countervailing arguments often leads to needless change and turnover.

It doesn't take a rocket scientist to determine that the structure or manager has performed poorly in the past. What fiduciaries need is to discern whether the structure has the potential to add value in the future. They require insight into why the portfolio has underperformed, and reasonable assurance that the managers should add value going forward, to help stretch their evaluation time frames and overcome their behavioural frailties such as the natural inclination for change. The three-legged stool approach, which adds the additional components of portfolio profile analysis and continued manager research to core performance measurement, is a superior approach which provides objectivity, sound judgment criteria, research and a better understanding of fund, asset class and manager performance.

Endnotes
1. "Manager Universes: The Solution or the Problem," A.I.M.R. Performance Evaluation, Benchmarks, and Analysis, June 1995, J.V. Bailey.
2. "Survival of the Fittest," Canadian Investment Review, Winter, 1995, B. Curwood, Steve Hadjiyannakis, Paul Halpern and Kathleen Taylor.
3. "The Information Ratio: More Than You Ever Wanted to Know About One Performance Measure," Russell Research Commentary, October 1997, Thomas H. Goodwin and also "Past Performance is No Guarantee of Future Results," Russell Research Commentary, August 1997, Ernest M. Ankrim.
4. "Performance Measurement - The losers game," unpublished article by Harry Marmer - Director, Investment Funds, Frank Russell Company.
5. "Improving Pension Fund Performance," Financial Analysts Journal (A.I.M.R.), November/December 1998, Ambachtsheer, Capelle and Scheibelhut.
6. "The Psychology of Risk," Quantifying the Market Risk Phenomenon for Investment Decision Making (A.I.M.R.), 1990, Amos Tversky.
7. "Behavioural Risk: Anecdotes and Disturbing Evidence," Investing Worldwide (A.I.M.R.), 1996, Arnold Wood.
8. "The Psychology of Risk," Quantifying the Market Risk Premium Phenomenon for Investment Decision Making (A.I.M.R.), 1990, Amos Tversky.
9. The Portable Pension Fiduciary, 2nd Edition, The Frank Russell Company, John H. Ilkiw.

Bruce Curwood, MBA, CFA, CCM, CIMA, is a senior consultant at Frank Russell Company in Toronto.