"We Are All Just Kind Of Guessing"

by Mitch Battros  (ECTV)

The article below by Sam Savage makes the point of subjective conjecture in many fields including science. Here is one statement that stands out. "While many of today`s managers still cling tenaciously to ``flat earth`` ideals, the innovators are abandoning averages and facing up to uncertainty. Those who dare discover a New World of managerial tools including simulation, decision trees, portfolio theory and real options".

Perhaps those in our government agencies are catching on, and finally realizing "we are all just kind of guessing"  (mb)
_________________________

The Flaw of Averages
Sam Savage - Standford University News

``The only certainty is that nothing is certain.`` So said the Roman scholar Pliny the Elder. And some 2,000 years later, it`s a safe bet he would still be right. The Information Age, despite its promise, also delivers a dizzying array of technological, economic and political uncertainties. This often results in an error I call the Flaw of Averages, a fallacy as fundamental as the belief that the earth is flat.

The Flaw of Averages states that: Plans based on the assumption that average conditions will occur are wrong on average.

A humorous example involves the statistician who drowned while fording a river that was, on average, only three feet deep.

But in real life, the flaw continually gums up investment management, production planning and other seemingly well-laid plans. The Flaw of Averages is one of the cornerstones of Murphy`s Law (What can go wrong does go wrong).

Fortunately, superfast computers can overcome this problem by bombarding our plans with a whole range of inputs instead of single average values. Today, this technique, known as simulation, is at the center of such diverse activities as Wall Street investing and military defense planning.

But back to the flaw, and an area that`s important to all of us: investing for the future.

Suppose you want your $200,000 retirement fund invested in the Standard & Poor`s 500 index to last 20 years. How much can you withdraw per year? The return of the S&P has varied over the years but has averaged about 14 percent per year since its inception in 1952. You use an annuity workbook in your spreadsheet that requires an initial amount ($200,000) and a growth rate for the fund. ``I need a number,`` you say to yourself, so you plug in 14 percent. Now you can play with the annual withdrawal amount until your money lasts exactly 20 years. If you do this you will be pleased to find that you can withdraw $32,000 per year (see Figure A).

Figure A. Funds remaining with annual withdrawal of  $32,000, assuming 14% return every year.

Even if the return fluctuates in the future, as long as it averages 14 percent per year, the fund should last 20 years, right?

Wrong! Given typical levels of stock market volatility there are only slim odds that the fund will survive the full time. The following charts simulate how this retirement strategy would have worked with actual S&P 500 returns starting at various points in time (see Figure B).

Start: 1973 Avg. Return 14% Tanks in 8 yrs.
Start: 1974 Avg. Return 15.4% Goes the distance.
Start: 1975 Avg. Return 15.4% Tanks in 13 yrs.
Start: 1976 Avg. Return 15.3% Tanks in 10 yrs.
 

Figure B. Simulated Fund performance if started in various years.

Notice that the level of average returns over any particular 20-year period is no guarantee of success. The real key is to get off to a good start. What separates 1974 from its neighbors is that the period started with two years of good growth, giving the nest egg just enough extra heft to weather the future storms.

For this example the Flaw of Averages states that: If you assume each year`s growth equals the average of 14 percent, there is no chance of running out of money. But if the growth fluctuates each year while averaging 14 percent, you are likely to run out of money.

The example above is not the result of a rigorous scientific study, and should not be used for making investment decisions, but it should at least have you asking yourself: Why isn`t someone doing something about this? People are. One of the first was William F. Sharpe, a Nobel laureate in economics, who recently left Stanford to spend full time simulating retirement benefits. ``I expected people to question the specifics of our simulation algorithms,`` reflects Sharpe about the launch of Palo Alto-based Financial Engines Inc., ``but to my surprise, everyone else out there was just plugging in averages.`` (As in Figure A.)

The Flaw of Averages distorts everyday decisions in many other areas. Consider the hypothetical case of a Silicon Valley product manager who has just been asked by his boss to forecast demand for a new-generation microchip.

``That`s difficult for a new product,`` responds the product manager, ``but I`m confident annual demand will be between 50,000 and 150,000 units.``

``Give me a number to take to my production people,`` barks the boss. ``I can`t tell them to build a facility with a capacity of between 50,000 and 150,000 units!``

So the product manager dutifully replies: ``If you need a single number, the average is 100,000.``

The boss plugs the average demand and the cost of a 100K capacity fabrication plant into a spreadsheet. The bottom line is a healthy $10 million, which he reports to his board as the average profit to expect. Assuming that demand is the only uncertainty, and that 100,000 is the correct average, then $10 million must be the best guess for profit. Right?

Wrong! The Flaw of Averages ensures that average profit will be less than the profit associated with the average demand. Why? Lower-than-average demand clearly leads to profit of less than $10 million. That`s the downside. But greater demand exceeds the capacity of the plant, leading to a maximum of $10 million. There is no upside to balance the downside.

This leads to a problem of Dilbertian proportion: The product manager`s correct forecast of average demand leads to an incorrect forecast of average profit, so on average he gets blamed for giving the correct answer.

A computerized cure for the Flaw of Averages is Monte Carlo Simulation, first used for modeling uncertainty during development of the atomic bomb. It generates thousands of scenarios covering all conceivable real world contingencies in proportion to their likelihood.

In the 1950s, Harry Markowitz, a brash young graduate student at the University of Chicago, dealt another blow to the flaw.  ``I was reading the contemporary investment theory, which was strictly based on averages,`` recalls Markowitz. ``I said to myself: "This can`t be right." His resulting portfolio theory, which was based on both risk and average outcomes, revolutionized Wall Street and won him a Nobel Prize. Markowitz also devoted much of his career to designing simulation systems.

Simulation-based acquisition is now used routinely in the military. Its instigator was William J. Perry, who in spite of a bachelor`s degree, master`s degree and doctorate in math, has had a remarkably well-rounded career as a Silicon Valley entrepreneur, U.S. secretary of defense and Stanford professor.

In 1996, while at the Pentagon, Perry issued a directive stating that models and simulations must be used to reduce the time, resources and risks of the acquisition process. Perry says in retrospect: ``With tens of thousands of uncertainties, it was just a perfect application for simulation.``

A dramatic example of the savings that resulted from Perry`s directive is related by John D. Illgen of Santa Barbara-based Illgen Simulation Technologies Inc., who says: ``In response to improvements in foreign weapon systems, the Navy was preparing to spend tens of millions of dollars to upgrade its shipboard defensive systems. With a $250,000 simulation we were able to show that the present defensive system was adequate to meet the increased threat.``

While many of today`s managers still cling tenaciously to ``flat earth`` ideals, the innovators are abandoning averages and facing up to uncertainty. Those who dare discover a New World of managerial tools including simulation, decision trees, portfolio theory and real options.

And what happens when one of these innovators is confronted by someone cloaking himself behind a single number? The story of the emperor`s new clothes says it all.
__________________________________

CONTACT: Dawn Levy, News Service (650) 725-1944
e-mail: dawnlevy@stanford.edu

COMMENT: Sam Savage, Management Science & Engineering
(650) 723-1670; e-mail: savage@stanford.edu