The Explanation: Why Indexing ‘Works’


Apr. 20 2018, Updated 9:52 a.m. ET

Four (not mutually exclusive) arguments have been advanced to explain why active managers fail much of the time.


Lower cost is the simplest explanation for the success of passive management. Imagine a market in which all assets are actively managed, and into which a passive alternative is, deus ex machina, inserted. This passive alternative buys a pro-rata slice of every company in the market. Since the passive managers buy a pro-rata share of every stock’s capitalization, their portfolio, in aggregate, will be identical to the aggregate portfolio of the active managers. Before costs, therefore, the passive and active portfolios will have the same return.

Article continues below advertisement

However, active managers’ costs—for research, trading, management fees, etc.—are inherently higher than those of passive managers. Thus, “properly measured, the average actively managed dollar must underperform the average passively managed dollar, net of costs. Empirical analyses that appear to refute this principle are guilty of improper measurement.”13

To illustrate the importance of costs, consider that the average expense ratio for active U.S. equity mutual fund managers in 2016 was 0.82%, compared to only 0.09% for their passive competitors.14 This difference of approximately 70 bps offers investors an automatic advantage for choosing a passive manager versus an active one. The growing popularity of index funds, along with industry consolidation and economies of scale, has the potential to lower the costs of passive vehicles further.

The Professionalization of Investment Management

Investment management is a zero-sum game. There is no natural source of outperformance; the outperformance of above-average investors is offset by the underperformance of below-average investors. “Investors” in this sense encompass not just professional money managers, but any owner of securities. These owners may well be undiversified owners of concentrated positions who are not aware that they’re in a zero-sum game. Indeed, they may not be aware that there’s a game at all.

Article continues below advertisement

For example, imagine a conservative retail investor who owns a few highquality, dividend-paying electric utility companies because he values their relatively secure income stream. Such an investor is a potential source of alpha for every professional manager who is underweight utilities. Similarly, every corporate manager who owns a concentrated position in his own company’s stock is a potential source of alpha for every professional manager who is underweight that industry or company. If professional investors represent a relatively small fraction of a market’s assets, such undiversified amateurs can be an important source of the professionals’ outperformance. The outperformance garnered by professionals, in other words, could be provided by the underperformance of amateurs.15

However, if professionals become the dominant force in a market and amateur investors are relatively unimportant, the game changes—the professionals are now competing against each other. In the U.S., professionals had come to dominate by the mid-1970s, as Ellis’1975 assessment makes clear: “Gifted, determined, ambitious professionals have come into investment management in such large numbers during the past 30 years that it may no longer be feasible for any of them to profit from the errors of all the others sufficiently often and by sufficient magnitude to beat the market averages.”16 This is one reason why, in our view, the 1970s saw so many calls for the establishment of market-tracking index portfolios.

Article continues below advertisement

It’s important here to distinguish between absolute and relative skill. Absolute skill in active investing requires managers to access information and to form, based on some combination of fundamental, technical, and quantitative metrics, an assessment of the difference between a stock’s current price and its true value. To criticize active managers’ performance is by no means to impugn their absolute level of skill.17 But managers don’t operate in a vacuum. Absolute skill may be necessary for success as an active manager, but it is not sufficient. It’s relative skill that determines outperformance and underperformance. It’s not enough to be good at valuing companies; a successful active manager has to be better than his competitors.

If investment management is not unique in this respect, it at least is highly unusual. An average physician may be able to cure most illnesses, and an average lawyer may be a perfectly adequate source of legal representation for most needs. Indeed, below-average physicians and lawyers may still be sources of considerable value to their clients. However, investment management is different: an average investment manager is of no value at all. “Investing is unusual, in that the collective judgement of all the participants (weighted by the amount of money they control) is…available for free…. If a professional investor is to earn excess returns for his client, being good is insufficient—he must be exceptional.18

Market Efficiency

Article continues below advertisement

“In investing, efficiency means that value and price are one and the same.”19 To the degree that price and value correspond, active managers will be unable to generate incremental risk-adjusted returns. The trouble with this convenient formulation, of course, is that while we can easily observe prices, the proper value of any security is always a matter of opinion and subject to dispute.

Eugene Fama coined the term “efficient market” in 1965, defining it as “a market in which prices always ‘fully reflect’ available information.”20 He concluded that stock market prices follow a random walk, causing analysts to be unable to outperform consistently via fundamental or technical analysis. The challenge for advocates of the efficient markets hypothesis is that it’s quite easy to find retrospective evidence of times when value and price did not correspond—for example, during the technology bubble of the late 1990s or immediately prior to the market’s recovery in early 2009.21

What such examples demonstrate is that markets are not infallible. But not even Fama claims infallibility for the efficient markets hypothesis. “It’s a model, so it’s not completely true. No models are completely true. They are approximations to the world. The question is: ‘For what purposes are they good approximations?’ As far as I’m concerned, they’re good approximations for almost every purpose. I don’t know any investors who shouldn’t act as if markets are efficient.”22 And if markets are efficient, active management is fruitless.


Article continues below advertisement

The skewness of stock returns is an underappreciated element in the performance difficulties of active managers. Exhibit 5 is a simple example of skewed returns; we posit a market with five stocks, one of which dramatically outperforms the others.23 We assume that at the beginning of the year, the stocks’ capitalizations are identical, so that the market’s return is 18%, driven by the outstanding performance of stock E.

We can form portfolios of various sizes from these five stocks, as shown in Exhibit 6. There are, for example, five possible one-stock portfolios, four of which underperform the market as a whole. Alternatively, there are also five possible four-stock portfolios, four of which outperform the market as a whole. Since the market, in this example, is up 18%, the average return of the portfolios is always 18%—if the market gives us 18%, it doesn’t matter how we slice it up. What changes is the distribution of returns across portfolios. Holding more stocks increases the likelihood of outperformance.24

Article continues below advertisement

The intuition here is simple: a manager’s picks are more likely to underperform than to outperform simply because there are more underperformers than outperformers from which to choose.25 If returns are positively skewed, more concentrated portfolios are therefore relatively likely to underperform, while more diversified portfolios are relatively likely to outperform. Since most active managers run fairly concentrated portfolios (at least relative to the universe from which they draw their stock picks), if returns in the real world are skewed, that helps us explain active underperformance.

Real-world returns are skewed. We might suspect that there is a natural tendency toward skewed equity returns—after all, a stock can only go down by 100%, while it can appreciate by much more than that. This intuition is confirmed by Exhibit 7, which plots the distribution of cumulative returns for the constituent stocks of the S&P 500 for the last 20 years. The median return was 48%, far less than the average of 215%. Importantly, the positive skew in equity returns demonstrated by Exhibit 7 is not simply a long-term phenomenon: in the 26 years between 1991 and 2016, the average S&P 500 stock outperformed the median 22 times.26

13. Sharpe, William F., “The Arithmetic of Active Management,” Financial Analysts Journal,” January/February 1991, p. 7-9.
14. Collins, Sean, and James Duvall, “Trends in the Expenses and Fees of Funds, 2016,” ICI Research Perspective, May 2017
15. Mauboussin, Michael J. and Dan Callahan, “Alpha and the Paradox of Skill,” July 15, 2013, p. 7.
Ellis (1975), op. cit., p.19
16. See Pastor, Lubos, Robert F. Stambaugh, and Lucian A. Taylor, “Scale and Skill in Active Management,” February 2014.
17. Arbit, Hal, “The Nature of the Game,” Journal of Portfolio Management,” Fall 1981, pp. 5-9. Emphasis added.
18. Mauboussin, Michael J., “The Paradox of Skill: Why Greater Skill Leads to More Luck,” Nov. 14, 2012, p. 12.
19. Fama, Eugene F. “The Behavior of Stock-Market Prices,” Journal of Business, January 1965, pp. 34-105, and “Efficient Capital Markets: A Review of Theory and Empirical Work,” Journal of Finance, May 1970, p.383-417. The weak form of the efficient market hypothesis assumes that current stock prices fully reflect all currently available security market information, so that technical analysis cannot be used to achieve excess returns. The semi-strong form assumes that current prices quickly adjust to the release of all new public information. Prices reflect available market and non-market public information, eliminating the possibility of achieving excess returns using fundamental analysis. The strong form of the efficient market hypothesis assumes that current stock prices fully incorporate all public and private information, so that realizing consistent excess returns is impossible.
20. Mauboussin (2012), op. cit.
21. Chicago Booth Review, “Are Markets Efficient?” June 30, 2016.
22. This example is drawn from Heaton, J.B., Nick Polson, and Jan Hendrik Witte, “Why Indexing Works,” October 2015.
23. Edwards, Tim and Craig J. Lazzara, “Fooled by Conviction,” July 2016. See also Livnat, Joshua, Gavin Smith, and Martin B. Tarlie, “Modified IR As a Predictor of Fund Performance,” October 2015, for evidence that among comparably-skillful active managers, greater diversification is an indicator of better future performance.
24. The challenge for stock pickers is exacerbated when the outperformers include the largest stocks in the index. See Chan, Fei Mei and 25. Craig J. Lazzara, “Degrees of Difficulty: Indications of Active Success,” December 2017, pp. 8-9.
26. We find similar results in other markets. The average stock outperformed the median in 15 of the last 19 years for the S&P/TSX Composite, 13 of 18 years for the S&P Europe 350, 20 of 21 years for the S&P/TOPIX 150, 9 of 16 years for the S&P/ASX 200, and 20 of 20 years for the S&P Pan Asia ex-Japan & Taiwan BMI. For a longer term perspective, see Bessembinder, Hendrik, “Do Stocks Outperform Treasury Bills?” November 2017.


More From Market Realist

  • uploads///AdobeStock_
    Partner Voice
    Why Don’t Kratom Companies Offer Public Stocks?
  • uploads///rupixen com QHmzKeQ unsplash
    Innovation, A.I., and eCommerce within Namaste’s Genius Platform Strategy Will Bridge the Market from Pot 2.0 to 3.0
  • uploads///AdobeStock_
    Partner Voice
    EdgeConneX Is Bringing Private Cloud Connectivity Solutions to Munich
  • uploads///thibault penin AWOlqqsffM unsplash
    Why Netflix Is a Don’t-Touch Stock for Some Investors
  • CONNECT with Market Realist
  • Link to Facebook
  • Link to Twitter
  • Link to Instagram
  • Link to Email Subscribe
Market Realist Logo
Do Not Sell My Personal Information

© Copyright 2021 Market Realist. Market Realist is a registered trademark. All Rights Reserved. People may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.