Beware of Backtested Market-Beating Strategies

As a general rule, if somebody is recommending a given strategy to you as a method for outperforming the market, you should neither be surprised nor impressed to learn that the strategy has impressive historical results.

If you get to make up as many strategies as you want, then backtest them all to see how they worked, you will come up with something that performed well. But that doesn’t mean anything about your skill at outperforming the market. Nor does it mean anything about the likelihood that the historically-winning strategy will continue to be successful in the future.

For example, if an advisor is trying to sell you a portfolio of actively managed funds, you can bet that the funds will have market-beating track records. But, if all we care about is historical results, it would only take a few minutes to come up with a portfolio of 3 individual stocks that beats the pants off the advisor’s active-fund portfolio. Does that mean that, going forward, the 3-stock portfolio is a better bet than the advisor’s mutual fund portfolio — and that both options are a better bet than an index fund portfolio? Of course not.

What if there is a convincing explanation for the historical data?

Consider the following piece of information:

Fund manager age and fund performance have a significant positive correlation. (On average, for each additional year of age, performance improves by almost 0.09%.)

Interesting isn’t it? That’s a sizable difference. And it’s easy to think of reasons why this effect might not be purely random (e.g., older managers have wisdom that comes with age, they have more developed social networks that allow them to get information more quickly, etc.).

Well, guess what?

I lied. (Please forgive me; I’m trying to illustrate a point.) According to the only study I’ve found on the topic, it’s actually the younger managers who tend to have better performance. And now that I’ve told you that — despite it being the exact opposite of what I said above — it’s still easy to come up with a credible-sounding explanation (e.g., younger workers have more energy and motivation to get ahead in their careers, they’re more up-to-date with various useful technologies, they’re more willing to take risks, etc.).

By definition, even if it’s the result of pure randomness, some age range is going to have the best-performing fund managers. And no matter what age range it is, our human brains will have no trouble coming up with an explanation for why that phenomenon makes sense and why it could be expected to continue.

As humans, we look for patterns. And when we find one, we try to identify a cause — even when there’s no cause to be identified. We’re naturally susceptible to being “fooled by randomness.”

Maintain Your Skepticism

If a person has anything to gain by convincing you of his ability to beat the market (e.g., a commission, your assets under his management, or your subscription to a paid newsletter), it would only be surprising if he did not have a strategy with impressive historical results and a convincing explanation.