The Problem with Market Timing Research

I recently took a continuing education class about retirement planning. The instructor was super enthusiastic about a recent piece of research that showed that, based on historical data, retirees’ ability to safely spend from their portfolios would be improved if they followed a set of rules in which they dramatically adjust their asset allocation each year based on current interest rates and price-to-earnings ratios.

Personally, while I find such research interesting, I don’t think I’d be able to trust it if it were my own money on the line.

My qualms can be best explained by an analogy I heard several years ago (though I don’t remember where).

Imagine that we want to test whether eating various types of mushrooms improves health outcomes among people with pancreatic cancer. So we find 50 people with pancreatic cancer who are willing to participate in a study, and we find 50 different species of mushrooms. Each person in the study gets one type of mushroom, and they eat that mushroom everyday for lunch. Person 1 eats Mushroom A everyday, Person 2 eats Mushroom B everyday, and so on.

As it turns out, Person 17’s cancer quickly goes into remission, while he’s eating Mushroom Q every single day.

What does that tell us? Not a lot, really. Mushroom Q might be useful as a treatment. Or perhaps Person 17 was just a lucky individual, and it has nothing to do with mushrooms.

Fortunately for the sake of our research, mushrooms (at least the ones we’re testing) are harmless and not especially expensive. In addition, there are lots of people with pancreatic cancer. So we could do a study in which many people with pancreatic cancer are given Mushroom Q (while others are given a placebo), and we see what happens. In just a handful of years, we’d have a decent idea of whether this mushroom really has any effect or not.

Unfortunately, for two reasons, this type of research isn’t really possible for market timing strategies.

Problem #1

With investing, somebody can look at historical data and come up with a strategy that would have worked wonderfully in the past. This is like the first part of our mushroom research in which we test a bunch of different mushrooms. But we can’t go out and do a large number of simultaneous tests of the new strategy. For example, if we want to test how a certain market timing strategy affects likelihood of portfolio depletion over a 30-year retirement, we have to wait 30 years to get any new independent data points.

And by then:

1. Your own personal opportunity to benefit from such rule has likely expired, and
2. There’s a decent chance that markets have changed in some way that invalidates the conclusion anyway. (When successful market-beating strategies become commonly known, they usually stop working as the market adjusts.)

Problem #2

Part of what makes our hypothetical mushroom trials feasible is that all of the people involved can still get whatever medical care is currently considered to be the best for fighting their cancer (e.g., radiation, chemo, surgery, etc.). That is, the patients don’t have to bet their lives on mushrooms being more effective than chemo/radiation/surgery.

With investing, current conventional wisdom says that using a “strategic asset allocation” strategy is the best approach (i.e., an allocation that varies with your life circumstances but not with market indicators). You cannot simultaneously use that type of strategy and a “tactical asset allocation” strategy in which you vary your allocation based on your chosen indicator(s). It’s one or the other.

In other words, to the extent that you want to try out the new approach, you have to abandon the conventional wisdom approach. There’s no “chemo, plus eat some mushrooms” option here.