New Here? Get the Free Newsletter

Oblivious Investor offers a free newsletter providing tips on low-maintenance investing, tax planning, and retirement planning. Join over 21,000 email subscribers:

Articles are published Monday and Friday. You can unsubscribe at any time.

Backtesting Investment Strategies

I just finished reading Jim Otar’s Unveiling the Retirement Myth (mini review: packed with important insights and bizarre hypothetical examples).

Otar explains his approach to investment planning this way:

“Instead of presenting a ‘forecast’ of a client’s future financial picture based on assumptions, [I present] an ‘aftcast’ of client’s potential outcomes based on actual market history.”

In other words, Otar argues that, rather than making potentially faulty assumptions about the future, we should stick to the only real data we have: market history. Otar’s methodology is similar to that of the famous Trinity Study: to determine the viability of an investment strategy, check to see how frequently it would have worked in the past.

I’m in agreement that the fewer assumptions you make, the more well-founded your conclusions will be. But there’s one assumption I’m comfortable making: The future won’t look exactly like the past. And as I wrote last week, there’s a big difference between:

  • “Historically, Strategy X has worked 90% of the time,” and
  • “Strategy X works 90% of the time.”

Conclusions from Hindsight

The fact that an investment strategy (a market timing method, for instance) has notworked historically may be a sufficient reason not to count on it to work in the future. But the fact that a strategy has worked in the past isn’t sufficient evidence that it will work in the future.

When you have the benefit of hindsight (and the computing power and data to back-test any strategy you can come up with), there are an infinite number of investment strategies that will have succeeded based on nothing other than randomness. (Investing based on last year’s butter production in Bangladesh comes to mind.)

Rather than investing based upon back-tested observations like these two provided by Otar:

  • Rebalancing on presidential election years has historically provided the best returns, or
  • If the last year’s market performance was worse than the average of the last six years, next year is likely to be better than average

…I prefer to base my investment strategies on simple concepts. Concepts like the following:

  • Diversifying across many asset classes and many companies helps to reduce risk,
  • Reducing investment costs helps to increase investment return, and
  • Given enough time, most economies will create wealth.

New to Investing? See My Related Book:

Book6FrontCoverTiltedBlue

Investing Made Simple: Investing in Index Funds Explained in 100 Pages or Less

Topics Covered in the Book:
  • Asset Allocation: Why it's so important, and how to determine your own,
  • How to to pick winning mutual funds,
  • Roth IRA vs. traditional IRA vs. 401(k),
  • Click here to see the full list.

A Testimonial:

"A wonderful book that tells its readers, with simple logical explanations, our Boglehead Philosophy for successful investing." - Taylor Larimore, author of The Bogleheads' Guide to Investing

Comments

  1. Everyone seems to backtest to prove their investment strategy will work in the future. Of course, backtesting proves nothing other than giving the investor that wonderful feeling that they are using the perfect investment strategy for yesterday’s results. In the worst cases this leads to performance chasing. Just read the financial magazines in the grocery stores, classic performance chasing.

  2. Back testing has different uses. For example, anyone can come up with a profitable stock market system by using a screen that is continually tweaked until the outperformance is impresive. This of course is totally bogus and is why it is always important when viewing performance data to ask how it was achieved.
    Using historical data to come up with appropriate withdrawal rates on retirees’ nest eggs is, I believe, useful. Since the 1920s the economy has experienced wild swings in inflation, stock performance, and bond performance. Withdrawal rates that worked well throughout should provide some comfort. And, more importantly, if they didn’t then they would be a source of concern.
    Otar’s work does stress the importance of being careful in interpreting averages. Sometimes people get an unwarranted sense of well being because they are looking at averages. These can cover up the important impact of the market over the first few years of retirement. I’m sure anyone who retired at the end of 2007 knows exactly what I’m talking about.
    Ask Long Term Capital Management about backtesting. That was how they came up with their so-called “value at risk” models.

  3. It should be noted that there’s a lot more to Otar’s book than the backtesting and re-balancing in presidential years, and in particular he has some provocative things to say about annuities (he’s for them) and asset allocation (he apparently thinks that doesn’t matter). He also takes 500+ pages to say what he probably could have said in 10.

  4. Indeed. I meant what I said about “important insights.”

    I think his explanation of the significance of sequence of returns risk makes the book worth reading. And his take on the buckets method of asset allocation in retirement is quite good.

    The book certainly got me thinking.

    At the same time, the thoroughly bizarre scenarios he puts forth in his examples often made me wonder whether he was “cherry picking” in some way or other to make his points appear stronger.

    6% withdrawal rate?
    Half as much allocated to Canada as to the U.S.?
    Assuming a 1% alpha over a 30-year period?

    And he has a tendency to shift variables around from example to example, even when he isn’t making a point about those specific variables.

  5. Mike: “I think his explanation of the significance of sequence of returns risk makes the book worth reading. And his take on the buckets method of asset allocation in retirement is quite good.”

    Can you please expand on this? I must have missed it somewhere around page 463.

    Mike: “Half as much allocated to Canada as to the U.S.?”

    Just think what he’d say if he were from Belgium rather than Canada . . .

  6. His “buckets method” explanation starts on page 163. I’ll probably write a post about it in the near future, as I think it does a better job of minimizing the negative impact of volatility than lots of other retirement asset allocation methods.

    “Just think what he’d say if he were from Belgium rather than Canada . . .”

    Ha! 🙂

  7. Bret @ Hope to Prosper says:

    I think back testing is extremely useful, if you are trying to sell a book on stock market investing. As for actual investing, I wouldn’t count on the past performance being an indicator of future results.

    Without getting into the whole “efficiencies of markets” theory, here is the problem. Investors will quickly rush in to adopt any strategy that has been proven to be successful, in the past. This reduces the chances that any historic strategy will work similarly, in the future. Not to mention, the market conditions will change with time.

    So, I use back testing for strategizing, but I never count on it.

  8. WealthWebGuru says:

    One more item for your list of essential simple concepts is rebalancing. Buy and hold is the most common management strategy for an investment portfolio. Investors need to think about a strategy to manage a portfolio other than just buy and hold because buy and hold does not always work.

    I think Otar’s book is excellent and is a wonderful resource for retirement.

  9. Monique says:

    The idea of rebalance during election years is interesting and sounds fun, but not really anything base an investing decision.

Disclaimer: By using this site, you explicitly agree to its Terms of Use and agree not to hold Simple Subjects, LLC or any of its members liable in any way for damages arising from decisions you make based on the information made available on this site. I am not a financial or investment advisor, and the information on this site is for informational and entertainment purposes only and does not constitute financial advice.

Copyright 2019 Simple Subjects, LLC - All rights reserved. To be clear: This means that, aside from small quotations, the material on this site may not be republished elsewhere without my express permission. Terms of Use and Privacy Policy

My new Social Security calculator (beta): Open Social Security