Get new articles by email:

Oblivious Investor offers a free newsletter providing tips on low-maintenance investing, tax planning, and retirement planning.

Join over 20,000 email subscribers:

Articles are published every Monday. You can unsubscribe at any time.

Insufficient Sample Size

“I created my system back in 2001 and have been using it since. It works. It’s not luck.”

This claim came from an investor explaining his method for picking market-beating mutual funds.

9 years is a long time in a person’s life. Can you think of something you’ve been doing for the last 9 years? Whatever it is, it’s probably an integral part of your identity by now.

In this person’s mind, his market-beating ability is a part of his identity. Tell him there’s a good chance that he’s just gotten lucky, and he’ll think you’re full of crap.

But it’s true. 9 years isn’t a large enough sample size to tell us with confidence whether this investor’s results were due to skill or luck.

Anecdotal Evidence

I particularly like this quote from John Cochrane, professor of Finance at the University of Chicago:

“Modern medicine doesn’t ask old people for the secrets of their health. It does double-blind clinical trials. And to that we owe a lot of our health. Modern empirical finance doesn’t ask Warren Buffett to share his pearls of investment wisdom.”

We have a natural tendency to assign too much significance to anecdotal evidence, especially when that evidence is our own life experience. Yet anecdotal evidence is worth almost nothing. Even if it’s yours.

The fact that somebody has successfully beaten the market by picking stocks, picking funds, or jumping in and out of the market at the right times doesn’t mean a thing…

  • unless we can combine that with other information about other investors who are doing something similar and (reliably) getting similar results, or
  • unless we can see that the degree to which (and reliability with which) the investor is outperforming the market is sufficient to show that it’s not due to randomness.

One investor (or one fund) over one decade just doesn’t cut it.

New to Investing? See My Related Book:

Book6FrontCoverTiltedBlue

Investing Made Simple: Investing in Index Funds Explained in 100 Pages or Less

Topics Covered in the Book:
  • Asset Allocation: Why it's so important, and how to determine your own,
  • How to to pick winning mutual funds,
  • Roth IRA vs. traditional IRA vs. 401(k),
  • Click here to see the full list.

A Testimonial:

"A wonderful book that tells its readers, with simple logical explanations, our Boglehead Philosophy for successful investing." - Taylor Larimore, author of The Bogleheads' Guide to Investing

Comments

  1. Mike,

    Love your book and investment style and it works for me, but I don’t get your’s (and other index fund guru’s) distrust of those that can consistently pick stocks.

    I am not sure if distrust is even the word…it seems more like putting on blinders to the fact that there are those that have the art/skill to pick stocks (whether it is Buffet or David Ricardo in the late 1700s).

    Again, I will reiterate that I am all for your view on personal investing where the person does not want to or can’t do the work needed to research individual stocks, but would you suggest to someone with institutional money to invest to do it with index funds?

  2. “Would you suggest to someone with institutional money to invest to do it with index funds?”

    Not index funds, as such–there are lower cost options once you have that kind of money. But passive investing? Yes. Absolutely.

    I’m not arguing that there are no investors with genuine ability to pick stocks. I don’t believe that to be the case. (And I don’t believe I’ve ever argued as such.) But I would argue that:

    1. The overwhelming majority of the time, we’re seeing false positives. We assume the presence of skill when there really is none, and
    2. Assuming the presence of skill without meaningful amounts of data is a mistake.

    As to Buffett specifically, what he does is different from simply picking stocks. He has access to investment opportunities that most investors do not. And when he buys companies, he often plays a role in managing the company.

    Not to say that he’s not a skillful stock picker. But he owes his superior returns to more than just security selection.

  3. Buffet is absolutely in a diff position, as evidenced by the preferred deals he worked over the Summer, but he is the easiest example I could come up with lol.

  4. It is telling that the only individuals we can come up with as examples of successful “stock-pickers” over the last century is Warren Buffett, and maybe Peter Lynch, until he gave up to write books – an even surer way to make money once you can point to a track record. Out of all the stock-pickers over this period, if it were a winning game there would surely be more names. Buffett himself is an outlier and even perhaps an anecdote

    Warren Buffett is lucky, too, at the very least in one sense. If, at a time when he surely had less knowledge than he does now, the first ten stocks he bought turned out to be losers, he’d have quit and done something else. I’m sure even he would admit that there had to be a little luck involved in his success along the way.

    JF

  5. JF,

    I don’t think it is telling – How many investment advisors are making a GREAT living, but don’t have the hundreds of millions to warrent a spot on CNBC? Its like saying since I can only name Dr. Oz as a doctor who is famous, there aren’t that many good doctors out there (For what it is worth I think there is a Dr. Gupta on CNN sometimes).

  6. Evan,

    I think people believe beating the market is far easier than it really is… Remember it just might be Warren Buffett or one of the company’s officers on the other side of your trade and they might just know something you don’t.

    If you can duplicate Warren Buffett’s results you need at least $500,000 to make it worth your time ( http://ponderingmoney.com/2009/10/23/you-need-half-a-million-dollars-to-pick-individual-stocks/).

    If you had a stock picking system that worked over many decades it may NOT work in some circumstances. As an example any strategy that had a lot of financial stocks would have gotten clobbered in 2008.

    With an index you will always get the market returns – costs. Since an index is so easy to manage the costs can be very low and it turns out that rate of return is pretty hard to beat.

    -Rick Francis

  7. Rick,

    I am not doubting anything you said, nor really anything Mike says (I really did enjoy his book) and my 401(k) is invested in mostly index funds.

    I just find it hard to believe that there are thousands upon thousands of registered investment advisors that are all bad at their job.

  8. “I just find it hard to believe that there are thousands upon thousands of registered investment advisors that are all bad at their job.”

    A few thoughts:

    1. RIAs do a lot more than just try to beat the market. (And, unsurprisingly, I’d argue that the best ones don’t even try.)

    2. By definition, if by “bad at their job” you mean “don’t deliver market-beating gross returns,” then half of those RIAs who try to beat the market must be “bad at their job.” (Potential exception being that RIAs have a tendency to earn above-market returns while other investors earn below-market returns. I’ve never seen any evidence to support that though.) And on an after-cost basis, more than half must be “bad at their job.”

    3. The fact that past performance has shown to be a terrible predictor of future performance for both mutual funds and stock newsletters (I’ve never seen any data for RIAs) strongly suggests to me that luck is the dominant factor in most above-market returns. Is it the only factor? Probably not. But it’s darned hard to separate situations where it is the only factor from ones where it isn’t.

  9. How about adding Anthony Bolton to the list of stock pickers.

    I also like to remember that for every person that wins there must be one that loses. Personally I’m a low fees and minimise tax index tracking kind of investor. Then to spice it up a little I bring 129 years of data into the argument.

  10. Mike, I wouldn’t agree that there are a lot of active fund managers who try to beat the market (which I think is what you mean by RIA – I’m not familiar with the acronym).

    Many (most?) managers, or those running retail funds anyway, hug their benchmark these days, because short term investors are so driven by recent performance and divergence from the indices.

    You get then the infamous closet tracker – only with a 1.5% charge!

    If you’re going to go for active funds, you may as well go for one who does something genuinely different and tries to outperform. (I’m not saying they will! 🙂 )

    Another point on funds – a manager can be pretty good and beat the index by say 2% a year, despite the various hurdles in his way, but after his/her fees and various other costs that’s probably not going to be enough to beat the index.

    Managers don’t have to just be better than the index. They have to be better than the index plus fees. Even harder! 🙂

    To address Evan’s point then, I actually think individuals picking stocks may have more hope (if they’re the right individuals) then individuals picking managers, although this is all now a million miles away from Mike’s extremely pertinent point about time frames.

    Nine years is nothing – big economic trends take decades. Interest rates have been broadly trending down since the 1980s. Whole careers have been built on that trade alone.

  11. p.s. Bolton is a great fund manager but the Americans don’t know him, my friend. 🙂

  12. Monevator: RIA stands for Registered Investment Advisor. It’s the license necessary to provide investment advice for compensation in the U.S.

    What I was trying to say was that many RIAs don’t see it as their job to beat the market. Instead, they help with other issues (asset allocation, retirement planning, etc).

    And you’re right. I had to look this Bolton gentleman up. Though in fairness, I wouldn’t know many US fund managers either. 🙂

  13. Evan,

    The doctor comparison is far different, as doctors have identifiable skills that can be causally linked to outcomes. Moreover, if doctors failed surgeries at a rate of greater than 50%, they wouldn’t be considered “good.” Finally, the health care world is not mainstream and high profile as the investment world is, so its no surprise that you can’t name great doctors off of the top of your head. I can easily name more investment managers than doctors, but that doesn’t make them great.

    I guess to be direct and respond your original question about why the distrust over active management, the answer is that there simply is no evidence to suggest that stock picking adds any additional return over index investing on a risk-adjusted and cost-adjusted basis. That’s not to say it can’t be done, but the odds are heavily against you and anyone else who tries to do it.

    JF

  14. Sadly, anecdotal evidence is the ONLY way most people know how to defend their positions anymore, Mike.

    I know that for a fact because I have lots of friends who argue anecdotally. 😉

    But seriously, I enjoyed the article and ensuing discussion.

    Best,

    Len
    Len Penzo dot Com

  15. “I know that for a fact because I have lots of friends who argue anecdotally.”

    Bravo! 😀

  16. I’m here all week, Mike. 😉

  17. “Bill Miller” – the greatest fund manager of all time (except for Lynch) for 15 years until he started to suck. 🙂

  18. Funny thing. For those I work with, they all beat the market this past decade. No secret. In a down market, any mix of stock and cash, even 80/20, will beat the S&P. So in a sense, I can say “I beat the market.” Yet, in a given up year, I’ll lag, by definition, as the cash/bonds will drag down the return.

Disclaimer: By using this site, you explicitly agree to its Terms of Use and agree not to hold Simple Subjects, LLC or any of its members liable in any way for damages arising from decisions you make based on the information made available on this site. The information on this site is for informational and entertainment purposes only and does not constitute financial advice.

Copyright 2023 Simple Subjects, LLC - All rights reserved. To be clear: This means that, aside from small quotations, the material on this site may not be republished elsewhere without my express permission. Terms of Use and Privacy Policy

My Social Security calculator: Open Social Security