The Wisdom of Arguing with Success

Last December, only a day before my son’s flight to Israel, the weather forecast predicted an 80% chance of a major snowstorm, which would cause both O’Hare and his connecting flight airport to shut down. The snow had already hit D.C. hard, and had led to a government shut down. Not wanting to spend long hours stranded in chaotic airports, my son re-booked his flight for a day later. The snowstorm never came. It was a beautiful sunny day, and his original flights arrived on time, only without him. The change of flight turned out to be very costly. The airline charged a hefty change fee and when he finally landed in Tel Aviv, Jerusalem was hit by the largest snowstorm in 50 years. He could not reach his grandmother in Jerusalem for two days. It was an unhappy event, and felt very much like an unwise decision.

When we evaluate our own decisions and performance, and certainly when we evaluate others’, we tend to focus on the results of the efforts. A manager whose project has a good outcome might be promoted, while a manager whose project failed might get fired. As the old saying goes, you can’t argue with success.

There is a very good reason to argue with success. Taking success or failure as a direct indicator of the value of an action is unwise because every outcome is a function of both skill and luck. Every human performance reflects a combination of the actor’s contribution and that of outside forces – unpredictable events, uncontrollable conditions, random noise – in short, luck. Even Michael Jordan’s performance on the court varied from one game to the next, reflecting not just his impressive skill but external elements such as how other team members played, who was playing on the other side, the mood of the referees, and unpredictable distractions from the audience.

Though skill and luck are featured in almost every performance, their relative contribution varies with the domain. Great success as a professional basketball player reflects more skill than luck. Great success as a “professional” gambler reflects more luck than skill. Luck plays a role, sometimes more sometimes less, in every human endeavor.

But people don’t naturally consider the contribution of luck when they evaluate performance. We have a strong bias to explain behavior in terms of elements internal to the actor such as skill, and it takes us a lot of effort to consider the contribution of external forces such as luck (Gilbert & Malone, 1995). That means that in many cases, we reward people for successful outcomes partly because they were lucky.

The most glaring example of this comes from a high-stakes area that rewards success in the billions of dollars – investing in equities. Take mutual funds that are mainly comprised of stocks. Thousands of such funds exist, managed by very smart and savvy individuals. They use a variety of methods to buy and sell stocks so that by year-end they can show a substantial increase in the value of the fund. Success is not just measured by how the fund increased in value. Real success is measured in beating the market, meaning, doing better than the overall market did. If the market as a whole increased by 10%, then a fund manager must do better than 10% plus the fees. A good benchmark is a passive index fund, one which simply owns a cross-section of stocks from the market, its value tracks the market’s value, and very little buying and selling takes place. In other words, the performance of an index fund reflects no skill at all, just the unpredictable variations of the market itself. So if you can’t beat an index fund, you clearly have no skill as you added no value.

But mutual fund managers do actually manage to beat the market. Some even outperform an index fund by 10- 20 percentage points or more, while a few even show such success year after year. Funds that perform this well get rewarded handily. Investors gravitate towards them and put millions of dollars into the fund, because they see the success as a reflection of great knowledge and skill. Successful fund managers are equally rewarded by their investment firms with extravagant annual bonuses the size of the annual budget of small towns. If you showed such profound skill, perhaps even wisdom, why not be rewarded accordingly?

Problem is, even such performance shows no skill at all. It only provides the illusion of wise actions. The University of Chicago economist Eugene Fama demonstrated decades ago that it is impossible to beat the market. The unpredictability of individual stocks is inevitable and therefore trying to pick individual stocks is futile.

If beating the market is truly impossible, then perhaps fund managers who beat the market several years in a row must have profound wisdom that is so deep it escapes the top academic minds? Not quite. Given that there are literally thousands of stock funds, largely working at random, a few of them are bound to perform very well. In fact, it would be very surprising if no fund had an outstanding year. By chance alone, some funds will also perform extremely well a few years in a row. But just as many funds will also perform far below the market index. This does not mean that their managers are unwise; it just means that they were unlucky. What is unwise is believing that any fund’s performance is a reflection of the manager’s skill.

Though Fama was awarded the Nobel prize for his discoveries on the nature of the stock market, this has not prevented a huge investment industry from claiming credit when luck strikes and from taking rewards as if the performance genuinely reflects deep wisdom.

Mistaking luck for skill when investing in stocks is an extreme case because no skill could be involved. It shows that when we know the outcome, it is very difficult for us to not infer skill as the root cause. Given that for most other things, performance is actually a mixture of luck and skill, what Jonathan Baron and John Hershey call “outcome bias” is very common. Baron and Hershey asked people to evaluate decisions after they learned their outcome. A surgeon decides to operate using a risky procedure. Is that a wise decision? The answer should not depend on whether or not the operation succeeded, but it very much does. People evaluated the quality of the surgeon’s decision more negatively if they later learned that the patient died than if he lived. Francesca Gino, Dan Moore and Max Bazerman show that the same is true for ethics. People are just as outcome biased when they evaluate how ethical an action is.

The good news is that people constantly learn from their experiences. Making many decisions every day gives us a large data set to build on over the years. The bad news is that because of the outcome bias we often learn the wrong things. You might follow the advice of a friend to invest in a very risky business, get lucky and make a windfall. This happy outcome would blind you to the folly of risking too much and you might be more willing to listen to that friend in the future.

It is even worse when a good decision has a bad outcome. Rebecca Ratner and Kenneth Herbst show that such unlucky events cause people to make worse decisions in the future. My son might not trust the weather forecast next time he travels because the forecasted snow never arrived. That would be a mistake because the weather forecast is extremely reliable, and far better than predicting the stock market. So that fateful day, when the prediction was an 80% chance of snow, meant that of all the days with such a forecast, 80 out of a 100 will in fact have snow. Therefore one should expect the 20 remaining days to not have snow. That day just happened to be one of those days. But the forecast’s “failure,” in our mind, reflects on its reliability and causes us to doubt the wisdom of relying on it.

So here is some simple advice that is hard to follow. When evaluating performance (of any sort), don’t argue with success. Just ignore it, and evaluate the wisdom of the decisions that preceded it. But if you know the outcome it is too late because you can no longer really evaluate the process independent of it. Instead, make sure you stay completely uninformed of the outcome. The only way to truly evaluate the wisdom of actions is therefore with the benefit of true ignorance.

References

Gilbert, D. T. and Malone P. S. (1995). The correspondence bias. Psychological Bulletin, 117, 21-38.

Baron J. & Hershey J.C. (1988). Outcome bias in decision evaluation. Journal of Personality and Social Psychology, 54 (4) Apr, 569-579.

Gino, F., Moore, D.A., & Bazerman, M.H. (2008). No harm, no foul: The outcome bias in ethical judgments, Working Knowledge, Harvard Business School.

Ratner, R.K., & Herbst, K.C. (2005). When Good Decisions Have Bad Outcomes: The Impact of Affect on Switching Behavior. Organizational Behavior and Human Decision Processes, 96 (1), 23-37.