Sunday, November 30, 2008

Fooled by Randomness [book]


In many ways this is better than Taleb’s more popular book, The Black Swan, which I’ve already noted is one of my favorite books of all time.  He started it before he was famous (before he knew it would sell), so he tried harder while writing it and it shows. The second edition (which is a third larger and better edited) is well-organized and easy-to-read, unlike Black Swan which sometimes can meander.

His central thesis is the same in both books: humans are hard-wired to demand an explanation for everything, even when there is none, leading us to be fooled into thinking that something is not random when it really is.

Much of this is easily demonstrated mathematically. My favorite example is the game where an evil investment advisor sends letters to a thousand or so prospective clients, telling half that a particular stock will go up and the other half that it will go down. The second month, repeat the same mailing to the half of the list where the prediction happened to be correct. Keep repeating each month and at the end of ten months he'll have a (short) list of people who think he was correct for ten months in a row. Apply the same math to the world's pool of actual investment advisors and mutual fund managers and you'll find that the number of people with 10 year successful track records is about what you'd expect from pure chance!

Ergodicity, the statistical concept behind the law of large numbers, says that the true properties of a process become clear only after many iterations.  The problem in life, says Taleb, is that “winners” (whether successful investment advisors or CEOs) are often there because of the survivorship bias:  since they’re graded on results, not process, many “successful” people are in their positions because of luck not skill.  If your boss thinks he’s so smart, strip him of his current position and force him to do something new without relying on his resume (which may be luck anyway).  Then you’ll see how smart he really is.

The danger of Taleb's thesis -- and one I haven't fully navigated around yet -- is that you'll succumb to fatalism. If everything is random, then why bother working hard?

Some of his suggestions are good ones, such as the idea of Buridan's Donkey: a donkey equidistant from food and water will starve unless you give him a slight nudge toward one or the other. In this case randomness is your friend, because it provides just enough of a push to get you off your duff and doing something, regardless of what it is.

You can't blame all success on luck. The investment advisor who was smart enough and worked hard enough to notice fraud in the company's annual report: I believe he'll be more successful long-term than the one who simply flipped coins.

As in the Black Swan, the weak parts of Taleb’s case are exposed in his uninformed discussion of the QWERTY problem, or so-called "path independence" : the idea that once a particular idea or product takes off, network effects kick in to give it an insurmountable advantage, even if the original is provably inferior to alternatives. Path independence theory has been refuted to my satisfaction by the works of Liebowitz and Margolis, who showed that most (all?) of the so-called examples of this are wrong, that in fact network effects are pretty good at giving the advantage to the best technology, not the worst.

As I get older, I become more humbled at how little you can trust even the smartest analysis for why and how things turn out the way they do. Newspapers and TV are so often wrong, especially on their first reports, that I wonder why we bother to pay attention at all.   I am intrigued by the Taleb style of reasoning that says randomness is at the heart of everything we do.  The trick is to internalize that fact, yet keep trying in spite it.


Anonymous said...

Yeah, I love this book. Taleb is a financial guy, so you can understand his ignorance of network effects, which are disproportionately strong in technology relative to other industries.

The other long-lasting lesson of Fooled by Randomness is that markets are ultimately driven by human psychology, not numbers. No matter how sophisticated we get with technical analysis of markets, ultimately they'll be governed by the endless cycles of euphoria and panic that are characteristic of the collective human psyche. Until we change what it is to be human, we won't tame the markets. And we learn that, though bust after bust after bust (we aren't very good at learning in booms.)

Mike Cero said...

I immediately thought of Fooled by Randomness after reading the WSJ’s review of 7 Deadly Scenarios by Andrew Krepinevich. Along the same line of Taleb's justification for planning against financial asymmetric outcomes, albeit infrequent, Krepinevich alerts readers (National Leaders) in the need to plan against the “worst plausible” threat scenarios. That is to say, does Krepinevich draw parallels to Taleb’s steadiness as a trader in accepting frequent small loses (or small relative gains) in exchange for a strategy secure against the big crash. I look forward in buying Krepinevich’s book at the local book store, partly to see if Krepinevich discusses the Guantanamo closing and to see how he resolves non-uniformed leaders resource allocation for addressing the many dooms day scenarios uniformed leaders are tasked to face.