Fooled by Randomness [book]

 

In many ways this is better than Taleb’s more popular book, The Black Swan, which I’ve already noted is one of my favorite books of all time.  He started it before he was famous (before he knew it would sell), so he tried harder while writing it and it shows. The second edition (which is a third larger and better edited) is well-organized and easy-to-read, unlike Black Swan which sometimes can meander.

His central thesis is the same in both books: humans are hard-wired to demand an explanation for everything, even when there is none, leading us to be fooled into thinking that something is not random when it really is.

Much of this is easily demonstrated mathematically. My favorite example is the game where an evil investment advisor sends letters to a thousand or so prospective clients, telling half that a particular stock will go up and the other half that it will go down. The second month, repeat the same mailing to the half of the list where the prediction happened to be correct. Keep repeating each month and at the end of ten months he'll have a (short) list of people who think he was correct for ten months in a row. Apply the same math to the world's pool of actual investment advisors and mutual fund managers and you'll find that the number of people with 10 year successful track records is about what you'd expect from pure chance!

Ergodicity, the statistical concept behind the law of large numbers, says that the true properties of a process become clear only after many iterations.  The problem in life, says Taleb, is that “winners” (whether successful investment advisors or CEOs) are often there because of the survivorship bias:  since they’re graded on results, not process, many “successful” people are in their positions because of luck not skill.  If your boss thinks he’s so smart, strip him of his current position and force him to do something new without relying on his resume (which may be luck anyway).  Then you’ll see how smart he really is.

The danger of Taleb's thesis -- and one I haven't fully navigated around yet -- is that you'll succumb to fatalism. If everything is random, then why bother working hard?

Some of his suggestions are good ones, such as the idea of Buridan's Donkey: a donkey equidistant from food and water will starve unless you give him a slight nudge toward one or the other. In this case randomness is your friend, because it provides just enough of a push to get you off your duff and doing something, regardless of what it is.

You can't blame all success on luck. The investment advisor who was smart enough and worked hard enough to notice fraud in the company's annual report: I believe he'll be more successful long-term than the one who simply flipped coins.

As in the Black Swan, the weak parts of Taleb’s case are exposed in his uninformed discussion of the QWERTY problem, or so-called "path independence" : the idea that once a particular idea or product takes off, network effects kick in to give it an insurmountable advantage, even if the original is provably inferior to alternatives. Path independence theory has been refuted to my satisfaction by the works of Liebowitz and Margolis, who showed that most (all?) of the so-called examples of this are wrong, that in fact network effects are pretty good at giving the advantage to the best technology, not the worst.

As I get older, I become more humbled at how little you can trust even the smartest analysis for why and how things turn out the way they do. Newspapers and TV are so often wrong, especially on their first reports, that I wonder why we bother to pay attention at all.   I am intrigued by the Taleb style of reasoning that says randomness is at the heart of everything we do.  The trick is to internalize that fact, yet keep trying in spite it.

Greenspan Recants (or did he?)

The current financial crisis is an interesting real-life experiment to free market fans like me.  It’s far too early to draw conclusions, but much of the generally accepted wisdom right now is that “pure capitalism” has failed and that the world needs more regulated markets to protect against “excess” and prevent or lessen these crises in the future.

But my belief in free markets is more about information than it is about money.  Most regulations involve one person (a bureaucrat) telling another person (you or me) what to do.  Sometimes that’s necessary:  for example we’re all better off when a central planner decides that everyone must drive on the right-hand side of the road.  But for this to be effective, bureaucrats need to be humble about their terrible disadvantage in information.  No regulator can possibly know as much as you do about your individual circumstances, and this information disparity is the source of why regulation often causes more harm than help.

Those of us who believe this idea have long thought of Former Federal Reserve Chairman Alan Greenspan as an ally, so I was intrigued when I saw news reports (now repeated over and over) that he “changed his mind”.

Except, he didn't. When I look at the actual transcripts, it's clear to me that careless news reporters have simply spun his remarks incorrectly.  Here's the key quote:

Chairman WAXMAN. Well, where did you make a mistake then?

Mr. GREENSPAN. I made a mistake in presuming that the self-interest of organizations, specifically banks and others, were such is that they were best capable of protecting their own shareholders and their equity in the firms.

And it's been my experience, having worked both as a regulator for 18 years and similar quantities, in the private sector, especially, 10 years at a major international bank, that the loan officers of those institutions knew far more about the risks involved and the people to whom they lent money, than I saw even our best regulators at the Fed capable of doing. So the problem here is something which looked to be a very solid edifice, and, indeed, a critical pillar to market competition and free markets, did break down. And I think that, as I said, shocked me. I still do not fuIly understand why it happened and, obviously, to the extent that f figure out where it happened and why, I will change my views. If the facts change, I will change.

What follows is his attempt to defend himself against the implication that somehow he was derelict at his duties as Fed Chairman, that ideology got in the way of his performing his legal responsibilities. As someone who obviously cares about integrity, much of Greenspan's testimony is simply responding to those charges, which he thinks are unfair.

But he eventually reaches his bottom line:

Mr. GREENSPAN. I think that it's interesting to observe that we find failures of regulation all the time, and one of the reasons is a very significant amount of regulation in the economic area is based on a forecast to know in advance whether or not particular products will go bad or the cycle will turn. If we are right 60 percent of the time in forecasting, we're doing exceptionally well. That means we are wrong 40 percent of the time, and when you observe the extent of the broad failure, the difficulty is that nobody can forecast. If you try to take a look at what the private sector does it's precisely the same thing that goes on in government. We at the Federal Reserve had a much better record forecasting than the private sector, but we were wrong quite a good deal of the time and that is reflected in how one views what the appropriate regulatory authorities are because unless you can anticipate the types of problems that are going to happen, it's very difficult to know what to do. And I think that's the problem that this type of thing confronts and I don't see any way in which that's going to be fundamentally changed. We can try to do better, but forecasting is never--never gets to the point where it's 100 percent accurate.

When you look at his entire testimony, it's clear that Greenspan in no way has changed his mind or even modified his fundamental pro-market position. He maintains his strong support for an unregulated market in derivatives, for example, which "are working well" (his quote). The part he doesn't understand (where he'll change his mind if the facts change) relates to credit default swaps--a market that barely existed when Greenspan was chairman and therefore was impossible to forecast.  And who would we be regulating anyway?   He reminds us of this:

"We are not dealing with people who are dumb. We are dealing with, by far, the most sophisticated, thoughtful people about the way markets work who created the major problems".

Enacting regulations to prevent a future collapse of the credit default swaps market would be a waste of time, since (as Greenspan notes) nobody is interested in that market anymore now anyway. But enacting other regulations would simply hamper a market that is already working well 60% of the time. Does anybody think they can get better odds on an alternative system? Exactly which regulations would you enact that would improve upon that record? Greenspan doesn't recant: he just admits he doesn't know.  Do his regulation-happy opponents or the careless reporters know?   Do you?

Maxwell wins

Many of us have been watching the shifting return counts since election night, but now it’s official:  Marcie Maxwell won the election for 41st district state representative over Steve Litzow.  The margin was extremely close:  748 out of 64,394 votes cast, much tighter than the 1591 votes that separated the two of them during the primary election.

I think she won for two reasons:

1. She’s not George Bush.   In a district that Obama carried by 64%,  anybody with a “D” in their name was guaranteed at least 748 voters who wanted their state legislator to end the war in Iraq, end subsidies to Big Oil, end all that corruption in Washington D.C., stop the Born Again Fundamentalists from running the government, help those poor Katrina victims, and make abortion legal again.  If you can fog up a mirror you can vote in this country, and this year the momentum was on the side of Democrats.
   
2. Steve went negative.  Too many people were put off by the petty, irrelevant attack mails that Steve sent, implying that Marcie somehow doesn’t care about student privacy.  Or something – I’m not even sure what he was implying about her.  Anyone who knows Marcie personally (or knows somebody who does – which is half the city of Renton) looked at his ads and rolled their eyes.  Same thing with his comments about Renton school performance; there was a right way to legitimately bring this up as an issue, but Steve came across as somebody who was just picking on the good people of Renton.

The full downloadable results will be ready in early December, and I can’t wait to pour through the numbers so I can update the analysis from the August primary.  Here’s what I’ll be looking for:

• Did Steve carry Bellevue?  I bet he did.  That would be interesting because it challenges the popular explanation that the Eastside is becoming more Democratic.  In other words, it’s possible (even likely) for a Republican to win District 41, even in a terrible year.
• Did education voters make the difference?  Since this is Marcie’s main campaign theme, it will be interesting to see how much the voters agreed.  This didn’t matter much in the primary, so I want to see if anything changed in the larger turnout for the final election.

Meanwhile, Marcie deserves congratulations, and I’m proud to stand behind her as my legislator for the next two years  (Argh, is that all it is?!  After nearly a year of hard work campaigning, the prize is a lot more of those 2-hour drives to Olympia, a diddly state legislator’s salary,  and then you have to do it all over again!

Civics test: 90%

Ouch! I’m disappointed that I got three answers wrong on a new 33-question civics test created by the Intercollegiate Studies Institute, giving me a score of 90.1%.

Still, that’s better than 49% mean score of the 2500 people test-takers nationwide. In fact, it’s way better than every single category sampled:

Group	Score
Income > $100K	55
Democrat	45
Republican	55
Age 18-24	47
Age 45-64	52
Doctorate	72
Undergraduate	57

The study's authors somehow convinced 164 elected officials to take the test, and their average score was 44% – five points lower than the general public.

These questions are pretty basic: multiple choice answers to questions about the First Amendment, the purpose of the Federal Reserve Bank, names for the three branches of government. Yes, some of them were a little tricky, but I don't think you can be an informed voter if you don't know this stuff.

Take the 5-minute test yourself and let me know how you did:

http://www.americancivicliteracy.org/resources/quiz.aspx

Cheap Gas

What’s going on with gas prices?  Look what I just paid at a 76 station right off the freeway near Ellensburg:

(That’s a ‘1’ at the beginning of the price, not a ‘2’)

Purple toes in Zillah

There are no wine lovers in the San Francisco Bay Area (or in Seattle for that matter) who haven’t been to Napa Valley.  But comparatively few people know about a  similar wine-growing region just two hours east of Mercer Island (over the soon-to-be-tolled I-90 bridge), in the Rattlesnake Hills area of the Yakima Valley.  We spent our weekend there, where some good friends are getting started with their own winery.

First, we picked some grapes:

then we crushed them:

and now look at the toes on my 6-year-old:

You should go too!  Best place to stay in Zillah is the Comfort Inn, for about $100/night, including a big breakfast, a pool, and free use of their grill!

Worst President Ever

I'm a big fan of Black Swan theory, which says that although leaders are measured by the Big Things that happen, they are rarely given credit for policies or actions that prevent Big Things from happening in the first place. Since it's impossible to wind back the clock and do the redo the experiment, we are left victims of the narrative fallacy, our desperate need to explain everything even when the real cause is unknowable, and to assign credit or blame to people (like the President) who have very little real influence compared to the collective, independent actions of the rest of us.

Think how history would look back on the past eight years if we had a President who had done this:

• Instead of focusing on the economy, he wasted his first year in office on a quixotic push to pass intrusive Homeland Security laws like the one forcing airlines to put locks on cockpit doors.  Experts agree this had no effect on safety, as shown by the complete lack of hijackings since.
• Second year in office, he forced burdensome new regulations on business transparency that devastated many innocent companies, forcing needless management changes at great American companies like Enron, Worldcom, Tyco, and others.  It’s a miracle these companies still exist.
• Iraq War: rather than leave the task to international inspectors, he wasted a trillion dollars removing a dictator who experts agree presented no serious long-term threat.
• New Orleans levee protection: in a blatantly corrupt attempt to reward his cronies and win votes, he poured excessive federal money into a crash program to over-build flood levees when it’s now obvious that even during a large storm like Hurricane Katrina, the original levees would have held just fine.
• Millions of poor families remain stuck in rental housing because of his refusal to continue the wise policies of the 1990s that offered a chance at home ownership to everyone.  The economy is now paying the price, with stalled housing values and a stock market that barely rises past the inflation rate.

Nobody, not even the wisest expert out there, can predict the Biggest Thing that will happen to the next President.  But any pre-emptive thing he does will be judged unkindly by history precisely because the Biggest Disaster he prevents is the one that never happens.