The Wisdom of Crowds, Part 3, A Case for Impeachment

This is the final installment of my three part series on The Wisdom of Crowds by James Surowiecki.

In the first part of the series, I explained that, according to Surowiecki, in order for a group to be "smart," that is, in order for groups to make accurate predictions and decisions, the group needs to be diverse, independent, and decentralized.

Diversity brings in new ideas, independence insures that those new ideas get expressed, and decentralization prevents the sort of small minded group think that can lead a herd over a cliff.

Now here's where it gets interesting: In the months leading up to the September 11, 2001 attacks on the World Trade Center and the Pentagon--the U.S. intelligence agencies were organized in exactly this manner--diverse, decentralized, and independent. As Surowiecki writes, the U.S. intelligence community was, "a collection of virtually autonomous, decentralized groups, all working toward the same broad goal--keeping the United States safe from attack--but in very different ways." (p. 67)

In fact, U.S. intelligence agencies had all of the necessary pieces of information to know that an attack was coming. The CIA monitored and took pictures of a 9/11 planning meeting of Al Qaeda operatives in Malaysia in January 2000. The CIA knew that two Al Qaeda operatives moved to San Diego in January 2000. In fact, the Presidential Daily Intelligence Briefing for August 6, 2001 was titled, "Bin Laden Determined to Attack Inside the U.S" and Bin Laden wanted to hijack a U.S. aircraft.

So if we had all of the relevant pieces of data to prevent the attacks on 9/11 then how did they happen anyway.

THERE WAS NO AGGREGATOR.

In order for diverse, independent, and decentralized groups to make smart decisions there has to be someone to aggregate the information. Think about the example of a crowd at the state fair guessing the number of jelly beans in a jar--the aggregator is the person who adds up all of the guesses and finds the average. In a stock market, the aggregator is the New York Stock Exchange which collects all of the buying and selling information and shows the price of the stock (which reflects the collective wisdom of the group). For sports betting, the aggregator is the bookmaker in the casino who sets the point spread for the game.

In every Presidency before George W. Bush, the aggregator was the President of the United States. Think about how John F. Kennedy handled the Cuban Missile Crisis. He gathered all of the different viewpoints from all of the different defense and intelligence agencies in one room, had them all make their best case (which reflected the collective wisdom of their group), and then came up with a plan that reflected the collective wisdom of the entire group.

With President Bush, there simply is NO ONE at the top who is sifting through all the information to figure out the collective wisdom of all of the intelligence agencies. As former Bush administration counter-terrorism adviser Richard Clarke indicated, the Bush administration simply wasn't interested in counter-terrorism. Despite the warning the the Presidential Daily Intelligence Briefing, the President continued his vacation in Crawford, Texas.

After 9/11, again there was no aggregator of the information to figure out who did it and how to prosecute them. Cheney had already made up his mind to invade Iraq. Bush had already made up his mind to invade Iraq. Never mind that 15 of the 19 hijackers were from Saudi Arabia. If you've already made up your mind, the evidence never gets examined, the collective wisdom of the group is never identified, and poor decisions are the result.

In this case, the failure to aggregate the collective wisdom of the U.S. intelligence agencies has led to 2,974 American deaths on 9/11 and 3,646 (and counting) soldiers' deaths in Iraq. For all his talk of being The Decider, Bush has never fulfilled his responsibility as The Aggregator.

Speaking of collective wisdom, a poll by the American Research Group shows that 45% percent of Americans support impeachment proceedings against President Bush and 54% support impeachment of Cheney.

Among independents, 50% favor starting impeachment proceedings against President Bush, to only 30% opposed. And 51% of independents are also for starting impeachment proceedings against Dick Cheney, to 29% opposed.

The Wisdom of Crowds, Part 2, The Republican "Circular Mill" of Death

In a previous post, I mentioned that I just finished reading The Wisdom of Crowds by New Yorker business columnist James Surowiecki.

Today I want to focus on what the Wisdom of Crowds teaches us about the sorry state of the Republican Party.



It seems to me that the Republicans' greatest strength is that they are hierarchical and extremely good at following orders. It's also their greatest weakness. When Republicans have good leadership at the top, they are incredibly unified, agile, and focused. But when they have bad leadership at the top (or incompetent people as they do in this adminstration), their follow-the-leader mentality is an absolute disaster.

The Wisdom of Crowds illustrates the dangers of being a blind follower with a brilliant example:

In the early part of the twentieth century, the American naturalist William Beebe came upon a strange sight in the Guyana jungle. A group of army ants was moving in a huge circle. The circle was 1,200 feet in circumference, and it took each ant two and a half hours to complete the loop. The ants went around and around the circle for two days until most of them dropped dead.
    What Beebe saw was what biologists call a "circular mill." The mill is created when army ants find themselves separated from their colony. Once they're lost, they obey a simple rule: follow the ant in front of you. The result is the mill, which usually only breaks up when a few ants straggle off by chance and the others follow them away.
    ...[T]he simple tools that make ants so successful are also responsible for the demise of the ants who get trapped in the circular mill. Every move an ant makes depends on what its fellow ants do, and an ant cannot act independently, which would help break the march to death." (The Wisdom of Crowds, p. 40 & 41)

Surowiecki is illustrating the dangers of "information cascades" where decisions are made sequentially (and each additional decision is dependent on the prior decision). I think its also a perfect description of the current state of the Republican Party. A circular mill of death where they all march in a circle, thinking they are following the leader, but actually just going around and around until they all (and many more of our soldiers) die.

To paraphrase, 'Every move a Republican makes depends on what his fellow Republicans do, and a Republican cannot act independently, which would help break the march to death.' When it comes to the Iraq war, Republicans in the House and Senate just march around and around in a circle (filibustering, delaying, denying), thinking they are following their leader when in fact, they never had one.

I'll finish my 3 part series on The Wisdom of Crowds in a subsequent post.

Why Secular Society Makes Better Moral Decisions than Organized Religion (The Wisdom of Crowds, Part 1)

A few weeks ago I was talking with friends, and announced that I wanted to write a book that asks (and answers!) the questions: why is it that secular society seems to find the best answers to big moral and ethical questions so much sooner than the world's great religions? Why is it that secular society is, more often than not, ahead of religion in figuring out pressing moral issues?

It seems to me that over the last several centuries, secular society, not religion, led the way in toppling monarchies and starting participatory democracies, abolishing slavery, and granting women the right to vote. Today it is once again secular society (and even corporations!), not religion, that is leading the way to granting equal rights to people who are lesbian, gay, bisexual or transgender.

In each case there were religious supporters of these positions (democracy, abolitionism, universal suffrage). But for the most part, my understanding is that these were secular movements that influenced religion, not the other way around.

The only counterpoint to these examples is the civil rights movement in the 1950s and 1960s where the religious community (as led by Dr. King) was out front on the issue of ending Jim Crow laws and insuring that African Americans had the right to vote and go to school and participate fully in society.

I imagined that it would take years of research to answer these questions satisfactorily. But just this week I picked up The Wisdom of Crowds by New Yorker business columnist James Surowiecki and it is absolutely brilliant.

The thesis of the book, is both stunning and exhilarating:

If you put together a big enough and diverse enough group of people and ask them to make decisions affecting matters of general interest, that group's decisions will, over time, be intellectually superior to the isolated individual, no matter how smart or well-informed he is.

The classic example is to ask a crowd of people (say, at a state fair) to judge how many marbles are in a jar, or how much an ox weighs. The average of all of the guesses from the crowd will tend to be more accurate than the guesses of even the best experts.

But it's not just true in connection with guessing the number of marbles in a jar. It's also true of stock market mutual funds.

"Between 1984 and 1999, for instance, almost 90 percent of mutual-fund managers underperformed the Wilshire 5000 Index [which basically represents the entire U.S. stock market]. The numbers for bond-fund managers are similar: in the most recent five-year period, more than 95 percent of all managed bond funds underperformed the market." (p. 33)

Groups don't always reach the best answer. Depending on how they are set up, groups can lead to a dumbing down or a lowest common denominator solution rather than excellence. According to Surowiecki, in order for a group to be smarter than the experts it needs 3 things--diversity, independence, and decentralization.

Diversity brings in new ideas, independence insures that those new ideas get expressed, and decentralization prevents the sort of small minded group think that can lead a herd over a cliff.

The section on diversity is amazing:

...[G]roups that are too much alike find it harder to keep learning, because each member is bringing less and less new information to the table. Homogeneous groups are great at doing what they do well, but they become progressively less able to investigate alternatives. Or, as March has famously argued, they spend too much time exploiting and not enough time exploring. Bringing new members into the organization, even if they're less experienced and less capable, actually makes the group smarter simply because what little the new members do know is not redundant with what everyone else knows." (p. 31)

Diverse teams come up with better answers than homogeneous teams, diverse nations are stronger than homogeneous nations, and Google is smarter than the world's smartest man.

Which brings us back to our original question. Why is it that secular society seems better than religion at addressing the biggest moral and ethical issues of the day? Because secular society, if set up properly, has all of the features of smart groups--diversity, independence, and decentralization. If a society is set up properly, it also has avenues for aggregating these opinions (e.g. elections).

Religions, it should be noted, tend to be characterized by all the factors that makes groups less able to come up with good answers--they tend to be homogeneous, hierarchical, and centralized, and lack both the mechanisms and the will to aggregate the opinions of a broad swath of people.

The Wisdom of Crowds also explains why the Iraq war is such a disaster, which I'll explain in a subsequent post.