Wednesday, November 16, 2011

Are crowds really wise?


Surowiecki, James. (2004). Wisdom of Crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations. New York, NY: Doubleday.
P.272. eISBN 0-307-27505-1.

The first lesson I learned about the Wisdom of Crowds was from my experience buying the book. I purchased the Kindle version of the book and when I got to sections of the book I wanted to highlight, I observed they were already highlighted. I wondered how Amazon had magically done that. I learned that what Amazon did was to aggregate portions that other readers had highlighted. Amazon collected independent ideas from diverse readers and aggregated sections of the book that was more useful to readers.
This is an example of Surowiecki’s argument on the Wisdom of Crowds. Surowiecki’s asserts that experts are overrated because with the right conditions groups are more intelligent than individual experts. He notes that for a crowd to be wise, four features must be present, diversity, independence, decentralization and aggregation, and these four features can be applied to solve cooperation, coordination, and cognition problems.
The book provides proofs on the wisdom of crowds and contends that average is not synonymous with mediocrity. The author does not explicitly define what a wise crowd is or how to build a wise crowd. What he does in the first half of the book is to provide a theory on the wisdom of crowds and discuss the features necessary for a crowd to be wise. Surowiecki discusses the importance of diversity and notes that diversity in a group exists when people have different private information. Independence and individual decision- making is the second concept he discusses. Here, judgments and decisions are formed without distortions such as herd mentality, information cascades, and social proof. As the author notes, if people begin to make the same decisions, they stop being smart. Surowiecki states that decentralization exists “when people are able to specialize and draw on local knowledge” (p.10). He posits that for decentralization to be useful it has to be aggregated. Aggregation is the method by which individual decisions are collated into a collective decision by a benevolent central planner.
      The second part of the book addresses how the wisdom of crowds can be used to cognition, coordination, and cooperation problems. Coordination problems are problems that require the alignment of the actions of different people. People need to take into account their own decisions and the decisions they believe others would make. Examples include; William H Whyte’s study on New York’s pedestrian movement, London’s traffic congestion problem and what day to visit one’s favorite bar as examples of coordination problems Cooperation problems require self interested and skeptical people to work together for a common good. The examples of corruption in Italian soccer, payment of taxes, addressing pollution, and trust in business illustrate this point. Cognition problems have definite solutions and are easily solved by wise crowds because all they require is estimation. The author cites examples of jellybeans in a jar, and numerous economic experiments to explain this point. At the end of the book, the author attempts to draw a link between democracy, informed citizenry and the wisdom of crowds
    The wisdom of crowds is replete with examples and anecdotes that illustrate the author’s contentions. Surowiecki cites Iowa Electronic Markets, Who Wants to be a Millionaire, Google, and the emergence of automobiles as exemplars on the value of diversity as a feature of wise crowds. A NFL anecdote and economic experiments are used to explain the importance of independence. A current example that also demonstrates autonomy is boids theory in agent based modeling where boids independently decide where to locate. At the end of the simulation there is cohesion of boids at certain point. The boids make independent decisions that lead them to the average position of other boids. The creation of Linux software is the author’s example of decentralization. The popularity and emergence of open source models, social media sites like Twitter, and crowdsourcing show the effects and merits of decentralization and also illustrate the importance of decentralization. The lack of aggregation on the part of Iraqi insurgents failed to sustain their struggle against the invasion of Iraq. Linus was the aggregator of Linux. If the inputs of the diverse contributors were not aggregated, the software would have been unusable. Surowiecki claims that for decentralization to be useful, it has to be collectively distilled and aggregated by benevolent central figure or authority like Linus. Today, similar central authorities might be new media tools such Govloop and Facebook
Surowiecki’s book is an interesting and engaging read. Its simplicity and ability to make complex ideas simple to readers with no economic or psychology background is a strength of this book. The author cites many examples and anecdotes to prove his theses. He also discusses cases where the features of a wise crowd were not utilized and the negative consequences. Two examples – the Challenger and Columbia disasters- demonstrate this point. The author also convincingly shows how the wisdom of crowds can be applied to businesses and markets. Despite the strengths of this book its flaws detract from its theses. First, the book concluded as an anti-climax for me, as the author fails to clearly state how the wisdom of crowds can be applied to democracies and governments. My guess is that it cannot because issues like democracy and politics are highly emotive. One only needs to view or listen to opinions on politics to see the failure of the wisdom of crowds. I wanted the author to be more convincing about his conclusions in the last chapter on democracy and participatory government and his conclusion that even though the crowds make not make a “wise” decision, the “decision to make them democratically does” was a cop out (p.272). Second, the author fails to clearly state how aggregation can be applied without losing the advantages of decentralization. For instance, in his discussion on information sharing among government agencies, it would have been very useful if the author had suggested how the information of the decentralized units could be aggregated. In terms of predicting markets, how often is the crowd wise? The housing bubble tells a story of the failure of the wisdom of crowds. The author also fails to clearly define concepts such as what the wisdom of crowds is, cooperation problems and aggregation. He uses case studies to illustrate his point.  Some chapters and case studies in the book do not flow and makes a reader wonder how it relates to the wisdom of crowds. For instance, the author seems to go off tangent with some examples such as the Milgram experiment on conventions. As someone who is from a non-western culture, I would have appreciated a more diverse focus as most of his examples are from the United States and western countries; his theories may not be applicable to other parts of the world
The author also offers readers like me a crash course in business, economics and psychology. I have found the book very useful in enlightening me about “short selling” as a price mechanism discovery and markets in the United States. The book has also been valuable in introducing me to conventions and cultural references in the United States. When I view threads on articles I read, I try to find or fail to find examples of the wisdom of crowds and other readers may also try this experiment after reading this book. In a period where we rely on experts and leaders, this book provides a counter narrative to the wisdom of expert groups and leaders. The author’s discussion how decentralization has to be aggregated and not just for its sake is valuable. Surowiecki has succeeded in promoting the wisdom of crowds and harnessing the resources of diverse people to solve problems. We one reads the book, notions of expert wisdom are challenged and an appreciation is developed for the features of a wise crowd. This is because a wise crowd does not have to be large in number.
The book further aided my understanding of the concepts in our open government and e-participation lectures on how government can act as platform and turn collective complaint into collective action. The wisdom of crowds has proven useful in gov 2.0. The federal government’s initiative such as challenge.gov taps into the collective wisdom of crowds. As Beth Noveck (2010) notes the failure of the EPA to utilize the wisdom of crowds may have impeded its ability to carry out one of its mandates. Another example of the wisdom of crowds is patientslikeme where patients come together to share information about their health challenges. Patientslikeme is an interesting case study because it is another instance of decentralization being aggregated. We can also associate the wisdom of crowds to crowd sourcing. This however comes with the caveat that the wisdom of crowds cannot be used to solve wicked problems such as climate change or in voting the right people into government.
As a reader, it would be insightful to read an updated version of the book to see the case studies Surowiecki would use in this age of Web 2.0 and Gov 2.0 where recent trends strengthen his theses on crowd wisdom compared to the exemplars in the book. It would also be interesting to read Surowiecki’s take on the Arab spring and Occupy Wall Street protests in the context of the wisdom of crowds. The book’s greatest contribution is that it has challenged expert wisdom and made a case for the wisdom of crowds. It has shown that crowds are not always unintelligent. Surowiecki’s book is intellectually engaging and well written. The author should consider writing a revised version of the book.

References
Noveck, B.S. (2010). The single point of failure. In Lathrop D & Ruma L. (Eds.). Open government: collaboration, transparency, and participation in practice (p 49-70). CA: O’Reilly Media Inc. Kindle edition.