Friday, September 4, 2009

Evolutionary Prediction - The Wisdom of the Crowds

From my point of view, the biggest single paradigm shift in human history can be marked by the introduction of theoy of evolution and the theory of natural selection. The theory not only gives us meaning in understanding the past, it is the basis for the most powerful ways we have to predicting the future.



First and foremost, it opened up a totally new perspective on the whole concept of humans and where we come from. Old school religious theories were thrown out (even though some hardcore bible-enthusiasts still cling to sci-fi concept of intelligent design).

Secondly, and maybe just as important it gave us a general reference model that can be used outside the domain of biology. Topics like desease spread, stockmarket behaviour, agriculture, development of societies, climate change to the control of robots are today modeled and inspired by evoutionary principles. Th set of theories analytical models are now so detailed that they are or close to be proven.

Due to this last reason, an evolutionary prediction model given these theories would be rather neat. We could predict - given a set of factors - how humans, may evolve, how a desease will spread or maybe how societies may evolve and in that way use the models to set in countermeasures that can reduce the impact of an unwanted trend. There will never be a psychohistory like Isaac Asimov describes in his foundation series. Still, I believe the predictive gains would outweigh the misses.

As can be expected, this evolutionary prediction idea is not new. Darvin himself set out several predictions and many were proven. Some examples can be found here. There are also several evolutionary and genetic algorithims that are used extensively in many scientific fields - like the ones mentioned above.

What is most exciting though in relation to evolution , is the insights so well described in The Wisdom of Crowds (Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations).



Its central thesis is that a diverse collection of independently-deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts. To exemplify, lets mention a few cases from book:
  1. The best predictor of the next president in the US is a betting market on the next president.
  2. The best predictor of the number of jelly beans in a jar is a average vote between a diverse group of people.
In many ways this is exactly how evolution descibes it. The best solution is the most likely to survive given a diverse and selfish set to select from. The volume of the crowds is the selector.

This evolutionary crowdsourced prediction method has just started to be implemented. One of the best examples is how google has set up internal crowdsourcing to perform strategic prediction. Nice results! I bet Google will survive for quite some time in the evolutionary game of corporations. ;D

No comments: