The Power and Beauty of the Insightful Statistical Prediction

Hans Rosling [1] have for some years now inspired me and many others in his enchanting way of presenting statistics and future prediction. This talk, describing the rise of Asia as the dominant force in the world is his best ever:

http://www.ted.com/talks/hans_rosling_asia_s_rise_how_and_when.html

Look at his humble intro and natural approach throughout.
Look at the elegant way of using dynamics in the graph to prove the prediction.
Look at how well he matches and lifts his audience and their final reaction to his presentation.

Great people project great ideas.




[1] http://www.gapminder.org/

Violence in Decline

In contrast to popular opinion, violence in the world has been continually declining since biblical times to the present. This continual decline correlates strongly with the gathering of societies in centralized states and democracies and in parallell to the increase in reason and technology. Especially, the 16th century - the start of the age of reason - marked the start of a unprecedented drop in violoence and rates of deaths in war.


See this great TED conference lecture by Steven Pinker for the whole inpiring story (see this to read the whole lecture). Enjoy!

According to Pinker, several theories have been noted to explain the effect. All of these have some truth to them I am sure:

1. "Hobbes got it right" - In a state of anarchy, there's a constant temptation to invade your neighbors pre-emptively, before they invade you. A centralized state this is not such a great problem.
   
2. "Life is cheap" - "When suffering & early death are common in one's own life, one has fewer compunctions about inflicting them on other. Technology and economic efficiency make life longer and more pleasant, one puts a higher value on life in general"
3. "The non-zero sum game" - "..in certain circumstances, cooperation or non-violence can benefit both parties in an interaction, such as gains in trade"
4. "The Expanding Circle" -Evolution bequeathed us with a sense of empathy. By default, we apply it only to friends & family, over history, the circle has expanded:
   -village->clan->tribe->nation->other races->both sexes.

Evolutionary Prediction - The Wisdom of the Crowds

From my point of view, the biggest single paradigm shift in human history can be marked by the introduction of theoy of evolution and the theory of natural selection. The theory not only gives us meaning in understanding the past, it is the basis for the most powerful ways we have to predicting the future.



First and foremost, it opened up a totally new perspective on the whole concept of humans and where we come from. Old school religious theories were thrown out (even though some hardcore bible-enthusiasts still cling to sci-fi concept of intelligent design).

Secondly, and maybe just as important it gave us a general reference model that can be used outside the domain of biology. Topics like desease spread, stockmarket behaviour, agriculture, development of societies, climate change to the control of robots are today modeled and inspired by evoutionary principles. Th set of theories analytical models are now so detailed that they are or close to be proven.

Due to this last reason, an evolutionary prediction model given these theories would be rather neat. We could predict - given a set of factors - how humans, may evolve, how a desease will spread or maybe how societies may evolve and in that way use the models to set in countermeasures that can reduce the impact of an unwanted trend. There will never be a psychohistory like Isaac Asimov describes in his foundation series. Still, I believe the predictive gains would outweigh the misses.

As can be expected, this evolutionary prediction idea is not new. Darvin himself set out several predictions and many were proven. Some examples can be found here. There are also several evolutionary and genetic algorithims that are used extensively in many scientific fields - like the ones mentioned above.

What is most exciting though in relation to evolution , is the insights so well described in The Wisdom of Crowds (Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations).



Its central thesis is that a diverse collection of independently-deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts. To exemplify, lets mention a few cases from book:

1. The best predictor of the next president in the US is a betting market on the next president.
   
2. The best predictor of the number of jelly beans in a jar is a average vote between a diverse group of people.

In many ways this is exactly how evolution descibes it. The best solution is the most likely to survive given a diverse and selfish set to select from. The volume of the crowds is the selector.

This evolutionary crowdsourced prediction method has just started to be implemented. One of the best examples is how google has set up internal crowdsourcing to perform strategic prediction. Nice results! I bet Google will survive for quite some time in the evolutionary game of corporations. ;D

The Spooky Explanation for Pretty Girls

According to [1], beautiful parents are 36 percent more likely to have a daughter than a son. Hence, beautiful women are easier to find than attractive men. The hypothesized reason for this statistical result is that "evolutionarily speaking, beauty is a trait that is more valuable for women than for men". Selection pressure hence gives the beautiful parents pretty girls.

Similar results apply to parents working in "empathic" occupations such as nurses, social workers and kindergarten teachers. On the other hand scientists, mathematicians and engineers are more likely to have sons than daughters.

Did anyone say politically incorrect?

[1] Why Do Beautiful Women Sometimes Marry Unattractive Men?