The Power and Beauty of the Insightful Statistical Prediction

Hans Rosling [1] have for some years now inspired me and many others in his enchanting way of presenting statistics and future prediction. This talk, describing the rise of Asia as the dominant force in the world is his best ever:

http://www.ted.com/talks/hans_rosling_asia_s_rise_how_and_when.html

Look at his humble intro and natural approach throughout.
Look at the elegant way of using dynamics in the graph to prove the prediction.
Look at how well he matches and lifts his audience and their final reaction to his presentation.

Great people project great ideas.




[1] http://www.gapminder.org/

Ten Greatest Thinkers of All Time

One of the great 20th century historians Will Durant, made in some of his assays rankings of what from his perspective were The Greatest Minds and Ideas of All Time. The essays are summed up in the book with the same name.



The book is a fun read, and is recommended as a reference for further study. Here I list his "Ten Greatest Thinkers of all Time". I think he is pretty close to a perfect list, but some of them might be more controversial than others - depending on your favorites:

1. Confucius
2. Plato
3. Aristotle
4. St. Thomas Aquinas
5. Copernicus
6. Sir Francis Bacon
7. Sir Isaac Newton
8. Voltaire
9. Immanuel Kant
10. Charles Darwin

The whole essay can be read under chapters published in a blog with entries for each person on the list here and here.

I'll come back with his list of The Greatest Ideas of all time.

The Wonder and Paradoxes of Penrose's World Projections

I am a big fan of Roger Penrose, and his work is absolutely brilliant reads, if you have the time and energy to truly concentrate on his writings (for my favorites see [1] and [2]).

One model that is introduced in his work is the "the three worlds" and its relations, mysteries and paradox. The idea is depicted below (taken directly from [1] ) and describes how the physical world of existance is governed by the laws of the platonic/mathematical world, while the mental world is a part of our mental world, while our mental world is a small part of physical existance!



If you - as I do - see the model as relevant notice the following 3 mysteries and one paradox:

1. Why is it that only a small part of the world of mathematics have relevance to the workings of the physical world?
2. Why does the mental world come about in association with only certain physical structures - healthy, wakeful human brains.
   
3. Why does only a small fraction of our mental minds concerns itself with mathematical/platonic truth?
4. Finally the paradox - how come the three worlds appear to encompass the next one in its entirety?
   

In Penrose's own words : "Maybe the three worlds are not separate at all, but merely reflect, individually, aspects of a deeper truth about the world as a whole which we have little conception of at the present time.

Whatever your standpoint, the model is so powerful and creates a reflection and controversy it deserves to be called a great idea.


[1] The Emperor's New Mind: Concerning Computers, Minds, and the Laws of Physics (Popular Science)


[2] The Road to Reality : A Complete Guide to the Laws of the Universe

Violence in Decline

In contrast to popular opinion, violence in the world has been continually declining since biblical times to the present. This continual decline correlates strongly with the gathering of societies in centralized states and democracies and in parallell to the increase in reason and technology. Especially, the 16th century - the start of the age of reason - marked the start of a unprecedented drop in violoence and rates of deaths in war.


See this great TED conference lecture by Steven Pinker for the whole inpiring story (see this to read the whole lecture). Enjoy!

According to Pinker, several theories have been noted to explain the effect. All of these have some truth to them I am sure:

1. "Hobbes got it right" - In a state of anarchy, there's a constant temptation to invade your neighbors pre-emptively, before they invade you. A centralized state this is not such a great problem.
   
2. "Life is cheap" - "When suffering & early death are common in one's own life, one has fewer compunctions about inflicting them on other. Technology and economic efficiency make life longer and more pleasant, one puts a higher value on life in general"
3. "The non-zero sum game" - "..in certain circumstances, cooperation or non-violence can benefit both parties in an interaction, such as gains in trade"
4. "The Expanding Circle" -Evolution bequeathed us with a sense of empathy. By default, we apply it only to friends & family, over history, the circle has expanded:
   -village->clan->tribe->nation->other races->both sexes.

Principle of Universal Options

Whatever the challenge, field or situation, there are options to invest in to hedge risk and unceirtainty.



This is a rather bold assertion, but I believe it to be very true. Some examples:

Stock market: Options and other derivatives.
Commodity and currency markets: Hedging or futures.
Business and personal economy: Insurance, portfolio management,
Personal health: Excercise, healthy food, vacations
Personal Career: Education, training, parallell ventures
Project Management: Real Options, Risk and change management
Engineering: Modularity, flexibility
Medicine and psychology: Alternative treatment forms

My note to self? Whatever the challenge, there will be heging possibilities that for a limited cost will reduce the risk we are facing.

Evolutionary Prediction - The Wisdom of the Crowds

From my point of view, the biggest single paradigm shift in human history can be marked by the introduction of theoy of evolution and the theory of natural selection. The theory not only gives us meaning in understanding the past, it is the basis for the most powerful ways we have to predicting the future.



First and foremost, it opened up a totally new perspective on the whole concept of humans and where we come from. Old school religious theories were thrown out (even though some hardcore bible-enthusiasts still cling to sci-fi concept of intelligent design).

Secondly, and maybe just as important it gave us a general reference model that can be used outside the domain of biology. Topics like desease spread, stockmarket behaviour, agriculture, development of societies, climate change to the control of robots are today modeled and inspired by evoutionary principles. Th set of theories analytical models are now so detailed that they are or close to be proven.

Due to this last reason, an evolutionary prediction model given these theories would be rather neat. We could predict - given a set of factors - how humans, may evolve, how a desease will spread or maybe how societies may evolve and in that way use the models to set in countermeasures that can reduce the impact of an unwanted trend. There will never be a psychohistory like Isaac Asimov describes in his foundation series. Still, I believe the predictive gains would outweigh the misses.

As can be expected, this evolutionary prediction idea is not new. Darvin himself set out several predictions and many were proven. Some examples can be found here. There are also several evolutionary and genetic algorithims that are used extensively in many scientific fields - like the ones mentioned above.

What is most exciting though in relation to evolution , is the insights so well described in The Wisdom of Crowds (Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations).



Its central thesis is that a diverse collection of independently-deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts. To exemplify, lets mention a few cases from book:

1. The best predictor of the next president in the US is a betting market on the next president.
   
2. The best predictor of the number of jelly beans in a jar is a average vote between a diverse group of people.

In many ways this is exactly how evolution descibes it. The best solution is the most likely to survive given a diverse and selfish set to select from. The volume of the crowds is the selector.

This evolutionary crowdsourced prediction method has just started to be implemented. One of the best examples is how google has set up internal crowdsourcing to perform strategic prediction. Nice results! I bet Google will survive for quite some time in the evolutionary game of corporations. ;D

Broken Windows

Have you ever left a cup on the kitchen table and an hour later the whole kitchen is a mess?



If you have you have experienced the broken window theory:

The observation that a few broken windows in an empty building quickly lead to more smashed panes, more vandalism and eventually to break-ins. The tendency for people to behave in a particular way can be strengthened or weakened depending on what they observe others to be doing.

The thory developed by Dr. Keller in the beginning of the eighties has been a role model for crime reduction in several US cities and studies show that it has some sense to it. According to the theory, the prevention of the effect is to stop all kinds of small crimes - like grafitti, shoplifting etc - and violent and serious crime will drop as well. Since the theory was implemented in practice in many US cites (most famous is New York), you can see that violent crime rates dropping in many US cities.



Yes there are critics to the theory, but it still makes sense. Personally, I see that the insight is relevant in so many other areas that affect us - from the private to our business lifes. Hence, a great idea.

Clean you desk, wash your car, put the cups in the washing machine, allow no sloppy work in your team. Who knows what may happen if you don't.

Hype Cycles

The technology maturity hype cycles of 2009 from Gartner were just released. Since working as a consultant during the boom years around Y2K the hype cycle model has fascinated.

It is such a simple way to chart a very complex phenomena and convey a smart insight at the same time.

Highlights this year is that cloud computing and e-book readers are at inflated exceptions and are about to crash. Another assertion is that microblogging services like Twitter/Facebook are about to crashing this instant due to spam and noise. Wikis on the other hand are on the return.

Gartner might be right about their predictions and they are probably wrong on several as well, but that is not the only point. Thanks to this model, discussions start and new ideas pop up. Sit down and design your own hype cycle it in your business area, and its almost magical how the pieces fall into place and insights start to flow.

Access to more info at: http://www.gartner.com/it/page.jsp?id=1124212

90-9-1 Community!

As described here before [1], power laws are very potent at describing many type of phenomena. Latest principle is the 90-9-1 principle of online communities [2].

The concept is simple. In an online community, 1 percent are creators of content, 9 percent are editors of content and the remaining 90 percent are just the curious audience watching. The phenomenal thing is that this rule seems to be proportional, so that getting more creators to a site, increases the editors and the audience.


My bet is that many users act in different roles depending on the site they use, but that there is a small minority of heroes out there that generates most of the community content - and hence drives the traffic.


[1] The Vital Few or the Trivial Many?
[2] 90-9-1 Principle