Evolutionary Prediction - The Wisdom of the Crowds

From my point of view, the biggest single paradigm shift in human history can be marked by the introduction of theoy of evolution and the theory of natural selection. The theory not only gives us meaning in understanding the past, it is the basis for the most powerful ways we have to predicting the future.



First and foremost, it opened up a totally new perspective on the whole concept of humans and where we come from. Old school religious theories were thrown out (even though some hardcore bible-enthusiasts still cling to sci-fi concept of intelligent design).

Secondly, and maybe just as important it gave us a general reference model that can be used outside the domain of biology. Topics like desease spread, stockmarket behaviour, agriculture, development of societies, climate change to the control of robots are today modeled and inspired by evoutionary principles. Th set of theories analytical models are now so detailed that they are or close to be proven.

Due to this last reason, an evolutionary prediction model given these theories would be rather neat. We could predict - given a set of factors - how humans, may evolve, how a desease will spread or maybe how societies may evolve and in that way use the models to set in countermeasures that can reduce the impact of an unwanted trend. There will never be a psychohistory like Isaac Asimov describes in his foundation series. Still, I believe the predictive gains would outweigh the misses.

As can be expected, this evolutionary prediction idea is not new. Darvin himself set out several predictions and many were proven. Some examples can be found here. There are also several evolutionary and genetic algorithims that are used extensively in many scientific fields - like the ones mentioned above.

What is most exciting though in relation to evolution , is the insights so well described in The Wisdom of Crowds (Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations).



Its central thesis is that a diverse collection of independently-deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts. To exemplify, lets mention a few cases from book:

1. The best predictor of the next president in the US is a betting market on the next president.
   
2. The best predictor of the number of jelly beans in a jar is a average vote between a diverse group of people.

In many ways this is exactly how evolution descibes it. The best solution is the most likely to survive given a diverse and selfish set to select from. The volume of the crowds is the selector.

This evolutionary crowdsourced prediction method has just started to be implemented. One of the best examples is how google has set up internal crowdsourcing to perform strategic prediction. Nice results! I bet Google will survive for quite some time in the evolutionary game of corporations. ;D

The Seven Principles of Influence

The book Influence [1] gives a unique guide to how people are influenced by 7 different principles. Here we list them:

1. Perceptual Contrast - when one item is presented after a more expensive one, the first item presented will be considered cheap.

2. Recipocation - when a person is given something he will try to repay, in kind, what he was given. The strategy is to give a person some small favor (a rose, a candy) and get big benefits in return (your purchase)

3. Commitment and Consistency - once a person has commited to something he will continue being consistent to this commitment. Hence, by getting an initial commitment from one you wish to influence, the target will likely by consistent further on with his initial commitment.

4. Social Proof - people will tend to base their beliefs or act upon what other similar people in the same situation would believe in or act after. The social proof principle is the strongest when similar people are the basis for the proof or if the person to decide is unceirtain in the situation.

5. Liking - people tend to say yes to other people they like. Liking can be increased by such things as the others physical attractiveness, similarity, though compliments or through continued contact with the other person.

6. Authority - people with some sort of authority (knowledge, titles, clothes etc) will tend to have influence over others.

7. Scarcity - an item or other sort of quantity that is percieved as scarce will be considered attractive. People assign more varue to an opprotunity when it is considered less available.





[1] Influence: Science and Practice

Spreading Ideas by Exclusiveness and Secrecy

The heading may seem like a large contradiction, but I believe it is not.

The Tip from a Friend
Why does a tip from a friend mean so much to you compared to an ad in a magazine or on television? Well, friendship is strongly related to honesty and the ability to keep secrets between each other. When you break either of the two, the good relation is sacrificed. Hence, if you get a tip from a friend, you believe it because (1) he is not likely to lie and (2) this may be an important secret your friend tells you about.

In sources [1] and [2], the word-of-mouth-marketing (WOMM) idea is advised to be used more by companies. But, a WOM epidemic is hard to start. It is more art than science, but some references exist that may give some valuable advice. In "The Tipping Point" [3], a WOM-model is introduced, and I like it.

The main thesis of the book is that ideas, products and social phenomena may spread like epidemics. "The Tipping Point" happens when "the vital few" as apposed to the the trivial many starts to use, apply or advertise the idea and the "epidemic" spreads. Gladwell calls this group phenomenon the Law of the Few, and from my point of view it is simply another way of stating the Pareto Principle [3] (find the vital few that produce the impact).

To apply the Tipping Point insight, the key is how to select the vital few. Gladwell asserts that you need to find the "Connectors, Mavens, and Salesmen" that can cause the tip. Finding who those are again is still an art.

A great example of a product where the WOMM effect is seen, is Gmail. Yes, there has been considerable amounts of media coverage on the service, but the fact of the matter is that the service has never been advertised. Hence, the media coverage has only started by WOM. The service was originally also only accessible to those who have been invited. Hence, there became a sense of exclusiveness and secrecy in the service. The product then exploded!

Relationships of Trust

WOMM is great, but is not possible in all situations. The thing is that not all products can be marketed through your friends. Some products have the characteristics so that there is little interest for someone to tip a friend. Furthermore, there is for instance a lot of research that indicates that buyers do not tell about products or services they like, but they are very likely to tell their friends about the ones they do not like. There are some success stories of WOM-campaigns that spread by giving consumers incentives to spread the word - but there are in many markets legal challenges for these type of models.

So when WOMM does not work, you as a company often need to spread the word yourself. To be able to do that successfully you must build a relationship of trust with your customers [6]. In order to do that, why not look at the model for friendship and look for inspiration. Hence, giving a customer secret and good offers while keeping a sense of exclusiveness to the relationship is most likely what will attract and retain them.

[1] Word of Mouth Marketing - WOMM!
[2] Word-of-Mouth: The World's Best-Known Marketing Secret
[3] The Tipping Point

[4] The Pareto Principle
[5] GMail
[6] It's Not About Permission, It's About Trust

The Medici Effect

The Medici's were a very influential family empire in Florence from the 1300s to the 1600s. An amazing number of scientific discoveries were made during their era - Leonard Da Vinci being one of the most famous and influential of the discoverers. The accepted theory for why this era was so productive is that the culture was very open and cross disciplinarian. That again made innovation more plausible.

The Medici Effect [1] is a book that looks at the innovation from such inhomogeneous processes and how you can accelerate the process. The main thesis of the book is very much in line with what the notes on this blog is about : Innovation comes nowadays from the intersection between disciplines. You take ideas and concepts from different disciplines and combine them into new concepts.

The big question is how you or an organization can step into this intersection. According to the book there are 3 key elements that are important driving factors:

1. By diversifying occupations.
2. By interacting with diverse groups of people.
3. By going intersection hunting.

1 and 2 are fairly obvious. Being in a culture of diversification, curiosity and open mindedness are key factors for innovation. 3 is mostly based in the idea of creating directed randomness in your thought process. One method is to go for a stroll and purchase or pick up objects that are not related to your current problem. Look at these and let your thought roll.

I won't say that the book is very revolutionary, and the fact of the matter is that you need to have a mind that is creative and curious in order to create this effect. Most ideas are created/formulated by a few people compared to the population. These few people make up many ideas - many which never make the test of reality. Still, the thesis of the book is important to remember, so if you want to create an innovative culture it would not be so silly to study the Medici's.

[1] Medici Effect: What Elephants and Epidemics Can Teach Us About Innovation

Power Laws and Long Tails

The standard normal distribution (bell curve) is today dominant when we model phenomena. It often works out quite nicely with reality, but it has one easily to forget limitation : It assumes that the occurrences are independent. In many cases independence is very limiting. Interdependence needs to be taken in account. In these cases power laws are more accurate (For a definition of a power law see [1]). The result is that extreme events become much more probable.

Look at the diagram below from [3] (also a great overview of power laws). As you can see, a bell curve will predict that unlikely events are very rare. In cases of independence of phenomena that is likely. A power law distribution looks similar, but has a "fat tail". The implication is that it predicts that rare events are much more likely (you may say that the interdependence creates a possible "butterfly effect").





An example from [2] (a brilliant book by the way) is illustrative. In the book business, the largest brick and mortar shops can keep an inventory of maximum 100.000 books. After that the business of keeping them in store becomes unprofitable. A site like Amazon.com has an inventory of over 1.5 million books and growing. Even though it sells only a a few copies of the millionth popular book, it still makes a profit on it. That's due to the low costs of storage, distribution and sales.

An example from [3] is another illustration : "In the case of market fluctuations, for example, the bell curve predicts a one-day drop of 10 percent in the valuation of a stock just about once every 500 years. The empirical power law gives a very different and more reliable estimate: about once every five years." As the reader may know, we see 10 percent drops fairly often!

[1] Wikipedia - Power Law
[2] The Long tail
[3] Power Laws & the New Science of Complexity Management

Principles of the Greatest Companies

What makes a company truly great?

The research by Jim Collins presented in his book Good to Great [1] from 2000 gives me a lot of inspiration. Collins researched the stock market and found the companies that had beat their market segment by 7 to 15 times over a 15 year period. The team then found the common qualities the great companies shared. Some of the results were really surprising. Interestingly, the list did not include companies like GE, Coca Cola and Microsoft. These did not qualify.

Below are the qualities :

1. Level five leadership
2. First Who, Then What. Pick the right leadership team
3. Confront the brutal facts - Murphy's Law
4. Hedgehog Concept (Strategy)
5. Discipline in culture
6. Use technology to accelerate

See [2] for a thorough walk through of the complete set of ideas that were introduced in the book. The book is also a very good read.

[1] Good to Great - cool study of the greatest business transformations.
[2] Jim Collins' web site