Some Predictions on Prediction Markets
With YAGA (yet another Google article) out recently on the use of prediction markets at the company, I thought I’d comment on whether prediction markets are, as many would have it, the next big thing. The article notes that Google runs a variety of prediction markets to get a bottom-up reading on questions like how many users a new product will attract, when a new office in a particular city will be opened, and so forth. There is not much doubt that prediction markets are useful, as not only Google but also Hewlett-Packard (HP used them for sales forecasts several years ago), the Hollywood Stock Exchange, the Iowa Electronic Markets, New Yorker writer Jim Surowiecki (in The Wisdom of Crowds) and a few other companies have discovered.
But why so few? If this is such a great way to generate useful insights for decision-making (and I believe it is), why don’t more companies take advantage of it? If the crowds are so wise (and they are, under the right circumstances), why do most senior executives ignore them? Will we see prediction markets spreading over lots of different organizations? Is this the new way to be wise?
I think the barriers to adoption of prediction markets are primarily cultural, and I don’t see them changing anytime soon. Let’s say that your company runs a prediction market on first-year sales of a new product, and the results come out not so positively. Let’s say that the employees who participate predict much lower sales than, say, the product manager for the new product, the division president, and the CEO. The fact that the crowd may be more accurate is not the point, or at least not the only point. The crowd has made the hierarchy look bad, and the hierarchy doesn’t generally like to look bad.
For a company culture to value prediction markets, its culture would need to have certain (rare) attributes:
• Confidence that executives have valuable roles to play even if they don’t always have the right answer;
• A high level of trust in the intelligence and capabilities of employees;
• The willingness to follow numbers and analysis wherever they lead (as long as they are more-or-less consistent with common sense);
• A pretty strong analytical and financial orientation (since futures markets aren’t something that every Joe or Jane understands).
Google seems to have all these attributes. HP had them at one point, but then lost them under Carly Fiorina -- which may be why we don’t hear much about prediction markets at HP anymore. Most organizations don’t have them. I wish we would see lots more prediction markets, but I suspect we won’t.
- Comments (3)
- Join the Discussion
- Email/Share
Tom Davenport holds the President’s Chair in Information Technology and Management at Babson College, where he also leads the 
Comments
I have spent some time looking at practical implementation of Prediction Markets. They have an intellectual logic that makes them intriguing, but so many negative practical aspects as to make them unusable (until a new format comes into existence). My experience, by the way, is in implementing enterprise collaborative innovation systems for 10 years, with 100 corporate clients.
My observations on Prediction Markets are (in no particular order):
- there is little to zero trading
--> they are meant to allow users to incorporate new information, and to trade on that information, but in fact users only go in once, occasionally twice, and do not update their trades
- there is no real commitment to any trade
--> individuals trading stocks tend to be working with their own money (or other people's money - which impacts their bonus). Therefore people care a lot about the outcomes of their trades. PM systems do not ask for corporate staffers to put in their own cash, so it is obviously play 'currency' at best, which negatively impacts the quality of their decisions
- there is either too little volume of choices, or way too many
--> most of the 'successful' PM systems I have studied include either very few choices (i.e. a = "drug will be FDA approved" and b = "drug will launch in Feb") or way too many (i.e. "please provide your views on 50 choices"). Where there are few variables, people can concentrate their thinking. Where there are many variables, effects such as 'reviewer fatigue' take place, whereby the reviewer scores consistently for the first 5 - 10 items, and the reviews chaotically for the rest. This reduces the quality and consistency of the group's overall score
- few PM advocates use the system twice
--> what is really fascinating about enterprise systems is that they are only good if they get repeat use. Anyone can get a story in a magazine for something new they have done (even in HBR). However, if no one wants to use the same system in the future, there is something wrong. Yes, it is possible, and indeed credible, to explain away any reluctance to reuse ("we paid for a pilot, but the real system was too expensive"). And yet, if a system really is this good, people would use it more and more. This is simply not the case - ask HP.
I do believe there is a cultural adoption problem, but it is more in management's reluctance to try new management tools, of which PM is just one example. In terms of Prediction Markets themselves, I honestly believe that the majority of models are sufficiently flawed as to make them unusable for more than just a one-off, play project.
Mark Turrell
CEO, Imaginatik plc
Personal Blog: (a href="http://innovationBBL.blogspot.com">http://innovationBBL.blogspot.com
- Posted by Mark Turrell
January 15, 2008 08:11
Tom,
I agree that there is tremendous value in prediction markets. I have argued that prediction markets, along with blogs and wikis are "bureacracy termites" -- eating away at the natural lethargy of large complex organizations.
One of the most important places where companies can use prediction markets is in product funding new ideas and products -- which yes, Google does. Capital budgeting is often a process of giving more and more condensed information to decision makers who know less and less of the rich context of the product or service.
This is a big problem because innovations are incredibly dependent on people who have local knowledge of the customer needs, customer setting, technology, likely implementation issues, and so forth. The expertise that resides with those people close to the problem, and close to the customer is very hard to transfer to others.
A prediction market allows for all the contextual expertise to come to bear, become aggregated and organiations thereby improving the likelihood of success. However, I agree with you Tom, we won't see widespread adoption of such a model - because it fights those bureacratic, hierarchical tendencies which have been with us since the Romans and before. Of course, those few firms that do use them well will pull ahead faster.
For the more academically minded: Michael Jensen's work on Agency Theory gives a wonderful theoretical framework for this issue noting that the relevant knowledge is "impacted" and therefore creates more agency costs. From this theortical point of view, prediction markets "unimpact" the knowledge, and therefore lower the agency costs of decisions dependent on impacted knowledge. Better principal agent alignment does result and creates value for the shareholders. See the Harvard University Press book by him Foundations of Organizational Strategy, 2001.
- Posted by John Sviokla
January 15, 2008 20:56
Dear Prof.Davenport,
A majority of organizations do not bother to use predictive tools due to the following reasons:
1. A basic assumption of organizations is that intelligence, the ability to innovate, the ability to look at the big picture - are all concentrated at the top. Uncomfortable questions are rarely, if at all, relished.
2. The best organizations (on any measurable parameter) are those in which trust is a way of life. And yet, others do not wish to learn from these. Suspicion is rampant in organizations.
3. We start with some pre-dispositions - vision, strategic intent - take your pick. Anything that challenges this notion, even when backed by facts and figures, is not tolerated.
4. Analysis is important and many organizations do have the required competencies. What is lacking is synthesis - the ability to place the pieces of a jigsaw in proper order and come up with a beautiful mosaic. Our education is tailor-made for analysis but does not nurture the qualities of synthesis.
As the saying goes, those who do not learn from history are condemned to repeat it. A significant proportion of organizations appear to be content in making history, although of the wrong kind.
Warm regards
- Posted by B V Krishnamurthy
January 16, 2008 23:50