The value of prediction markets
March 8th, 2009There has recently been quite a bit of discussion on MidasOracle over the value of prediction markets. Part of this was sparked off by the recent Economist article on prediction markets, but part of it has to do with things Chris Masse has been writing for some time now. Chris is very vocal about his views, and I wanted to put a different perspective forward here.
Economist article
A recent article on prediction markets in the Economist was titled “An uncertain future.” I would describe its attitude toward prediction markets as mixed. Here are a few selected quotes:
“But although they have spread beyond early-adopting companies in the technology industry, they have still not become mainstream management tools. Even fervent advocates admit much remains to be done to convince sceptical managers of their value.”
“Koch says the results so far have been pretty accurate compared to actual outcomes, but stresses that markets are complementary to other forecasting techniques, not a substitute for them.”
“Another reason prediction markets flop is that employees cannot see how the results are used, so they lose interest. [...] its most effective trials took place in areas where managers could do something with their findings, making staff feel that trading was worthwhile.”
“Bosses may also be wary of relying on the judgments of non-experts. Yet many pilot projects run so far have shown that junior staff can often be surprisingly good forecasters.”
So I believe the article is generally positive toward the possibility of prediction markets, but generally negative toward the lack of adoption. Which is fair; though I think some good success stories have been kept quiet by some companies for competitive advantage and corporate ego.
Chris Masse’s reaction
Chris Masse has been writing about the prediction market industry for a number of years, and started his blog MidasOracle in about 2006. Strangely, Chris’ writings on prediction markets have become quite negative in recent months. As I understand it, he’s become quite irritated that prediction markets have been touted as “vastly superior” to polls in elections, and other general marketing in the industry. The front page of MidasOracle now has the following statement (his emphasis removed):
The Truth about Prediction Markets
The social utility of the prediction markets is marginal. Number one, the aggregated information has value only for the totally uninformed people (a group that comprises those who overly obsess with prediction markets and have a narrow cultural universe). Number two, the added accuracy (if any) is minute, and, anyway, doesn’t fill up the gap between expectations and omniscience (which is how people judge forecasters). In our view, the social utility of the prediction markets lays in efficiency, not in accuracy. In complicated situations, the prediction markets integrate expectations (informed by facts and expertise) much faster than the mass media do. Their accuracy/efficiency is their uniqueness. It is their velocity that we should put to work.
Remember, dear readers, you heard it here first —on Midas Oracle.
Part of my issue is his definition of “social utility”: namely that I’m not exactly sure what he’s talking about. I completely understand the concept of utility, but “social utility” is a phrase that could mean a variety of things, and I don’t know exactly what Chris means when he writes about it.
My other response to Chris is that I feel he doesn’t distinguish between public and internal prediction markets when he discusses marketing and usage of these markets. I’d like to discuss these issues here.
Public prediction markets
Public prediction markets are tricky things. They’re tricky to get a critical mass of people involved, they’re tricky to get a flow of interesting and valuable contracts, and they’re tricky when events happen that don’t easily conform to how the contracts were established. That said, there are a handful of good and interesting public PM’s. InTrade and Betfair both offer real-money markets, and the better play-money markets include Hubdub, Inkling, Newsfutures, and HSX.
These markets do get press, particularly InTrade in the run-up to the last two presidential elections. InTrade in particular seems to be irritating Chris because of the data point that they forecasted all 50 states correctly. (Which as I’ve written before probably means that the markets weren’t running as efficiently as they could; you’re supposed to get market “failures”.) And I agree with him that too much has been made of this data point.
Overall, I think Chris rightly addresses the issue of the value of public prediction markets. In many cases they’re purely entertainment, and in other cases (even the elections), they’re not necessarily better predictors than other methods. That said, they do incorporate new information faster than any other forecasting model. But I believe Chris wrongly applies these same criticisms to a completely different model, internal prediction markets.
Internal (corporate) prediction markets
Internal prediction markets seek to do either of two things:
- forecast something important to the company where they already have a prediction
- forecast something that’s never been predicted before
When it comes to the first point, forecasting something that the company already forecasts, prediction markets may or may not be an excellent solution. I’ve seen one set of markets that absolutely blew away the accuracy of current forecasts, and I’ve seen other markets that were consistent with current forecasts with little or no accuracy edge. In this case, prediction markets can serve as a fairly low-cost “reality check” on the official forecasts. When they deviate too much, it will raise a flag for investigation. This deviation is particularly helpful since prediction market forecasts are real-time, and can react instantly to new news.
I think the second point above is perhaps the most valuable opportunity. Simple ideas like using a prediction market to establish a RAG status could potentially be very powerful in a company. Depending on the value of the information forecasted, even a moderately accurate prediction market could be incredibly useful to a company.
Chris also over-reacts to some of the marketing from software vendors. Yes, some may mention events like InTrade predicting all 50 states in 2004, but that’s just a party trick to get people interested and in the door. Companies first need to be convinced that the tool works, then they need to be convinced that it works for them. The second part is done individually with their clients so we don’t see it. (Though I really like the case studies that Inkling have put on their website.)
I personally think that he gives too much credit to the popular press in raising the profile of prediction markets; far more credit is due to James Surowiecki and “The Wisdom of Crowds
Summary
Prediction markets are interesting tools, but the lessons learned from public prediction markets are different than those learned from internal prediction markets. It’s important that these two applications are not confused. There will always (and rightly) be questions about the accuracy of prediction markets. In some cases it’s clear that markets are superior, in some internal cases it doesn’t matter since nothing is forecasted right now, and in other cases markets will be about as accurate as what’s already predicted. But in all three cases there is still a valuable argument as to why prediction markets should be used. It all comes down to the specific needs of an industry or company, which is where the vendors step in to help.
The results speak for themselves. Each year there are more software vendors, and each year the existing software vendors hire more people to serve their clients. I’m not going to try and predict the long-term future, but I believe the short-term future is positive.
