Mercury's Blog

The field of prediction markets seems to be going through a bit of a crisis of confidence recently. I would personally trace it to the recent election (where other forecasters like Nate Silver made forecasts as good as PM’s), as well as recent press like the Economist article. The general feeling is a questioning of prediction markets: if they’re so good at forecasting, why aren’t they being used much more widely? I think I have the start a good reason why.

Core Issue

Traditional forecasting is done through highly analytical techniques using past data. Statistical measures are used to generate forecasts, with probability ranges. This industry is quite large, and is highly exacting.

Prediction markets take an orthogonal approach to traditional forecasting. Instead of a “top-down” approach where huge data sets are analyzed, prediction markets use a “bottom-up” approach that combine individuals’ forecasts.

The reason prediction markets haven’t been adopted widely is because they are a tool that approaches the forecasting problem from a completely different perspective.

An example – Enterprise Business Intelligence

I’ve recently been looking into the Enterprise Business Intelligence/Business Management industry, and came across what I think is a similar phenomenon. The vast majority of the industry is composed of massive analytical solutions from the likes of SAP, Oracle, IBM, etc. They are massive companies, and implementing a “solution” can easily take a year or more. Their clients design the system from a “top-down” perspective, determining from the outset what the processes and procedures are going to be.

But then there is software like Thingamy. Thingamy is the creation of Sig Rinde, a Norwegian living in the south of France. Instead of looking at enterprise business intelligence from the top-down, he has created software that approaches the problem from the bottom-up. Instead of establishing pre-defined processes (that may not even work or will be changed by the time the software is configured), Thingamy tracks emergent processes as they happen. It can start with a very small, hard-to-define process and then scales up as the business needs it.

While Thingamy has gotten some good press and attention over the years, it’s still a fairly small company. Again, I believe this is because it takes a fundamentally different approach to the problem compared to the rest of the current industry. Hugely different approaches cause cognitive dissonance, which slow adoption.

What does this mean?

There are new types of technologies that approach business problems from entirely different directions. Prediction markets is one of these technologies. Using PM’s means companies have to upset some of their current notions about how power and influence flow in a company, relying on “soft” information from lower-level employees. A different approach also means that in certain situations they’ll be clearly superior, but also that in other situations they won’t be. Traditional methods and thumb-rules for situations just don’t automatically work.

For example, prediction markets where there is a lot of public information (like election markets) may prove to integrate new news and information more quickly, but may not be quite as accurate as other methods in the final analysis. But where information is scarce (like some internal corporate forecasts), a prediction market may be ideal. In general, new ways of thinking have to be established to know when and where to use this new tool effectively. That’s why I believe prediction markets will take quite some time to see any sort of a spike in growth; expect a slow burn for a long time.

Quick note

Just a quick note for you all. I’m curious about how well Google’s AdSense can be used to monetize a blog, so I’m going to be running AdSense on this blog on a one-month trial. If you have any opinions on this, please feel free to e-mail me or comment below.

I wanted to take a minute and mention that HubDub is running a March Madness “Predict-A-Thon” prediction market tournament. For someone like myself that hasn’t followed college basketball in a while, this is a far more interesting way to see the tournament unfold.

As I write this, Jason Trost of Smarkets fame is currently atop the leaderboard, but I expect this may very well change over the course of the next days/weeks. HubDub’s founder Nigel has climbed from the basement of the list, where he was third from the bottom in the early days!

HubDub is trying out a new model with this tournament, and will be awarding cash prizes to the top traders! If you think you’ve got what it takes just click here to sign up now.

On a final note, I’d like to say that it’s about time that my alma mater’s basketball team, the University of Michigan Wolverines, finally made it back to the NCAA tournament. Unfortunately, this was likely due to cosmic imbalance in sporting karma from the Wolverines football team collapsing this year. It would be great if they could at least make it through the first round of the tournament to start evening things out a bit.

There has recently been quite a bit of discussion on MidasOracle over the value of prediction markets. Part of this was sparked off by the recent Economist article on prediction markets, but part of it has to do with things Chris Masse has been writing for some time now. Chris is very vocal about his views, and I wanted to put a different perspective forward here.

Economist article

A recent article on prediction markets in the Economist was titled “An uncertain future.” I would describe its attitude toward prediction markets as mixed. Here are a few selected quotes:

“But although they have spread beyond early-adopting companies in the technology industry, they have still not become mainstream management tools. Even fervent advocates admit much remains to be done to convince sceptical managers of their value.”

“Koch says the results so far have been pretty accurate compared to actual outcomes, but stresses that markets are complementary to other forecasting techniques, not a substitute for them.”

“Another reason prediction markets flop is that employees cannot see how the results are used, so they lose interest. [...] its most effective trials took place in areas where managers could do something with their findings, making staff feel that trading was worthwhile.”

“Bosses may also be wary of relying on the judgments of non-experts. Yet many pilot projects run so far have shown that junior staff can often be surprisingly good forecasters.”

So I believe the article is generally positive toward the possibility of prediction markets, but generally negative toward the lack of adoption. Which is fair; though I think some good success stories have been kept quiet by some companies for competitive advantage and corporate ego.

Chris Masse’s reaction

Chris Masse has been writing about the prediction market industry for a number of years, and started his blog MidasOracle in about 2006. Strangely, Chris’ writings on prediction markets have become quite negative in recent months. As I understand it, he’s become quite irritated that prediction markets have been touted as “vastly superior” to polls in elections, and other general marketing in the industry. The front page of MidasOracle now has the following statement (his emphasis removed):

The Truth about Prediction Markets

The social utility of the prediction markets is marginal. Number one, the aggregated information has value only for the totally uninformed people (a group that comprises those who overly obsess with prediction markets and have a narrow cultural universe). Number two, the added accuracy (if any) is minute, and, anyway, doesn’t fill up the gap between expectations and omniscience (which is how people judge forecasters). In our view, the social utility of the prediction markets lays in efficiency, not in accuracy. In complicated situations, the prediction markets integrate expectations (informed by facts and expertise) much faster than the mass media do. Their accuracy/efficiency is their uniqueness. It is their velocity that we should put to work.

Remember, dear readers, you heard it here first —on Midas Oracle.

Part of my issue is his definition of “social utility”: namely that I’m not exactly sure what he’s talking about. I completely understand the concept of utility, but “social utility” is a phrase that could mean a variety of things, and I don’t know exactly what Chris means when he writes about it.

My other response to Chris is that I feel he doesn’t distinguish between public and internal prediction markets when he discusses marketing and usage of these markets. I’d like to discuss these issues here.

Public prediction markets

Public prediction markets are tricky things. They’re tricky to get a critical mass of people involved, they’re tricky to get a flow of interesting and valuable contracts, and they’re tricky when events happen that don’t easily conform to how the contracts were established. That said, there are a handful of good and interesting public PM’s. InTrade and Betfair both offer real-money markets, and the better play-money markets include Hubdub, Inkling, Newsfutures, and HSX.

These markets do get press, particularly InTrade in the run-up to the last two presidential elections. InTrade in particular seems to be irritating Chris because of the data point that they forecasted all 50 states correctly. (Which as I’ve written before probably means that the markets weren’t running as efficiently as they could; you’re supposed to get market “failures”.) And I agree with him that too much has been made of this data point.

Overall, I think Chris rightly addresses the issue of the value of public prediction markets. In many cases they’re purely entertainment, and in other cases (even the elections), they’re not necessarily better predictors than other methods. That said, they do incorporate new information faster than any other forecasting model. But I believe Chris wrongly applies these same criticisms to a completely different model, internal prediction markets.

Internal (corporate) prediction markets

Internal prediction markets seek to do either of two things:

• forecast something important to the company where they already have a prediction
• forecast something that’s never been predicted before

When it comes to the first point, forecasting something that the company already forecasts, prediction markets may or may not be an excellent solution. I’ve seen one set of markets that absolutely blew away the accuracy of current forecasts, and I’ve seen other markets that were consistent with current forecasts with little or no accuracy edge. In this case, prediction markets can serve as a fairly low-cost “reality check” on the official forecasts. When they deviate too much, it will raise a flag for investigation. This deviation is particularly helpful since prediction market forecasts are real-time, and can react instantly to new news.

I think the second point above is perhaps the most valuable opportunity. Simple ideas like using a prediction market to establish a RAG status could potentially be very powerful in a company. Depending on the value of the information forecasted, even a moderately accurate prediction market could be incredibly useful to a company.

Chris also over-reacts to some of the marketing from software vendors. Yes, some may mention events like InTrade predicting all 50 states in 2004, but that’s just a party trick to get people interested and in the door. Companies first need to be convinced that the tool works, then they need to be convinced that it works for them. The second part is done individually with their clients so we don’t see it. (Though I really like the case studies that Inkling have put on their website.)

I personally think that he gives too much credit to the popular press in raising the profile of prediction markets; far more credit is due to James Surowiecki and “The Wisdom of Crowds.” (Amazon link)

Summary

Prediction markets are interesting tools, but the lessons learned from public prediction markets are different than those learned from internal prediction markets. It’s important that these two applications are not confused. There will always (and rightly) be questions about the accuracy of prediction markets. In some cases it’s clear that markets are superior, in some internal cases it doesn’t matter since nothing is forecasted right now, and in other cases markets will be about as accurate as what’s already predicted. But in all three cases there is still a valuable argument as to why prediction markets should be used. It all comes down to the specific needs of an industry or company, which is where the vendors step in to help.

The results speak for themselves. Each year there are more software vendors, and each year the existing software vendors hire more people to serve their clients. I’m not going to try and predict the long-term future, but I believe the short-term future is positive.