Mercury's Blog

Bo Cowgill, Justin Wolfers and Eric Zitzewitz released their paper “Using Prediction Markets to Track Information Flow: Evidence from Google” a little over two months ago. I’ve been meaning to post about it for some time now, and wanted to put some of my thoughts on it out in the blogosphere to hopefully provoke further discussion. The paper deals with two separate but linked issues: how prediction markets worked at Google, and how prediction markets demonstrated how information flowed within Google.

I’m not going to do an in-depth discussion or criticism of the paper here, but instead try to draw out some key points for your corporate prediction market project.

Takeaways for other prediction market projects:

• Google’s employees show an optimism bias:
  To me, this is not surprising. Google is on an incredible growth streak, and positions there are highly competitive. It could certainly still be seen as an entreprenurial company. That new recruits are very positive about the company and that this shows in their trading is normal.

  This will likely NOT be the case in your average corporate prediction market. I believe that the market will reflect the nature of the organisation itself. If you’re running a market in an poorly performing company or industry, I think the market could show a pessimism bias instead. In your average company, there will likely be a bit of an optimism bias, but likely not as dramatic as what Google has seen. (Let’s face it, people are generally pretty positive about their own work.)

• People learned to overcome optimism bias:
  Despite a fairly strong optimism bias early on in the Google prediction markets, one of the other elements that came out through the paper was that people quickly learned to overcome the optimism bias. Feedback, particularly through poor performance, is a powerful correction mechanism.

  I’ve seen this in other markets I’ve run, as well. At the beginning traders are all over the place, with a fair bit of volatility and inaccuracy. But pretty quickly this settles out. I believe it’s a combination of “dumb” traders getting frustrated and stopping, and all traders becoming more sensitive to the risks they’re taking.

• Active participants were NOT diverse:
  The Google prediction market traders were more likely to be based in either the Mountain View or New York offices, and have a quantitative background and/or an interest in investing or gambling. They were most certainly NOT representative of Google as a whole.

  This is not a problem, with Google or with your company’s prediction market! What matters most is that the traders don’t have a single source of information and analysis on which to trade. As long as each trader (or even cliques of traders) are doing independent analysis on the data, preferably on diverse data, the demographics of the traders don’t really matter.

• Significant number of “fun” markets:
  As many as 30% of the Google prediction markets were “fun” markets, on such things as films, gas prices, etc. I highly recommend this for all corporate prediction markets. In corporate markets companies simply need people to trade. If they can create a steady source of attention and interest from “fun” markets, so be it. Fun markets keep people talking about the tool and both trading interest and trading skills will spill over to the more serious markets. Even the authors mentioned in their paper:

  “[fun markets] might help create, rather than crowd out, liquidity for [serious markets].”

• Arbitrage opportunities were plentiful, but taken advantage of:
  Google used a CDA model for their markets. This created arbitrage opportunities, when the sum of the bid prices was more than $1, and when the sum of the ask prices was less than $1. The authors found 1,747 instances of the former, and 495 instances of the latter. As they noted, this demonstrates an aversion to short selling contracts.

  What was interesting to me was how the market reacted to this. The median duration of any of these arbitrage opportunities was just two minutes, demonstrating that they were correctable. Even better is this:

  One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle.

  All it takes is one person and a little programming knowledge to make the market much more efficient and liquid! I would never discourage this kind of behaviour.

The second half of the paper examined the transmission of information within Google based on the authors’ analysis of the traders and their behaviours. While there is some really interesting analysis there, it has more to do with organisational behaviour than being directly applicable to prediction markets, so I’m not going to discuss it here. But if you’re interested, I highly recommend going to read the paper!

Summary

First of all, congratulations to Bo, Justin and Eric for a very well written and interesting paper. As discussed above, they demonstrate some great results and point to principles that can and should apply to corporate prediction markets as a whole. It’s very important for the prediction market industry to have such a highly esteemed company as Google talk about their experiments as widely as they have, and these results do bode well for other companies looking to do similar projects.

As I mentioned in my last post, HubDub went into a public beta last week by officially announcing at the DEMO 2008 conference. All I have to say now is a big “Well Done” to Nigel and the HubDub team.

When I first thought about it, I wasn’t sure how HubDub would be received by the “tech elite” that attend the DEMO conference. Would it be seen as a bit of a toy, or would attendees understand how news and forecasting could work together. (Perhaps more importantly, could they see how HubDub could make money, which Alex Kirtland talks about in his post here.)

Well, it looks like they certainly made an impression. One blog put them #3 (out of 72 demo companies there) for “the best venture capital investment candidates” there. This is certainly a step in the right direction for HubDub.

Does this mean anything for the prediction market industry? Perhaps. It means that prediction markets have yet another great broad-based, public and high-profile example. (The Hollywood Stock Exchange being the most prominent up to now, though the Foresight Exchange, NewsFutures, Media Predict and Inkling could also be included.) HubDub has prediction markets based on news events, where HSX is based on media, films, and stars. While for media buffs HubDub will never surpass the HSX, I think HubDub will be popular with a broader range of potential users.

HubDub should develop into a great example of a prediction market, and make people more and more familiar with how forecasting and current news fit together. I think that any attention like this will be positive for the industry in general. How HubDub currently allows any user to create any question does seem to be a bit risky to me, and we’ll see how that works out as the community of users grows. Overall, it’s great to see another strong example of a prediction market

Embedded here is Nigel’s actual demo. I was pleased to see that the market he used to demo that day was one I created!

It’s now official… HubDub is out of private testing and into public beta as they launch at DEMO 2008! (They were recently covered in both TechCrunch UK and TechCrunch.)

I’m really quite happy for Nigel and his team up in Edinburgh. Taking advantage of Scotland’s cold, dark winter, they created a truly new way of approaching prediction markets. By integrating a news aggregator with a prediction market, users will have a unique opportunity to see how news and events interact.

Part of the reason I’m intrigued by HubDub is because I’m a bit of a news junkie. So much of what I read/scan (or just skip over) day-to-day is regurgitated information used just to fill the pages of the newspaper. It can be difficult to tell what information is actually relevant to an event amidst the daily filler. This is where HubDub gets interesting, and provides a unique solution.

Where other public prediction markets (such as NewsFutures, Inkling Markets, HSX, or the Foresight Exchange) offer lots of contracts to trade, they don’t necessarily link back to any information on those contracts. HubDub not only links to news stories, but through algorithms and voting can show which news stories actually matter on a given contract. Though it’s not fully working right now, eventually we’ll be able to see a Google-Finance-like interface, where key news stories can be traced to changes in a contracts price.

This also makes HubDub a better news aggregator. By tying the news stories back to how relevant they are to questions people have created, it will allow more interesting stories make it to the top of the list.

All of this doesn’t mean that the other public prediction markets mentioned above aren’t good or relevant… they are! But by integrating markets (which feed on news) to the news itself, they will make it much easier and more desirable for users to participate and continue participating.

HubDub is just launching, so there’s quite a bit more to come. (Some features/options have a bit of refinement left to do.) I absolutely encourage you to sign up for an account and play around. And while I’m doing well on the leaderboards right now, I don’t expect that to last for long, though I will be doing my damnedest to keep my spot!