Mercury’s Blog

If you're new here, you may want to get Mercury's Blog by Email or subscribe to my RSS feed. Thanks for visiting!

Bo Cowgill, Justin Wolfers and Eric Zitzewitz released their paper “Using Prediction Markets to Track Information Flow: Evidence from Google” a little over two months ago. I’ve been meaning to post about it for some time now, and wanted to put some of my thoughts on it out in the blogosphere to hopefully provoke further discussion. The paper deals with two separate but linked issues: how prediction markets worked at Google, and how prediction markets demonstrated how information flowed within Google.

I’m not going to do an in-depth discussion or criticism of the paper here, but instead try to draw out some key points for your corporate prediction market project.

Takeaways for other prediction market projects:

• Google’s employees show an optimism bias:
  To me, this is not surprising. Google is on an incredible growth streak, and positions there are highly competitive. It could certainly still be seen as an entreprenurial company. That new recruits are very positive about the company and that this shows in their trading is normal.

  This will likely NOT be the case in your average corporate prediction market. I believe that the market will reflect the nature of the organisation itself. If you’re running a market in an poorly performing company or industry, I think the market could show a pessimism bias instead. In your average company, there will likely be a bit of an optimism bias, but likely not as dramatic as what Google has seen. (Let’s face it, people are generally pretty positive about their own work.)

• People learned to overcome optimism bias:
  Despite a fairly strong optimism bias early on in the Google prediction markets, one of the other elements that came out through the paper was that people quickly learned to overcome the optimism bias. Feedback, particularly through poor performance, is a powerful correction mechanism.

  I’ve seen this in other markets I’ve run, as well. At the beginning traders are all over the place, with a fair bit of volatility and inaccuracy. But pretty quickly this settles out. I believe it’s a combination of “dumb” traders getting frustrated and stopping, and all traders becoming more sensitive to the risks they’re taking.

• Active participants were NOT diverse:
  The Google prediction market traders were more likely to be based in either the Mountain View or New York offices, and have a quantitative background and/or an interest in investing or gambling. They were most certainly NOT representative of Google as a whole.

  This is not a problem, with Google or with your company’s prediction market! What matters most is that the traders don’t have a single source of information and analysis on which to trade. As long as each trader (or even cliques of traders) are doing independent analysis on the data, preferably on diverse data, the demographics of the traders don’t really matter.

• Significant number of “fun” markets:
  As many as 30% of the Google prediction markets were “fun” markets, on such things as films, gas prices, etc. I highly recommend this for all corporate prediction markets. In corporate markets companies simply need people to trade. If they can create a steady source of attention and interest from “fun” markets, so be it. Fun markets keep people talking about the tool and both trading interest and trading skills will spill over to the more serious markets. Even the authors mentioned in their paper:

  “[fun markets] might help create, rather than crowd out, liquidity for [serious markets].”

• Arbitrage opportunities were plentiful, but taken advantage of:
  Google used a CDA model for their markets. This created arbitrage opportunities, when the sum of the bid prices was more than $1, and when the sum of the ask prices was less than $1. The authors found 1,747 instances of the former, and 495 instances of the latter. As they noted, this demonstrates an aversion to short selling contracts.

  What was interesting to me was how the market reacted to this. The median duration of any of these arbitrage opportunities was just two minutes, demonstrating that they were correctable. Even better is this:

  One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle.

  All it takes is one person and a little programming knowledge to make the market much more efficient and liquid! I would never discourage this kind of behaviour.

The second half of the paper examined the transmission of information within Google based on the authors’ analysis of the traders and their behaviours. While there is some really interesting analysis there, it has more to do with organisational behaviour than being directly applicable to prediction markets, so I’m not going to discuss it here. But if you’re interested, I highly recommend going to read the paper!

Summary

First of all, congratulations to Bo, Justin and Eric for a very well written and interesting paper. As discussed above, they demonstrate some great results and point to principles that can and should apply to corporate prediction markets as a whole. It’s very important for the prediction market industry to have such a highly esteemed company as Google talk about their experiments as widely as they have, and these results do bode well for other companies looking to do similar projects.