Mercury’s Blog

If you're new here, you may want to get Mercury's Blog by Email or subscribe to my RSS feed. Thanks for visiting!

After reading my last post, specifically this section:

The second half of the paper examined the transmission of information within Google based on the authors’ analysis of the traders and their behaviours. While there is some really interesting analysis there, it has more to do with organisational behaviour than being directly applicable to prediction markets, so I’m not going to discuss it here. But if you’re interested, I highly recommend going to read the paper!

Bo wrote the following on his blog:

This is not the first reaction along these lines. I am perplexed by the response. I can understand why other companies may not want to replicate our analysis of information flows. Perhaps it wouldn’t be worth the effort. Perhaps they would get identical results. And perhaps the company wouldn’t have the all the necessary data.

However, I expected that people could easily see value in the analysis of granular trade-by-trade data — especially if that data is joined with data about traders and outside events happening at the moment of the trades.

To which I respond here:

Perhaps this was a bit of lazy writing at the end of what felt like a longish post. But what it really came down to was my focus on the very tangible, common questions that many people I talk to have when they’re first starting with their prediction market projects. The first issue for many is just getting a market working, with sufficient participation and liquidity, and hopefully accurate predictions. These were the main points and examples from the paper that I wanted to address in my post.

I believe Bo’s concern is that with just a “first-order” discussion on the paper, we leave out the “second-order” potential. Once a market is up and running, there is a LOT of data available to the company. If you know enough about your employees (and Bo’s subsequent post suggests that any decent-sized company does) this is information you likely already have. Specifics had to be left out of the Google paper for sensitivity reasons, but the paper demonstrated the kinds of things you can discover about your organisation through a prediction market.

Addressing these questions takes time and careful attention. There is certainly a privacy issue here, but that’s more an issue of managing perceptions. (You want to make sure people trade their true beliefs, and not some altered reality because they think the corporate Big Brother is watching them.) Both the data collection and analysis would certainly take some time.

However, there is some great data and metrics available to companies that are ready to take advantage of these “second-order” advantages of prediction markets. Again, to quote Bo:

The data contains real-time metrics on the distribution of knowledge and attitudes within a firm at a highly granular level. You can get metrics on for specific of the firm, for specific classes of employees and for specific topics. You can do this for either customers or employees, and have the metrics for any moment in time. The quality of these metrics will be extremely strong, because participants have been incentivized to reveal their true expectations.

So I don’t disagree with Bo at all, I just had a slightly different focus in my discussion. Perhaps this could provoke a little further discussion on the long-term potential of prediction markets in corporations.

[UPDATE]: My sincere apologies for the spam in the original post. I have no idea what happened, but must be related to the fact I didn’t use my regular posting program to put this post up. It should be fixed now.

Bo Cowgill, Justin Wolfers and Eric Zitzewitz released their paper “Using Prediction Markets to Track Information Flow: Evidence from Google” a little over two months ago. I’ve been meaning to post about it for some time now, and wanted to put some of my thoughts on it out in the blogosphere to hopefully provoke further discussion. The paper deals with two separate but linked issues: how prediction markets worked at Google, and how prediction markets demonstrated how information flowed within Google.

I’m not going to do an in-depth discussion or criticism of the paper here, but instead try to draw out some key points for your corporate prediction market project.

Takeaways for other prediction market projects:

• Google’s employees show an optimism bias:
  To me, this is not surprising. Google is on an incredible growth streak, and positions there are highly competitive. It could certainly still be seen as an entreprenurial company. That new recruits are very positive about the company and that this shows in their trading is normal.

  This will likely NOT be the case in your average corporate prediction market. I believe that the market will reflect the nature of the organisation itself. If you’re running a market in an poorly performing company or industry, I think the market could show a pessimism bias instead. In your average company, there will likely be a bit of an optimism bias, but likely not as dramatic as what Google has seen. (Let’s face it, people are generally pretty positive about their own work.)

• People learned to overcome optimism bias:
  Despite a fairly strong optimism bias early on in the Google prediction markets, one of the other elements that came out through the paper was that people quickly learned to overcome the optimism bias. Feedback, particularly through poor performance, is a powerful correction mechanism.

  I’ve seen this in other markets I’ve run, as well. At the beginning traders are all over the place, with a fair bit of volatility and inaccuracy. But pretty quickly this settles out. I believe it’s a combination of “dumb” traders getting frustrated and stopping, and all traders becoming more sensitive to the risks they’re taking.

• Active participants were NOT diverse:
  The Google prediction market traders were more likely to be based in either the Mountain View or New York offices, and have a quantitative background and/or an interest in investing or gambling. They were most certainly NOT representative of Google as a whole.

  This is not a problem, with Google or with your company’s prediction market! What matters most is that the traders don’t have a single source of information and analysis on which to trade. As long as each trader (or even cliques of traders) are doing independent analysis on the data, preferably on diverse data, the demographics of the traders don’t really matter.

• Significant number of “fun” markets:
  As many as 30% of the Google prediction markets were “fun” markets, on such things as films, gas prices, etc. I highly recommend this for all corporate prediction markets. In corporate markets companies simply need people to trade. If they can create a steady source of attention and interest from “fun” markets, so be it. Fun markets keep people talking about the tool and both trading interest and trading skills will spill over to the more serious markets. Even the authors mentioned in their paper:

  “[fun markets] might help create, rather than crowd out, liquidity for [serious markets].”

• Arbitrage opportunities were plentiful, but taken advantage of:
  Google used a CDA model for their markets. This created arbitrage opportunities, when the sum of the bid prices was more than $1, and when the sum of the ask prices was less than $1. The authors found 1,747 instances of the former, and 495 instances of the latter. As they noted, this demonstrates an aversion to short selling contracts.

  What was interesting to me was how the market reacted to this. The median duration of any of these arbitrage opportunities was just two minutes, demonstrating that they were correctable. Even better is this:

  One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle.

  All it takes is one person and a little programming knowledge to make the market much more efficient and liquid! I would never discourage this kind of behaviour.

The second half of the paper examined the transmission of information within Google based on the authors’ analysis of the traders and their behaviours. While there is some really interesting analysis there, it has more to do with organisational behaviour than being directly applicable to prediction markets, so I’m not going to discuss it here. But if you’re interested, I highly recommend going to read the paper!

Summary

First of all, congratulations to Bo, Justin and Eric for a very well written and interesting paper. As discussed above, they demonstrate some great results and point to principles that can and should apply to corporate prediction markets as a whole. It’s very important for the prediction market industry to have such a highly esteemed company as Google talk about their experiments as widely as they have, and these results do bode well for other companies looking to do similar projects.