Mercury’s Blog

If you're new here, you may want to get Mercury's Blog by Email or subscribe to my RSS feed. Thanks for visiting!

This will be my last post on the CFTC and prediction markets.

Thank you to those who e-mailed me with comments and left comments on my posts. As I hoped, it helped clarify my own thinking, particularly on how public event markets can and should work. While I still believe that the CFTC should make public, real-money event markets work, the reality that this will need strong regulation from the CFTC is reflected in my draft response. My final draft is attached below, and will be submitted to the CFTC later this week.

What do I expect to happen?

I expect that the CFTC will choose to provide a “safe harbor” for academic, corporate and low-stakes prediction markets. There is very strong (I would venture unanimous) support for this, and it is sorely needed.

I sincerely doubt that the CFTC will take the steps necessary to allow public real-money prediction markets at this point. It is simply a very significant change, and I doubt that it would happen before a major election. That said, I am quite hopeful that the CFTC will sketch out a roadmap as to how this can or should happen in the coming months and years. Knowing that governmental bodies are quite risk-averse, there is really no way that we will go from the current state to a position where real-money prediction markets are completely legal in one stroke of the CFTC’s regulatory pen. But I believe that the current environment is positive for change and that if enough interest is shown in this comment period the right steps will be taken to make “event markets” legal in the United States.

This really will have little to no effect on most corporations and prediction market projects. Companies will be able to run them just as easily in the coming years as they can now. However, once real-money prediction markets are legal, there should be much more familiarity and acceptance of the concept, and thus greater acceptance at the corporate level. When that happens I will be very happy!

CFTC-DraftResponse.pdf

After reading my last post, specifically this section:

The second half of the paper examined the transmission of information within Google based on the authors’ analysis of the traders and their behaviours. While there is some really interesting analysis there, it has more to do with organisational behaviour than being directly applicable to prediction markets, so I’m not going to discuss it here. But if you’re interested, I highly recommend going to read the paper!

Bo wrote the following on his blog:

This is not the first reaction along these lines. I am perplexed by the response. I can understand why other companies may not want to replicate our analysis of information flows. Perhaps it wouldn’t be worth the effort. Perhaps they would get identical results. And perhaps the company wouldn’t have the all the necessary data.

However, I expected that people could easily see value in the analysis of granular trade-by-trade data — especially if that data is joined with data about traders and outside events happening at the moment of the trades.

To which I respond here:

Perhaps this was a bit of lazy writing at the end of what felt like a longish post. But what it really came down to was my focus on the very tangible, common questions that many people I talk to have when they’re first starting with their prediction market projects. The first issue for many is just getting a market working, with sufficient participation and liquidity, and hopefully accurate predictions. These were the main points and examples from the paper that I wanted to address in my post.

I believe Bo’s concern is that with just a “first-order” discussion on the paper, we leave out the “second-order” potential. Once a market is up and running, there is a LOT of data available to the company. If you know enough about your employees (and Bo’s subsequent post suggests that any decent-sized company does) this is information you likely already have. Specifics had to be left out of the Google paper for sensitivity reasons, but the paper demonstrated the kinds of things you can discover about your organisation through a prediction market.

Addressing these questions takes time and careful attention. There is certainly a privacy issue here, but that’s more an issue of managing perceptions. (You want to make sure people trade their true beliefs, and not some altered reality because they think the corporate Big Brother is watching them.) Both the data collection and analysis would certainly take some time.

However, there is some great data and metrics available to companies that are ready to take advantage of these “second-order” advantages of prediction markets. Again, to quote Bo:

The data contains real-time metrics on the distribution of knowledge and attitudes within a firm at a highly granular level. You can get metrics on for specific of the firm, for specific classes of employees and for specific topics. You can do this for either customers or employees, and have the metrics for any moment in time. The quality of these metrics will be extremely strong, because participants have been incentivized to reveal their true expectations.

So I don’t disagree with Bo at all, I just had a slightly different focus in my discussion. Perhaps this could provoke a little further discussion on the long-term potential of prediction markets in corporations.

[UPDATE]: My sincere apologies for the spam in the original post. I have no idea what happened, but must be related to the fact I didn’t use my regular posting program to put this post up. It should be fixed now.

Bo Cowgill, Justin Wolfers and Eric Zitzewitz released their paper “Using Prediction Markets to Track Information Flow: Evidence from Google” a little over two months ago. I’ve been meaning to post about it for some time now, and wanted to put some of my thoughts on it out in the blogosphere to hopefully provoke further discussion. The paper deals with two separate but linked issues: how prediction markets worked at Google, and how prediction markets demonstrated how information flowed within Google.

I’m not going to do an in-depth discussion or criticism of the paper here, but instead try to draw out some key points for your corporate prediction market project.

Takeaways for other prediction market projects:

• Google’s employees show an optimism bias:
  To me, this is not surprising. Google is on an incredible growth streak, and positions there are highly competitive. It could certainly still be seen as an entreprenurial company. That new recruits are very positive about the company and that this shows in their trading is normal.

  This will likely NOT be the case in your average corporate prediction market. I believe that the market will reflect the nature of the organisation itself. If you’re running a market in an poorly performing company or industry, I think the market could show a pessimism bias instead. In your average company, there will likely be a bit of an optimism bias, but likely not as dramatic as what Google has seen. (Let’s face it, people are generally pretty positive about their own work.)

• People learned to overcome optimism bias:
  Despite a fairly strong optimism bias early on in the Google prediction markets, one of the other elements that came out through the paper was that people quickly learned to overcome the optimism bias. Feedback, particularly through poor performance, is a powerful correction mechanism.

  I’ve seen this in other markets I’ve run, as well. At the beginning traders are all over the place, with a fair bit of volatility and inaccuracy. But pretty quickly this settles out. I believe it’s a combination of “dumb” traders getting frustrated and stopping, and all traders becoming more sensitive to the risks they’re taking.

• Active participants were NOT diverse:
  The Google prediction market traders were more likely to be based in either the Mountain View or New York offices, and have a quantitative background and/or an interest in investing or gambling. They were most certainly NOT representative of Google as a whole.

  This is not a problem, with Google or with your company’s prediction market! What matters most is that the traders don’t have a single source of information and analysis on which to trade. As long as each trader (or even cliques of traders) are doing independent analysis on the data, preferably on diverse data, the demographics of the traders don’t really matter.

• Significant number of “fun” markets:
  As many as 30% of the Google prediction markets were “fun” markets, on such things as films, gas prices, etc. I highly recommend this for all corporate prediction markets. In corporate markets companies simply need people to trade. If they can create a steady source of attention and interest from “fun” markets, so be it. Fun markets keep people talking about the tool and both trading interest and trading skills will spill over to the more serious markets. Even the authors mentioned in their paper:

  “[fun markets] might help create, rather than crowd out, liquidity for [serious markets].”

• Arbitrage opportunities were plentiful, but taken advantage of:
  Google used a CDA model for their markets. This created arbitrage opportunities, when the sum of the bid prices was more than $1, and when the sum of the ask prices was less than $1. The authors found 1,747 instances of the former, and 495 instances of the latter. As they noted, this demonstrates an aversion to short selling contracts.

  What was interesting to me was how the market reacted to this. The median duration of any of these arbitrage opportunities was just two minutes, demonstrating that they were correctable. Even better is this:

  One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle.

  All it takes is one person and a little programming knowledge to make the market much more efficient and liquid! I would never discourage this kind of behaviour.

The second half of the paper examined the transmission of information within Google based on the authors’ analysis of the traders and their behaviours. While there is some really interesting analysis there, it has more to do with organisational behaviour than being directly applicable to prediction markets, so I’m not going to discuss it here. But if you’re interested, I highly recommend going to read the paper!

Summary

First of all, congratulations to Bo, Justin and Eric for a very well written and interesting paper. As discussed above, they demonstrate some great results and point to principles that can and should apply to corporate prediction markets as a whole. It’s very important for the prediction market industry to have such a highly esteemed company as Google talk about their experiments as widely as they have, and these results do bode well for other companies looking to do similar projects.

As I mentioned in my last post, HubDub went into a public beta last week by officially announcing at the DEMO 2008 conference. All I have to say now is a big “Well Done” to Nigel and the HubDub team.

When I first thought about it, I wasn’t sure how HubDub would be received by the “tech elite” that attend the DEMO conference. Would it be seen as a bit of a toy, or would attendees understand how news and forecasting could work together. (Perhaps more importantly, could they see how HubDub could make money, which Alex Kirtland talks about in his post here.)

Well, it looks like they certainly made an impression. One blog put them #3 (out of 72 demo companies there) for “the best venture capital investment candidates” there. This is certainly a step in the right direction for HubDub.

Does this mean anything for the prediction market industry? Perhaps. It means that prediction markets have yet another great broad-based, public and high-profile example. (The Hollywood Stock Exchange being the most prominent up to now, though the Foresight Exchange, NewsFutures, Media Predict and Inkling could also be included.) HubDub has prediction markets based on news events, where HSX is based on media, films, and stars. While for media buffs HubDub will never surpass the HSX, I think HubDub will be popular with a broader range of potential users.

HubDub should develop into a great example of a prediction market, and make people more and more familiar with how forecasting and current news fit together. I think that any attention like this will be positive for the industry in general. How HubDub currently allows any user to create any question does seem to be a bit risky to me, and we’ll see how that works out as the community of users grows. Overall, it’s great to see another strong example of a prediction market

Embedded here is Nigel’s actual demo. I was pleased to see that the market he used to demo that day was one I created!

It’s now official… HubDub is out of private testing and into public beta as they launch at DEMO 2008! (They were recently covered in both TechCrunch UK and TechCrunch.)

I’m really quite happy for Nigel and his team up in Edinburgh. Taking advantage of Scotland’s cold, dark winter, they created a truly new way of approaching prediction markets. By integrating a news aggregator with a prediction market, users will have a unique opportunity to see how news and events interact.

Part of the reason I’m intrigued by HubDub is because I’m a bit of a news junkie. So much of what I read/scan (or just skip over) day-to-day is regurgitated information used just to fill the pages of the newspaper. It can be difficult to tell what information is actually relevant to an event amidst the daily filler. This is where HubDub gets interesting, and provides a unique solution.

Where other public prediction markets (such as NewsFutures, Inkling Markets, HSX, or the Foresight Exchange) offer lots of contracts to trade, they don’t necessarily link back to any information on those contracts. HubDub not only links to news stories, but through algorithms and voting can show which news stories actually matter on a given contract. Though it’s not fully working right now, eventually we’ll be able to see a Google-Finance-like interface, where key news stories can be traced to changes in a contracts price.

This also makes HubDub a better news aggregator. By tying the news stories back to how relevant they are to questions people have created, it will allow more interesting stories make it to the top of the list.

All of this doesn’t mean that the other public prediction markets mentioned above aren’t good or relevant… they are! But by integrating markets (which feed on news) to the news itself, they will make it much easier and more desirable for users to participate and continue participating.

HubDub is just launching, so there’s quite a bit more to come. (Some features/options have a bit of refinement left to do.) I absolutely encourage you to sign up for an account and play around. And while I’m doing well on the leaderboards right now, I don’t expect that to last for long, though I will be doing my damnedest to keep my spot!