Mercury's Blog

The US election is just over a week away, and with that there are a few different topics I’d like to touch on. With the explosion in new prediction markets since the last presidential election, we should see some interesting (but hopefully consistent) results.

• First, a great post from Koleman Strumpf on Midas Oracle points out that half of all trades on the Betfair exchange in 2004 occurred on Election Day! While I personally think that was quite likely due to the early exit poll news for Kerry and the subsequent swing back to Bush, it proves that there are still quite a few people that may be waiting until the very last day to trade.
• Jason Ruspini just started a new thread on the Prediction Markets e-mail group regarding some divergences he’s seen between prices on InTrade and fivethirtyeight.com.

  While I think some of the things he’s observed is due to the way Nate Silver presents data on his site, Jason brings up a very good point. A thorough analysis of movement in the InTrade prediction markets should be compared to the daily calculated win percentage from fivethirtyeight (where all data comes strictly from polls). I think it could be very revealing, and give the public quite a bit more data on the accuracy of polling, aggregation of polls, and prediction markets.

• A long time ago I started four different markets on Inkling Markets that will hopefully predict control over each house of Congress, and the number of seats each party will have after the election. Data is shown here:
  
  
  (as I write this, the Democratic percentage is 53.8%, which corresponds to 234 seats in the House of Representatives.)
  
  
• I may need to update my post on prediction market software soon. Xpree (founded by Mat Fogarty, and recently joined by Leslie Fine, a well known prediction market researcher from HP) may be changing their name. The top three picks according to their contest on NameThis were:
  1. Metricast
  2. UREprojection
  3. Keymet

  Personally, I don’t like #2 or #3, but Metricast sounds interesting. It also sounds like a much better fit to what they do than “Xpree.”

  Good luck to them, if they choose to go down this route.

• Do you speak Danish? Nosco is hiring!
  For English speakers, so is InTrade and Xpree.

In my previous post on categorizing prediction markets, one of the key differences is whether a market is public or private. (The “P” in the ICROP criteria.) There is fundamentally a very different value to the operators of a public market compared to a private market. My train of thought is below, starting with some notes on recent prediction market news.

Recent news from Hubdub

Hubdub recently announced a partnership with Reuters, which I think is a great step for public prediction markets. They previously had announced a partnership with the Huffington Post blog. When that was announced two weeks ago I was a little torn of what to think. It was great they were able to work with a major internet brand, but the implementation was pretty weak. You could find Hubdub’s markets on Huffington Post tag pages, but even knowing it was there I really had to search to find them on the page. It seemed to be something that was being treated as just a minor experiment rather than something serious.

The Reuters announcement is much more important. It looks like it’s kicking off a wider partnership program at Hubdub, which is quite exciting. Reuters has a dedicated section on the Hubdub site which will apparently be regularly updated by Reuters staff. (They’ve only created 4 questions so far.) The only scheme that I think would be better than this is if Reuters had put a dedicated Hubdub section on their (Reuters) site, and that could certainly happen down the road.

By opening up Hubdub to a wider partnership program, they will help other companies and bloggers build a reputation by allowing those people and organizations to generate interesting questions and predictions. This is great news, and should spur even more growth for both the participating bloggers and Hubdub itself.

But this got me thinking about public versus private prediction markets…

Value in Public versus Private markets

The value to operators of public markets is significantly different than private markets. I think this is why we are seeing significantly different types of growth in the two types of markets.

The value to operators of public markets comes from generating an active, thriving community of users. These users may be targets for advertising, subjects for demographic research (HSX), see examples of technology (Inkling, NewsFutures), or another unique model to be determined. This is where Hubdub looks to be pioneering.

Nigel Eccles, the CEO of Hubdub, has mentioned on many occasions some interesting statistics about an average subscriber’s interaction with a newspaper’s online presence. For most newspapers it is extremely limited; a person will check a story or two and leave. There is little real engagement, and so page views and advertising rates aren’t as high as they could be. (Alex Kirtland talked about this here.) However, prediction markets have shown themselves to be hugely engaging, and can also be made highly local and relevant to a small audience. This looks to be the way they’re going with their partnership program. Each partnership will add and build another sub-community of users, which adds value to both the partner and Hubdub.

The value is significantly different for private markets. When I think of private markets, I think of a corporate prediction market where the company is looking to get useful and accurate business intelligence from their employees. The business intelligence is the value many companies claim they are trying to capture.

The difficulty in private markets is that there is no obvious, traceable value chain. In that I mean that most companies cannot say that because the market told management that there was X% chance of event A occurring, the company changed strategy and saved $Y. Many companies are in reality treating them as non-actionable market intelligence, where they examine only after the fact how accurate the predictions were.

Even if a company did trust their employees enough to take action directly based off of what their internal markets were telling them, it may still be difficult to calculate the value of that intelligence. Particularly since management wants to be (or at least appear) smarter than their employees, it is quite easy to claim after the fact that they would have taken the same actions based off of other intelligence.

Fundamentally, it takes a company that both trusts their employees enough to take action on the market indicators and management that is honest enough in what would have happened without that intelligence in order to calculate the value of those prediction markets. For example, Mat Fogarty has talked about how he used prediction markets at EA to quantify game quality scores, which is certainly useful. But where that can directly turn into additional profit is if EA (or any similar company) took prediction market intelligence to adjust how they filled their distribution channels. They could save money by not creating unnecessary copies of bad games, and could make more money by ensuring they had enough copies of hit games ready when they went on sale. As far as I am aware, few companies have taken that final step to action based on market results.

Prediction markets certainly add value even where the elements I mentioned above aren’t present, it’s just that the direct value cannot be easily calculated. And until an executive can directly point at how the cost of a prediction market is more than made up by the direct value added, the growth of prediction markets will be limited.

That said…

Prediction markets are clearly a growth industry, even in private markets. Inkling Markets, NewsFutures, and Xpree have all recently hired great new people into their businesses, so the market for private markets is clearly growing. But until the value calculation above can be directly made, that growth rate just isn’t as high as I wish it would be.

(Gratuitous photo is from a recent holiday, specifically a section of the Great Wall of China outside of Beijing.)

I’m fascinated when real-money markets can be directly compared to play-money markets, particularly when I can potentially make some money.

For those of you that haven’t read my research paper, I created a series of play-money prediction markets on rowing events in the summer of 2006. The results were as what you might expect; quite accurate. When there were just 16 or more traders involved in a market, the results could be relied upon for good forecasts. I’ve turned that project into a longer-running project, which has also taken place last summer and this summer. (I’m collecting data to analyze how the number of traders required has changed since Inkling adopted Robin Hanson’s MSR, which wasn’t in place for the original research.) It’s proven quite popular amongst amateur rowers in the UK.

[I'd like to thank Inkling for providing the platform; I plan to analyze and publish the results of the last two summers' research this fall once the Olympics markets are closed out.]

A few weeks ago I created a prediction market for each of the 14 Olympic rowing events. A few days ago I was checking Betfair and realised that they had also created markets for all 14 Olympic rowing events. (Both to determine the winners as well as the podium places for each event.) When I compared the two marketplaces there were some potentially profitable discrepancies.

There seem to be two types of market-makers in the Betfair markets. One simply entered lay bets for each entrant at very poor odds (ie, 1.01 decimal) just so there was something to trade. That’s not very useful. But another type of market-maker entered more realistic odds, but odds that were dramatically skewed toward long-shots. For example, in the Men’s Double Sculls event, the play-money prediction market forecasted a win for New Zealand with about an 80% probability. However, I was able to buy shares on Betfair at 2.56, or about 39% probability! While the over-round on the market was quite large, it was because the odds on the long-shots were unreasonable. Odds on the favourites in these markets were very good, and this was the case on as many as half of the events.

So if my play-money markets are accurate (as I expect them to be), I should be able to make a little money on the Olympics, courtesy of the initial market-makers on Betfair. Some people may argue that the play-money predictions won’t be as accurate because they don’t involve real money, but looking at the current Betfair market there’s so little liquidity to have quality forecasts. Unfortunately the lack of book depth means that the mis-pricing won’t last long and won’t be hugely profitable in absolute terms, but that should be interesting to watch.

I think this is an excellent example of where a play-money market can profitably inform real-money market trading.

Questions, comments? Please enter your comments below. (No registration or sign-up required.)

(Reminder: You can subscribe to receive new posts here by e-mail, just click here.)

This will be my last post on the CFTC and prediction markets.

Thank you to those who e-mailed me with comments and left comments on my posts. As I hoped, it helped clarify my own thinking, particularly on how public event markets can and should work. While I still believe that the CFTC should make public, real-money event markets work, the reality that this will need strong regulation from the CFTC is reflected in my draft response. My final draft is attached below, and will be submitted to the CFTC later this week.

What do I expect to happen?

I expect that the CFTC will choose to provide a “safe harbor” for academic, corporate and low-stakes prediction markets. There is very strong (I would venture unanimous) support for this, and it is sorely needed.

I sincerely doubt that the CFTC will take the steps necessary to allow public real-money prediction markets at this point. It is simply a very significant change, and I doubt that it would happen before a major election. That said, I am quite hopeful that the CFTC will sketch out a roadmap as to how this can or should happen in the coming months and years. Knowing that governmental bodies are quite risk-averse, there is really no way that we will go from the current state to a position where real-money prediction markets are completely legal in one stroke of the CFTC’s regulatory pen. But I believe that the current environment is positive for change and that if enough interest is shown in this comment period the right steps will be taken to make “event markets” legal in the United States.

This really will have little to no effect on most corporations and prediction market projects. Companies will be able to run them just as easily in the coming years as they can now. However, once real-money prediction markets are legal, there should be much more familiarity and acceptance of the concept, and thus greater acceptance at the corporate level. When that happens I will be very happy!

CFTC-DraftResponse.pdf

After reading my last post, specifically this section:

The second half of the paper examined the transmission of information within Google based on the authors’ analysis of the traders and their behaviours. While there is some really interesting analysis there, it has more to do with organisational behaviour than being directly applicable to prediction markets, so I’m not going to discuss it here. But if you’re interested, I highly recommend going to read the paper!

Bo wrote the following on his blog:

This is not the first reaction along these lines. I am perplexed by the response. I can understand why other companies may not want to replicate our analysis of information flows. Perhaps it wouldn’t be worth the effort. Perhaps they would get identical results. And perhaps the company wouldn’t have the all the necessary data.

However, I expected that people could easily see value in the analysis of granular trade-by-trade data — especially if that data is joined with data about traders and outside events happening at the moment of the trades.

To which I respond here:

Perhaps this was a bit of lazy writing at the end of what felt like a longish post. But what it really came down to was my focus on the very tangible, common questions that many people I talk to have when they’re first starting with their prediction market projects. The first issue for many is just getting a market working, with sufficient participation and liquidity, and hopefully accurate predictions. These were the main points and examples from the paper that I wanted to address in my post.

I believe Bo’s concern is that with just a “first-order” discussion on the paper, we leave out the “second-order” potential. Once a market is up and running, there is a LOT of data available to the company. If you know enough about your employees (and Bo’s subsequent post suggests that any decent-sized company does) this is information you likely already have. Specifics had to be left out of the Google paper for sensitivity reasons, but the paper demonstrated the kinds of things you can discover about your organisation through a prediction market.

Addressing these questions takes time and careful attention. There is certainly a privacy issue here, but that’s more an issue of managing perceptions. (You want to make sure people trade their true beliefs, and not some altered reality because they think the corporate Big Brother is watching them.) Both the data collection and analysis would certainly take some time.

However, there is some great data and metrics available to companies that are ready to take advantage of these “second-order” advantages of prediction markets. Again, to quote Bo:

The data contains real-time metrics on the distribution of knowledge and attitudes within a firm at a highly granular level. You can get metrics on for specific of the firm, for specific classes of employees and for specific topics. You can do this for either customers or employees, and have the metrics for any moment in time. The quality of these metrics will be extremely strong, because participants have been incentivized to reveal their true expectations.

So I don’t disagree with Bo at all, I just had a slightly different focus in my discussion. Perhaps this could provoke a little further discussion on the long-term potential of prediction markets in corporations.

[UPDATE]: My sincere apologies for the spam in the original post. I have no idea what happened, but must be related to the fact I didn’t use my regular posting program to put this post up. It should be fixed now.