Early this year I posted my predictions for 2009. In the best spirit of Robin Hanson (getting better predictions by simply tracking how close predictions matched reality), I want to see how I did.

• “Prediction markets in 2009 are going to become even more well-known and wide-spread, but there will be no single event that brings them to the attention of the public. It’s going to be a slow, but steady, growth.” – This was spot on. All of the vendors appear to be doing well, but there was no big “event” that brought everyone attention.
• “All of the prediction market vendors will mature their business offering/proposition.” – Not having been on the receiving end of any of their sales pitches, I can’t say this one is true for sure. But from their blog posts and public statements I would assess this as likely.
• “HubDub will continue to only be the only strongly popular play-money prediction market.” – Put me down as wrong on this one. Nigel and the creators of HubDub have focused their time and effort on FanDuel instead. (Rightly, for revenue reasons!) So while HubDub is still active, it’s not the hive of activity it was for a while.

That said, I’m still a fan of niche, public, popular sites like HSX. They managed to turn play-money prediction markets into a real-money revenue stream by analyzing trader behaviour (which can only be seen by administrators) and selling that business intelligence to the studios. This could be replicated in many industries, such as video games or television.

• “While a couple additional software vendors may appear, I get the feeling that the market for prediction market software is largely saturated.” – This was also spot on.
• “I’m looking forward to see how the CantorExchange develops.” – Not so much a prediction, but a hope that it proved interesting. It’s taken a long time to get up and running apparently, and won’t be widely launched for real-money contracts until 2010. (If I read the website correctly.)

A great development from InTrade

Just yesterday John Delaney of InTrade posted on his blog that InTrade will soon be offering some historical market data to the public for free. (As he notes, this means they’re losing a source of potential revenue, as historical data can be quite valuable.) This is a great development, and should be a solid source of data for people to dig in and get interested in how traders operate in a prediction market. Kudos to John for doing this.

a

Tomorrow is going to be a landmark day for prediction markets. The 2008 US election cycle has been the most-polled, most-predicted, and likely the most-analyzed election in history. It’s been going for nearly two years, and I for one am glad the election will soon be over and governing (by whomever wins) will soon begin.

But why will it be a landmark day for prediction markets? Simply put: the data.

Prediction Markets and Polls – The Data

There are prediction markets on a wide variety of sites, with both play-money and real-money incentives. Iowa Electronic Markets, InTrade, and Betfair for real-money; HubDub, Inkling, NewsFutures for play-money. There are more, but these are the sites I’ve seen cited most often. (It’s too bad ConsensusPoint didn’t push TheWSX.com this election cycle.)

More importantly, there are a few sites that offer incredibly deep (and also probabilistic) analysis into polls. Most notably fivethirtyeight.com, which I seem to be checking a couple of times a day, now. There are national polls, national tracking polls, state polls, and even some state tracking polls! Fivethirtyeight in particular does deep-level statistical analysis to determine from poll results and demographic data how likely each state is to vote for each candidate.

The number of data points, from different prediction markets, polls, and poll trendlines/analysis will be immense. The sum total of data that will be available after this election should be a treasure trove for researchers, and should finally prove the accuracy of prediction markets.

But there’s a hitch…

Yes, there’s always a hitch, and it’s something I’ve discussed before. In elections, polls and prediction markets are measuring two different things.

Polls are measuring the percentage support for a candidate. Generally around 40-60% or so, unless it’s a total blow-out.

Prediction markets measure the percentage chance that the candidate will win their election. When the election is tight, around 50%, when it’s a blowout can regularly be 95%+. (Few prediction markets exist for candidates’ vote share; really only on the presidential level.)

What should you expect on Election Day?

I expect that a number of news outlets will be quoting percentages from InTrade in the run-up to the end of polls closing in the evening. Already final results contests are springing up, including one in the New York Times where you earn points based on current InTrade odds. You can also expect a LOT of volume on the markets tomorrow. But once the results start rolling in, the news is going to focus on the candidates alone. Wednesday will start the morning-after evaluation of the polls and markets, which will likely last for quite some time.

What does this mean in the end?

Comparing the results of polls and prediction markets is certainly like comparing apples and oranges. There are certainly some similarities, but they are fundamentally different.

What we need to do is evaluate how each forecasting method performed independently. For prediction markets, that means that a “failure” (where a prediction >50% didn’t happen) is quite likely a success. For polls, that means that a result just a few percentage points off (outside its MOE) is a failure.

I believe that prediction markets will come out looking quite good in this election. They’ve already proven their worth to me; when poll results might indicate a close or tightening race in places, the prediction market magnifies the difference, and in many cases demonstrates the poll volatility is just noise.

In the end, are the results from the prediction markets useful? Based on the number of times I’ve seen them cited this year… the answer is an unqualified YES. Are they perfect? No, and neither is any other forecasting system or technique.

I’m really looking forward to tomorrow…

a

• First, a great post from Koleman Strumpf on Midas Oracle points out that half of all trades on the Betfair exchange in 2004 occurred on Election Day! While I personally think that was quite likely due to the early exit poll news for Kerry and the subsequent swing back to Bush, it proves that there are still quite a few people that may be waiting until the very last day to trade.
• Jason Ruspini just started a new thread on the Prediction Markets e-mail group regarding some divergences he’s seen between prices on InTrade and fivethirtyeight.com.

  While I think some of the things he’s observed is due to the way Nate Silver presents data on his site, Jason brings up a very good point. A thorough analysis of movement in the InTrade prediction markets should be compared to the daily calculated win percentage from fivethirtyeight (where all data comes strictly from polls). I think it could be very revealing, and give the public quite a bit more data on the accuracy of polling, aggregation of polls, and prediction markets.

• A long time ago I started four different markets on Inkling Markets that will hopefully predict control over each house of Congress, and the number of seats each party will have after the election. Data is shown here:
  
  
  (as I write this, the Democratic percentage is 53.8%, which corresponds to 234 seats in the House of Representatives.)
  
  
• I may need to update my post on prediction market software soon. Xpree (founded by Mat Fogarty, and recently joined by Leslie Fine, a well known prediction market researcher from HP) may be changing their name. The top three picks according to their contest on NameThis were:
  1. Metricast
  2. UREprojection
  3. Keymet

  Personally, I don’t like #2 or #3, but Metricast sounds interesting. It also sounds like a much better fit to what they do than “Xpree.”

  Good luck to them, if they choose to go down this route.

• Do you speak Danish? Nosco is hiring!
  For English speakers, so is InTrade and Xpree.

a

McKinsey & Company, the famed consulting firm, recently published a roundtable interview in its McKinsey Quarterly on prediction markets. The eleven-page article (click here to access) featured a discussion with Bo Cowgill of Google, James Surowiecki, Jeff Severts of Best Buy, and Todd Henderson (an ex-McKinsey consultant).

Some of the article had the standard “what is a prediction market” explanation. (A quick reminder that you can watch my video which explains the same thing here.) But I want to point out a number of interesting notes/quotes:

• Best Buy is rapidly catching up to Google in terms of the size of the prediction market effort. At the time of the interview, Google had run 275 markets with about 80k trades since April of 2005. Best Buy had run 147 markets with 70k trades, and involved 2,000 traders. (I believe they have been active since late 2006 or early 2007.) I wrote about the Best Buy initiative after a presentation at the ConsensusPoint New York prediction market conference. While they haven’t moved quite as fast as was mentioned there, Best Buy looks to be a strong champion of prediction markets.
• Jeff Severts of Best Buy initially started doing better forecasting with a simple survey. While not as sophisticated as a prediction market (and more time consuming to calculate averages for the forecast), it generated much better forecasts than the business had ever done before. These experiences showed that they could go further with the idea.
• Bo Cowgill at Google is clearly doing the most advanced work with the underlying prediction market data. They’re the only company that I know of that looks into the underlying organisational dynamics that prediction market data can provide. (We’ve had a bit of a blog conversation about this a few weeks back.)
• Jeff Severts also noticed that any forecasts they had done about their main competitor were not very accurate. I think this speaks to a lack of diversity amongst individuals in a company when critically examining a “competition” in which they’re involved, and similar to the optimism that Bo noted that some Google traders exhibited toward similar Google markets.
• From Bo, traders were more accurate the longer they participated, and tended to be more profitable the lower they were on the org chart.
• I intend to discuss this in more detail later, but James Surowiecki brought up that it is still an open question if prediction markets “are good at forecasting genuinely discontinuous innovations or leaps.”
• Jeff of Best Buy mentioned that “support from very senior executives is essential if you want to issue contracts on anything that might be controversial.” Particularly in traditional companies, I wholeheartedly agree. While less senior executives can make good prediction markets happen within their organisational purview, any company-wide markets need the senior executive “air cover.”
• One thing that I always try to tell people when discussing prediction markets (including in the video), is that they’re really a new way to communicate within a company. I really appreciated Jeff Severts when he said:
  

  Smartly applied, this tool can help management listen to voices, throughout the company, that otherwise go unheard.

  This is a great quote, and I believe a very wise way to think about the benefits of a prediction market.

It’s great to see a well-respected company such as McKinsey publish information on prediction markets widely. This is yet another positive step toward companies viewing prediction markets as mainstream tool, and provided some interesting insights into some of the top active prediction markets operating today.

a

The second half of the paper examined the transmission of information within Google based on the authors’ analysis of the traders and their behaviours. While there is some really interesting analysis there, it has more to do with organisational behaviour than being directly applicable to prediction markets, so I’m not going to discuss it here. But if you’re interested, I highly recommend going to read the paper!

Bo wrote the following on his blog:

This is not the first reaction along these lines. I am perplexed by the response. I can understand why other companies may not want to replicate our analysis of information flows. Perhaps it wouldn’t be worth the effort. Perhaps they would get identical results. And perhaps the company wouldn’t have the all the necessary data.

However, I expected that people could easily see value in the analysis of granular trade-by-trade data — especially if that data is joined with data about traders and outside events happening at the moment of the trades.

To which I respond here:

Perhaps this was a bit of lazy writing at the end of what felt like a longish post. But what it really came down to was my focus on the very tangible, common questions that many people I talk to have when they’re first starting with their prediction market projects. The first issue for many is just getting a market working, with sufficient participation and liquidity, and hopefully accurate predictions. These were the main points and examples from the paper that I wanted to address in my post.

I believe Bo’s concern is that with just a “first-order” discussion on the paper, we leave out the “second-order” potential. Once a market is up and running, there is a LOT of data available to the company. If you know enough about your employees (and Bo’s subsequent post suggests that any decent-sized company does) this is information you likely already have. Specifics had to be left out of the Google paper for sensitivity reasons, but the paper demonstrated the kinds of things you can discover about your organisation through a prediction market.

Addressing these questions takes time and careful attention. There is certainly a privacy issue here, but that’s more an issue of managing perceptions. (You want to make sure people trade their true beliefs, and not some altered reality because they think the corporate Big Brother is watching them.) Both the data collection and analysis would certainly take some time.

However, there is some great data and metrics available to companies that are ready to take advantage of these “second-order” advantages of prediction markets. Again, to quote Bo:

The data contains real-time metrics on the distribution of knowledge and attitudes within a firm at a highly granular level. You can get metrics on for specific of the firm, for specific classes of employees and for specific topics. You can do this for either customers or employees, and have the metrics for any moment in time. The quality of these metrics will be extremely strong, because participants have been incentivized to reveal their true expectations.

So I don’t disagree with Bo at all, I just had a slightly different focus in my discussion. Perhaps this could provoke a little further discussion on the long-term potential of prediction markets in corporations.

[UPDATE]: My sincere apologies for the spam in the original post. I have no idea what happened, but must be related to the fact I didn’t use my regular posting program to put this post up. It should be fixed now.

a

It seems that each month brings one or two new entrants to the prediction market software industry. (I can’t bring myself to use the term “space.”) There is now a lot of variety in the industry, with well-established companies, open source options for the ambitious, and new entrants that are pushing the envelope of what we consider a prediction market. As a consultant that works with a number of these companies to help meet my clients’ needs, I thought it would be useful to review the options as of November 2007.

This is my review of the different software available. Please comment below or contact me with comments, criticism, or corrections. All views below are mine and mine only.

I’ve tried to address software that you can license/download to use on your site, as well as public sites. This post is long, but is comprehensive.

Top Vendors

All of the following prediction market software vendors are well established in the industry, and have a background of successful clients and projects. I have personally talked to or worked with all of these companies, and would certainly recommend any of them. They are listed in alphabetical order.

ConsensusPoint - ConsensusPoint licenses their ForesightServer software, either as a hosted, on-demand solution or on a server within your network. Written largely by Ken Kittlitz, the software has evolved over a number of years and with feedback from a significant number of clients. A particular benefit is the potential separation between the front-end user interface and the back-end market software and database. Depending on how much time and effort you want to invest, you can completely design your own user interface for a custom solution.

Who to talk to – Dave Perry

Gexid - Gexid is a new software package, and has been used by Nokia in a project run by Bernd Ankenbrand, who just finished his PhD. I don’t know much about the software, but Bernd had some interesting results that he presented at the London prediction markets conference. From what I understand Gexid integrates a good chunk of consulting effort to complete a prediction market solution. Company established in Germany.

Who to talk to – Bernd Ankenbrand

Hollywood Stock Exchange - HSX is still the most prominent play-money prediction market out there. It, too, has over ten years worth of development effort. From what I understand, using HSX as your prediction market platform is most suitable for quite large-scale projects. Their past clients include big media prediction markets associated with magazines, television shows, etc. The back-end tools around data reporting and demographics on the system are particularly powerful, as they are the core of the main HSX business.

Who to talk to – Alex Costakis

Inkling - Inkling is one of the “new” kids on the block, having only been started in fall of 2005. (It was developed and partially funded through Paul Graham’s Y Combinator program.) Developed using the Ruby on Rails platform/framework, it has a slick user interface and can easily be rolled out for your needs. A particular benefit is their recent automatic free trial promotion… just sign up on their site and get going!

Who to talk to – Adam Siegel

InTrade/Tradesports - While InTrade and TradeSports are two separate companies now, the same software platform powers them both. They have occasionally licensed this platform for use by other organisations. Most prominently, the Financial Times used the InTrade platform to run the FTPredict site this summer (as discussed at the London conference here.) Though it can potentially be a bit expensive, the biggest benefit of this software is the “battle-hardened” nature of the platform.

Who to talk to – John Delaney

Newsfutures - Newsfutures is one of the few veteran prediction market software platforms, having been originally developed in 2000. They have a strong team, and have developed further related products for companies that don’t feel comfortable with a strict prediction market. (These include their competitive forecasting and idea pageant products.) The benefit of Newsfutures is that they do have these innovative ways of getting the same information in a structure that may be more appropriate for a particular company.

Who to talk to – Emile Servan-Schreiber

Nosco - Nosco has been around for about 18 months now, and has developed a great looking suite of software packages around prediction markets and collective intelligence. They talked about these at the London prediction market conference, and they include a news exchange, idea exchange, and traditional prediction market. So far they’ve had two top-tier clients in Denmark. They clearly have an excellent web design and graphic design team; everything they produced (slides, screenshots) were extraordinarily well done. Company is established in Denmark.

Who to talk to – Jesper Muller-Krogstrup and Oliver Pedersen

Pro:kons – Pro:kons has been in the prediction markets business for a decade, in some form or another. Their work now consists of prediction markets and new media projects. They’ve done some impressive work getting a prediction market platform rolled out for Swiss public television to predict the recent Swiss elections. In order to do this, they had to work with seven different television station identities and four official Swiss languages! They also presented at the London prediction market conference. Company is established in Austria.

Who to talk to – Gunther Fadler and Peter Gollowitsch

OpenSource Options

A number of open source prediction market software packages have been developed over the years. The software listed below is Beta-level or more stable.

Zocalo – Zocalo is written by Chris Hibbert, who was partially supported by CommerceNet during one phase of the development. It is certainly still in development, and you can contact him for more details.

IdeaFutures (aka ConsensusPoint, version 1) – IdeaFutures is the original version of ConsensusPoint’s software. When the ConsensusPoint ForesightServer software was completely rewritten, the previous version of source code was open-sourced. Though clearly not as fancy as the current version, this is certainly a stable and well-used platform.

Serotonin – I’m not terribly familiar with this piece of software, but it is listed as a Stable/Production-level in SourceForge.

MarMix – MarMix is based on USIFEX, which was previously developed by Peter McCluskey. It’s listed as Beta-level in SourceForge.

USIFEX – USIFEX is a prediction market platform that is no longer running, but does openly provide the source code at the link above.

Idea/Innovation Management specific software

These companies/solutions don’t offer prediction market software, but do offer solutions that are close cousins – software for idea selection.

Rite Solutions – Rite-Solutions “Innovation Engine” software was featured in a New York Times article last year. While not strictly a prediction market, their software certainly embodies collective intelligence and crowd wisdom. Employees can submit new ideas, help fund ideas with play-money, and also help move the ideas ahead by completing small tasks that are necessary to move the project forward. In addition to an Idea selection software, it’s really an Idea Management software.

InnovateUs – Again, InnovateUs is not a traditional prediction market software, but uses funding and voting to help identify new ideas out of a pool. They have at least one great client, and are based out of Chicago.

New prediction market software sites

Some of these companies provide private markets for organisations, and some just maintain a public site. I’ve tried to detail the differences below.

PrediCom – PrediCom has been working with prediction markets for a few years, and they have their own platform. They don’t seem to do much work here, and their software is always tied into their consulting services.

Xpree – Xpree was started this summer by Mat Fogarty, who previously championed and ran prediction markets at Electronic Arts using Inkling. They have gotten a beginning version of their software running that you can trial here. Xpree is focusing on a solution that is extremely simply for people to use, to help encourage maximum participation in a business environment. (Mat discussed this at the London prediction market conference.)

Nimanix – Nimanix is a new prediction market software company headquartered in Tel-Aviv. They maintain a (lightly-traded) public market, and are actively selling their solution to business clients. Their software appears to be a strict CDA model, with no market maker.

Qmarkets – QMarkets is a new software company, and is headquartered in Tel Aviv. It is very similar to the Inkling model, with a public marketplace and quick & easy setup of subdomains on their network. The site is quite new, and there is currently little activity on the public site.

AskMarkets – AskMarkets has been developed by George Tziralis in conjunction with his PhD thesis. While currently in private beta, it will hopefully be rolling out soon.

Foresight Markets – Foresight Markets is yet another prediction market platform. While their website is up, it appears that the software is still under development.

New sites, not prediction markets, but “Wisdom of Crowds” generally

GuessNow – GuessNow is probably the most prediction market-like of all these sites, and many people might call it so. You can choose which outcome of an event you believe is most likely, and most importantly, express your confidence level. The more confidence you express, the more points you win if you’re right and the fewer points you win if you’re wrong. (In their scheme, you always win some points just for participating, even if you lose.) My only problem with the site is the number and size of the ads… it’s a little chaotic to my eyes.

Predictify – Predictify has been in the mainstream blogs a bit recently with a couple of mentions on the Freakonomics blog. Unfortunately, Predictify really isn’t a prediction market… it’s a sophisticated polling and market research mechanism. That’s not to say it doesn’t use the “wisdom of crowds,” just that some of the assumptions that James Surowiecki talks about in his book don’t necessarily apply. The reason I say it’s not a market is that a user cannot express their confidence in their prediction. It makes it difficult to sort out the knowledgeable forecasts from the un-knowledgeable forecasts. Unique about Predictify is that companies can submit market research questions, and those that predict right can win the fees that the company pays to Predictify for the listing.

Foretal – Foretal is an interesting site, and has been running for just a couple of months. It combines a Free side, where you can vote (not trade) on what you believe will happen in an event, with a Free and Cash side. Predictions that are Free and Cash have the Free/voting system running concurrently with a Cash betting system that is essentially a pari-mutuel market. You choose how many 1 Euro contracts you wish to bet on whatever options are available. (They take a 5% commission.) On the legal side, Foretal says they are based in Malta, and don’t take cash bets from the United States.

BluBet – BluBet is another site that’s not really a prediction market. It somewhat resembles a pari-mutuel betting system, where you bet on a winner and the people who successfully picked the winner split the pot. However, in this case (like others discussed in this section) there is no way to express the confidence in your bet. If you are absolutely confident in your bet, you have no way to be able to bet any more than the person who chose a random selection.

ZiiTrend – ZiiTrend is really trying to integrate a prediction market and a social network. The prediction market software is quite slick, but stretches what it means to be a prediction market. While you can specify values, like you can specify the price at which you want to trade, there is no way to vary how much you want to risk. It’s simply a vote, but a variable vote. From what it looks like, they want to focus on the social network aspect of this site, and aren’t looking to sell/licence the software.

RIP – No longer available

CrowdIQ – CrowdIQ started around the same time as Inkling, but never seemed to get traction. You can still see elements of CrowdIQ around everywhere, as ConsensusPoint uses the CrowdIQ page layout/style in their default prediction market “look and feel.”

FreeMarket – FreeMarket was developed by Jesse Gillespie, but has since taken the software down. (See here.) Essentially even though he said he couldn’t support it, he kept getting quite a number of questions and requests for help. He’s moved on in his interests, so he just took the software down. That said, I have an old copy, so e-mail me if you’re interested.

Summary

Wow, there are a lot of options out there for prediction market software, prediction market public sites, and general “wisdom of crowds” (such as idea selection and voting systems.) Whether you’re looking for software to use internally in an organisation or you’re looking for public sites where you can try different types of interaction, there are many choices for you. I hope that the list above is useful for you.

As always, please feel free to contact me with questions or concerns. If you don’t want to miss any posts, why don’t you sign up for this blog via e-mail? Just click here and you’ll be all set.

a

Thanks to Lionel Page, I had the chance yesterday to present and talk to a group of academics across a number of different departments at the University of Westminster Business School. (Lionel is a Research Fellow there.)

My talk made a general introduction to prediction markets, and then I discussed my paper from the Journal of Prediction Markets. (The part covering my paper was largely the same as what I presented at the New York and London conferences.)

The start of the discussion, around prediction markets in general, was very useful and stimulating. In a discussion around measuring risk and probabilities, we talked about some interesting potentials for using prediction markets around credit risk, prompted by this summer’s events in the credit markets. While direct assessment of credit risk would likely be too complex for individuals to aggregate, other factors in risk equations might be very appropriate for prediction markets. This could very applicable for large international firms, as a market could aggregate information far more quickly than standard reporting routes.

We also talked about some of the challenges of prediction markets, particularly regarding incentives. Again, there is always a balance that needs to be struck when designing a market and incentive programme. A person should have incentives to participate, but never so much of an incentive (and so little a potential cost) that a trader could gain by sabotaging a project or incentive. (For example, they should not be able to massively short the chance that a project will be done on time, and then work against it being done on time.) This is a delicate question, and also reaches back to whom you invite to the market and what kind of questions you’re asking them.

Overall, it was a great afternoon with Lionel and his colleagues. I hope more of their students hear about prediction markets and how they can be used in a modern business.

————–

I’ve recently upgraded the comment system for Mercury’s Blog. It now uses Disqus, and makes it much easier to comment and see threaded comments. If you have something to say, it’s now much easier to say it!

a

1- Structuring your market
This post talks about the scope of different options you have when assembling the structure of your prediction market.  I discuss continuous double auction (CDA), algorithms (such as MSR or DPM), and other options.

2- Developing a business case for Prediction Markets
In this post from February I discuss three different broad uses of prediction markets, and how to develop a business case for each.

3- Setting Initial Conditions for Prediction Markets
In this post I wanted to impress upon readers the various considerations for a prediction market’s “Initial Conditions,” the initial stake provided to traders and timescales of contracts.

4- Motivating the ranks
To have a successful prediction market means that you need to get people to participate.  This post talks about how you can make that happen.

5- Forecasting Error Calculator
My apologies if you’ve had trouble accessing this before, but the problem should be fixed.  Follow this link to download an Excel Forecasting Error Calculator, that should help you determine how much your current forecasting errors are costing you, and generate a business case for starting a prediction market.

5 Other top posts – Prediction Market design and Industry commentary

• Notes from the San Diego Prediction Markets Conference
• Prediction Markets for the Stock Market? Huh?
• Thinking about Real Money Trading
• Educating Traders – Including Lessons from Betfair
• Industries Ripe for Prediction Markets

Thank you to all of you for reading, and I look forward to posting more on these and other topics in the coming weeks and months.

—————–

I’d like to make a quick reminder that you can click on this link to receive new Mercury’s Blog posts directly in your e-mail inbox.  Or if you’re into RSS feeds, you can click here to add Mercury’s Blog to your feed reader.

a