Mercury’s Blog

If you're new here, you may want to get Mercury's Blog by Email or subscribe to my RSS feed. Thanks for visiting!

When pitching a prediction market forecasting solution within a company, everyone from the consultant like myself to the software vendor to the internal company champion needs to recognise the real value of hosting prediction markets inside the organisation. The value to the business is NOT that prediction markets are accurate. Let me repeat that: the value to the business is NOT that prediction markets are accurate.

Why is that? Well, it’s because accuracy is a bad story. If you start talking accuracy, then you need to discuss calibration, scoring rules, and the like. This gets complicated fairly fast, particularly when you forecast probabilities. (David Pennock has a good run-down of the topic here if you want to read further.) The real reason that accuracy isn’t the primary business value is because increased accuracy simply isn’t that valuable to executives. Forecasting has always been imperfect because the future can never be perfectly known, and an improvement here is still imperfect.

So what is the value of prediction markets in an organisation? I would recommend listening to Todd Proebsting talk about the first business-related prediction market that Microsoft held. (Link here is to yahoo confab conference on prediction markets.) His story described a prediction market that was very accurate. But the real power of his prediction market story was that it revealed undeniable evidence to the project leadership that there were problems with their product. Using a prediction market connected the leadership of the project with their team. Without having to read reports, talk with scores of employees, or other time-intensive activities, the project leadership was able to immediately receive feedback from the team on a key metric: when the project was going to be completed.

This is what I try to impress upon companies that are looking into prediction markets: they are tools for connecting with employees throughout an organisation. In the best case scenario, the prediction market forecast is exactly what the project manager / executive expects, or it matches up with previous forecasting methods. In the best case nothing has been lost (except a marginal cost for operating a market), and the company has gained by interacting with their employees and fostering another type of community at the company.

In the alternate scenario, the prediction market forecast is dramatically different than what the executives believe, or what other forecasting models show. This case is where prediction markets hold their true value: in forecasting surprises. If a company’s leadership can be informed of potential surprises with sufficient time, they can take action to mitigate or solve problems. Though the managers in Todd’s example probably didn’t like being told of a surprise, in the end I’m sure they appreciated being able to manage the problem proactively.

• How much is it worth to your company to realise six months ahead of time that the project isn’t going to finish on time?
• How much is it worth to your company to realise that if the new product’s quality will be poor?
• How much is it worth to your company to realise that your employees believe that the new product won’t get nearly the number of sales that you’re counting on?

I’d like to repeat that the value of prediction markets to a business is NOT their accuracy. The value of prediction markets to a business is that they quickly and efficiently connect company and project leaders with their employees and teams to get insights on important issues.

—————

Quick notes: if you have QuickTime Pro, if you click on the link above you should be able to download the two and a half hours of conference as a QuickTime movie. It’s around 590 MB, though, so it will take some time! I’m working on burning a DVD from the MP-4, please contact me if you’re interested.

Finally, if you like reading this blog, please click this link to subscribe via e-mail, or click this link to subscribe via an RSS reader.

Cross-posted at Midas Oracle.

Writing effective contracts can be very difficult. Chris Masse has blogged about the InTrade/TradeSports problems closing out the markets regarding North Korea launching missiles, as well as some issues regarding BetFair and control of the US Senate after the 2006 elections, all of which occurred because of poorly designed contract settlements.

I’ve created four different markets on Inkling to test a different type of contract and settlement criteria. What many people are interested in for each Congress are:

1. Which party will control the Senate and House of Representatives?
2. How many seats will each party have in the Senate and House of Representatives?

More people are generally interested in the “control” question than the “number of seats” question, probably because it takes much more knowledge and research to effectively trade on the number of seats each party will have. Unfortunately, this can cause problems depending on how the contracts were written.

How do you define control of the Senate or House of Representatives? Do you define it by a majority of seats? Or do you define it by a plurality of seats? The results may be dramatically affected in a close election when third party candidates are involved. After the 2006 elections, who would you say controlled the US Senate? There are currently 49 Democrats and 49 Republicans; with the Vice-President, it could be argued that the Republicans controlled the Senate. Unfortunately for Republicans, the two Independents caucus with the Democrats, so in reality the Democrats control the Senate.

What I have done in the Inkling markets (the House market, and the Senate market) is to base the settlement of the markets off of which party elects a key leader of each chamber. The statement I used for the Senate market is here:

[Party] will control a majority of the seats in the Senate after the 2008 election. If a third party (or independents) prevents either the Democrats or Republicans from an outright majority of seats, this market will settle based on the party membership of the President Pro Tempore.

This covers both scenarios: the much easier scenario where a party has a clear majority of seats thus winning control, and the scenario where a coalition results. Given the nature of the two parties in Washington today and the election procedures of the US Senate, the President Pro Tempore will certainly be a member of either the Democratic or Republican party. I used the same type of settlement criteria in the House of Representatives market, basing it on the party membership of the Speaker of the House. I believe this is an effective way to determine which party will really control each chamber, since it goes beyond the mechanics of getting control (number of seats and such) and speaks to the party that actually has the power in each chamber (who has the authority).

The other type of market (how many seats will each party have) is quite a bit simpler. In the US Senate, with 100 seats, I created markets for the Democrats, Republicans, and Third Party candidates. In the US House of Representatives, I decided to use a market based on percentage of seats instead of the exact number of seats. This is because I wanted to use the Inkling market which forced all of the options to sum to 100%. Unfortunately, this scaled claim may be a bit confusing to traders, but I tried to explain it with this statement:

[Party] will control this percentage of seats after the 2008 elections. Each percentage point corresponds to 4.35 seats. (Alternately, each seat corresponds to 0.23 percentage points.)

While in a perfect world I could have created a market that automatically summed to 435 instead of 100, this still works to elicit the information I’m interested in for the 2008 elections.

Elections are like many business problems; writing contracts that are based on what your organisation really wants to know can be difficult. I hope that this post provokes some different ways of looking at potential prediction markets in your company. Please comment here or contact me with any questions.

Cross-posted at Midas Oracle.

One of the hardest parts of building a community around a prediction markets site, whether it’s a public site or an internal corporate site, is educating traders.  In the research I completed last summer, I found that most individuals (when observed with little education outside help files) don’t “trade” so much as they “vote.”

What does this mean?  Well, many traders will pick the contract they believe to be the best among the options presented, and put money down on that option.  What’s important is that this is where they stop.

A few other traders did slightly more complex trades; they may vote one option up while at the same time short-selling another option they they don’t believe will occur.  But whatever they believed, they did not vary from their original opinion by selling their shares or purchasing their short-sells back.  These two very basic styles of trading were exhibited by over half of the traders that I studied.  Little knowledge of trading is required to trade in this way, but there are fewer profit opportunities as well.  Only about 7% of the traders participating in the project were highly active traders, and these were the traders that drove the market’s performance.  (These numbers don’t cover the people that registered but chose not to trade, which also speaks to trader education.)

In order to drive higher levels of adoption, traders need to be properly educated about how to trade.  New users to BetFair receive a series of e-mails that instruct them, step-by-step, in how to login to their account, transfer money into their account, trade, and trade in increasingly more complex methods.  This is exactly what any prediction market site (real-money, play-money, corporate, non-profit, etc.) needs to do.  By teaching their users how to trade, they will become more comfortable trading and trade more.  This will increase liquidity, efficiency and accuracy of any prediction market.

I only have two criticisms of BetFair’s approach.  One is that they were essentially not permission-based.  Instead of reading the next step when they felt they were ready, a new user instead gets approximately an e-mail a day for a couple of weeks.  My second criticism follows from this; there was no real proof or encouragement of education going on.  Compare a set of e-mails to the training materials Kathy Sierra discusses on her blog.  In the horse training packs that Kathy blogs about, users have a huge variety of ways to try out their new learning, prove it to themselves, and receive encouragement to progress further.

In my ideal world, trader education would be a key backbone of any prediction market.  New users would register and would start on a journey to become a master user.  They would receive materials at their own pace; once they had accomplished the planned lesson, there would be feedback and more materials would be sent to the user.  (This could include both electronic lessons as well as physical materials, promotion of user groups, etc.)  Each step would bring them more knowledge of how the market worked and more ability to earn a profit.  This applies to every prediction market, and not just real-money sites!  I use BetFair as an example since they have the best education programme I’ve seen thus far, even though there are significant improvements that they can still make.

Prediction market operators should NOT view educational development as a cost, though it may appear that way financially.  Investing in trader education will pay off significantly through greater word-of-mouth, greater liquidity and more opportunities for traders.

————-

Remember to subscribe to this blog by either clicking here to use an RSS reader or clicking here to subscribe via e-mail.

I’d like to thank everyone who has taken the time to check out the Prediction Market Resources lens at Squidoo. Users like yourselves have added a number of papers and books that are great prediction market resources. Hopefully this will gradually build to be a primary starting point for people new to the industry.

I’m a huge fan of blogs, particularly blogs that feature original thinking and thought provoking content. For some time now I’ve been keeping up to date with these sites by using an RSS reader, specifically Google Reader. I’ve “shared” a number of items, and thought many of you may enjoy some of the posts that I’ve found interesting, and at times earth-shattering. (Many seem to be written by Seth Godin, Garr Reynolds, Kathy Sierra and others.) If you’d like to sign up to see this feed, just click on this link and add it to your favourite feed reader.

Speaking of feed readers, if you’d like to read this blog in your RSS reader like Google Reader and haven’t already subscribed, please click on this link. If you would prefer another method, you can sign up to receive these posts by e-mail instead! To do so, just click on this link.

(If you’re one of those that prefers checking out the sites directly, you can see my shared items by clicking here.)

Finally, I will be attending the Second Workshop on Prediction Markets in San Diego in June. I’d like to arrange drinks and a dinner that evening, if anyone is interested. (Perhaps this is already being done as part of the conference; if so, please let me know!) Either leave a comment here or e-mail me and we’ll arrange the details.

There is a lot of information out on the web, in papers and books about prediction markets. However, I’ve been at a loss when trying to point people in the direction of a central repository of key resources.

Well, that’s now changed. After hearing about Squidoo a number of times from Seth Godin (one of the most forward-thinking marketers and businessmen alive - he started the site), I decided to create a “lens” focused on the top resources for people interested in learning more about prediction markets. What this lens does have:

• Great books on prediction markets (and market design)
• Top academic papers on prediction markets
• Blog posts from myself and Midas Oracle
• Google Alerts on the keywords “prediction markets”

This lens is not meant to be the central repository of everything that is available on the web regarding prediction markets. For that, I would recommend Chris Masse’s site. However, by focusing on the key books and papers it will hopefully help people that are new to prediction markets get started without being overwhelmed.

Perhaps most importantly, this page isn’t meant to be static. You can add your own book and paper suggestions and then vote them all up or down in importance, and also add your comments on the page messageboard. You can also add this to del.icio.us, subscribe to the RSS feed to keep up to date with changes, and more. While the graphic design is a little “busy” for my tastes, I hope that by focusing on key resources, everyone can understand prediction markets just a little bit faster.

Please take a look, and let me know what you think by writing a message on the lens!

Cross-posted at Midas Oracle.