Mercury’s Blog

If you're new here, you may want to get Mercury's Blog by Email or subscribe to my RSS feed. Thanks for visiting!

It’s been two weeks since the Second Conference on Prediction Markets took place in San Diego, CA.  I wanted to give my impressions of the conference and some of the more relevant papers for real-world practitioners.  (My apologies for not posting this sooner, but I took a bit of a holiday after the conference and have been suffering through some significant jet lag!)

Many of the papers presented were quite theoretical in nature.  I’d like to discuss two and how they apply to your prediction market issues.

“Information Aggregation Mechanisms and Environmental Complexity“:
Authors: Paul J. Healy, John O. Ledyard, Sera Linardi, and J. Richard Lowery.

This paper discussed the results of laboratory experiments.  The authors looked at which mechanisms (CDA, MSR, Pari-mutuel and iterative polling) performed best at different levels of complexity.  In general, CDA performed best in less complex environments while MSR and polling performed best in more complex environments.

While this is a highly academic paper, the overall theme can help guide our future prediction markets.  Where an issue is largely black or white (such as who will win an event) CDA is an effective mechanism.  Where there are more shades of grey (such as a theoretical combinatorial market) MSR is a more effective mechanism.

The paper is much more specific, but this is a good “takeaway” for practitioners.

“A Field Experiment on Monetary Incentives in Prediction Markets“:
Author: Stefan Luckner

Stefan Luckner did some interesting research on incentives.  He ran a series of prediction markets on last summer’s football (soccer) World Cup.  By separating traders into different groups with different incentive systems, he was able to start analysing how the different systems perform.

The three systems he examined were:
- Fixed Price: Each trader earned 50 Euros without consideration of performance
- Rank Order: Top three places earned 500, 300, and 200 Euros, respectively.  All other traders earned nothing.
- Performance-based: Each trader’s earnings were based on their final deposit.  (Starting deposit equalled 50 Euros)

His data revealed that the Rank Order incentive system was significantly more accurate than the other two systems.  In the Fixed Price system, there was no incentive to perform well, while in the Performance-based system risk-averse behaviour would be rewarded through maintaining the original stake.  In the Rank Order system, traders were incentivised to speculate and maximise their market earnings to climb the rankings for the final (real-money) payout.

Like Bo Cowgill, I’m a big fan of rank order incentives.  They encourage speculation and competition among traders.  (It’s how I know, for example, that I performed better on the Washington Stock Exchange than prediction market luminary Prof. Justin Wolfers!)  Depending on an organisations culture, this can be a fantastic motivator to keep trading levels consistent on your company’s prediction market site.  Stefan’s research is a good start in this area, but much more research is due.

Summary

Overall, it was a good conference, though focused largely on academic topics in the field of prediction markets.  The day finished with a good Industry Panel consisting of Dave Perry (ConsensusPoint), Emile Servan-Schreiber (NewsFutures), Russell Anderson (HedgeStreet) and Matthew Fogarty (Electronic Arts).  I was pleased to finally meet in person a number of people with whom I’ve only previously traded e-mails and phone calls.

Finally, a quick bullet-point list of trivia:

• A surprising number of MacBooks among attendees!  I’d estimate 50% or more.
• In honor of Chris Hibbert, a number of us went to the Zocalo restaurant on Tuesday evening in Old Town San Diego.  Great food, and great conversation.
• Congrats to the organisers.  It was very well-run, and everything ran on time!

Trying out new software, new tools and new processes is critical for long-term success.  A trial will provide you with very important feedback about features, and help you figure out how exactly you can best use the software/tool/process in your organisation.

Doing a trial is just as important when implementing a prediction market in your company.  Luckily, most of the prediction market software vendors are able to do fairly inexpensive (to free) trials of their software to help get you started.  Prediction markets clearly fill forecasting gaps in organisations, because from my discussions with the various software vendors, it is quite rare for a company to do a trial and not purchase the full-fledged solution.

What are some of the key things you need to accomplish in a trial?

• Solve a problem (even if it’s a small one)
• Start with a small, diverse group
• Keep your head down

Solve a problem (even if it’s a small one)

This is important because prediction markets in a company should not be seen as entertainment only.  A large manufacturing firm shouldn’t be focusing their prediction markets on the 2008 presidential election or the winner of the baseball World Series, at least at first.  Instead, start a market to see if people can predict data like number of quality defects in the month, number of workplace accidents, etc.; provided that the final number is known and published by the company.  The key is that prediction markets should show they can forecast data, even if that data is seemingly inconsequential.

Solving real problems will show that prediction markets are a tool, and not a time-wasting device.  It will also provide for small successes that will lead to increased willingness for prediction markets on more strategic data.  All you need to prove in a trial is that it works on a small level; the rest will happen naturally.

Start with a small, diverse group

For a trial to succeed, the entire company doesn’t need to be invited to trade.  Instead, get a small group of people that you know are interested in the idea.  What’s more important is that the initial group of traders is diverse.  It’s no good for the market if the twenty people you ask to participate all work in the same group of desks, in the same division.  Get people from sales, from finance, from production, from administration, from whatever divisions your company has.  You will find that a small handful of people from each division will be quite accurate in their predictions, and will certainly have better quality than one division alone.

Keep your head down

This may not apply to all trials, but I would recommend it if your trial isn’t sponsored by a top executive.  There are a lot of managers who are threatened by prediction markets.  Middle managers in particular don’t like their employees being able to essentially go around them and provide information to executives, particularly if a project or product isn’t going well.  If they find out about your prediction markets project before you have a track record of success, they will probably try and kill it outright.  So keep your head down, and don’t try and attract too much attention until you’ve had successes that you can build on.

Trials are critical in order to find out if prediction markets are right for your company, and if they’re valuable for your company.  I believe that most organisations will find quite a number of uses for prediction markets, and that a trial is the best way to discover these uses.  The points above should help you ensure that your trial is a success.

———–
Just like any blogger, I’m very interested in your feedback.  I’ve implemented ClickComments below.  Instead of registering and leaving a comment, you can click on any of the buttons shown.  Choices include: Write More, Insightful, Cool Stuff, Creative, Entertaining, Inspired Me, etc.  I’m interested in your feedback!

If you’re interested in what other readers have liked, here are links to some of my most popular posts.  (ClickComments are available for archived posts, too.)

- Structuring your market
- Developing a business case for Prediction Markets
- Designing markets for real-life use