Mercury's Blog

I wanted to provide some quick, incremental results on some research that I’ve been conducting this summer.  For those of you not familiar with my paper from the Journal of Prediction Markets, I examined how small a group of traders was necessary in order to have calibrated results.  (Using the Inkling algorithm before they instituted Robin Hanson’s MSR, that point was at about 15 traders.)  Unfortunately, there was no alternate “expert” picks opinion that I could turn to in order to compare the performance of a prediction market to any other prediction mechanism.

This year, however, was different.  I created a set of markets that encompassed the nineteen events of Henley Royal Regatta, the oldest and most premier rowing regatta in the world.  Then I discovered that another site had been created (in conjunction with the Regatta Radio station) that allowed individuals to submit their picks for who was going to win each event.  That site required that a pick be made for each event.

While I will be working with them to examine the detailed results, initial indications are clear.  Just before the deadline when picks had to be entered, I submitted an entry that represented the top-ranked competitor (by percentage) in each market.  That set of picks ranked #9 out of well over 200 entries, properly selecting 12 winners out of 19 events.  This is a similar performance to that discussed in the paper “Prediction Markets: Does Money Matter?” by Emile Servan-Schreiber, Justin Wolfers, David Pennock, and Brian Galebach.  Only two people selected more winning crews.  (One picked 13, the other 14 winners.)

I think the prediction market could have performed even better, but there was a lag between information coming available and the market moving.  Specifically there was a significant injury to one of the favourites just before entries for the picks event closed.  While that information was public, the prediction market didn’t see the effect for a number of hours afterward.

Again, I’ll be digging into the detail of this event (and others I’ve been running this summer), and will hopefully have a paper ready later this year.

As a last note, I would like to thank Inkling Markets, who agreed to keep the markets running for the research this summer.

This entry was posted on Monday, July 16th, 2007 at 10:12 am and is filed under General.