I am really excited about this development in the prediction market industry. In our discussions at the conference, we really wanted to create a community, accountable to its membership, that promoted and educated the wider public on prediction markets. The site will have a directory of vendors, academics and their research, public prediction markets, and eventually a searchable, live database of contracts across the various public markets. This last feature will allow anyone to search for contracts on a given topic, and find current prices across the spectrum of markets. A user could compare prices (and then trade) on, for example, a Barack Obama democratic nomination contract on the Washington Stock Exchange, Newsfutures, InTrade, Inkling, and more. Some other contracts might have an even more world-wide appeal, with contracts on exchanges developed by Nosco or Pro:kons.

None of this means competition in the prediction market industry will be easing anytime soon. To me it means that all of us interested in prediction markets will be working together to “grow the pie” of the marketplace.

Finally, I was honored to be voted in as one of the first members of the Board of Directors. If you have any questions or concerns about the PMIA, please feel free to contact me. I look forward to sharing more news on the PMIA with you here in the future!

a

** Jed Christiansen – Mercury Research & Consulting (me) **

My talk was essentially the same as my talk at ConsensusPoint’s New York conference in September. I explained a bit more about my research project, which was published in the first issue of the Journal of Prediction Markets. I ran a series of prediction markets (using the Inkling platform) in the summer of 2006.

The problem with many business prediction markets is that only a small number of people may know enough to trade on a market. The question I wanted to answer was: how small could a prediction market be and still provide calibrated results?

What I found was that in markets with 16 or more traders, the results were calibrated. (Estimated probability = Observed probability) Additionally, events with a small probability of occurring were well calibrated. I also examined the behaviours that traders exhibited, and effects of manipulation.

These results were all written up in the article; please click here to contact me if you have any questions.

As I mentioned, I first ran these markets in the summer of 2006. I also ran these markets this past summer, and hope to publish the results soon.

** Peter Sorensen – University of Copenhagen **

Peter has been studying the academic aspects of prediction markets for some time, now. He has written papers with Marco Ottaviani, such as “Aggregation of Information and Beliefs in Prediction Markets.” (Marco was at the London Business School, but has since moved to Northwestern University in Chicago.)

Peter’s talk centered around the theory of prediction markets. He gave a good background of the general theory, and talked specifically about some aspects of the Favorite-Longshot bias. It was a good talk, and Peter clearly has done some interesting work in prediction markets.

** Gunther Fadler & Peter Gollowitsch – Pro:kons **

Gunther and Peter’s work in prediction markets stretches back to 1999! (Well before the “Wisdom of Crowds.”) Their original company ran from 1999 to 2002, and ran over 20 prediction markets. In 2002, the company was re-organised/re-structured as BDF-net. The new company’s focus was on prediction markets, consensus finding, and new media projects.

Their talk centered around a huge prediction market project they have been doing with SRG, the Swiss public broadcasting organisation. (Essentially the Swiss version of the BBC.) This was a large and complicated project that is currently on-going, revolving around news coverage of the 2007 Swiss elections.

The project is particularly complex, as they the prediction markets are served on seven different websites (for the different television/radio station website), and perhaps more importantly, there are four official languages in Switzerland! Additionally, Switzerland has a very unique political system, and the markets had to take those complexities into account.

Their solution is a master database, with “skins” for each individual website. So far they have 2000+ users, with 26,000 visitors per month. They are planning on integrating wikipedia, YouTube, GoogleMaps and other services into the solution, too.

Finally, congratulations to Pro:kons for winning the Cisco Austria Web 2.0 aware for social software! This shows that there is some very exciting prediction markets work being done that simply hasn’t made it to the forefront of the english-speaking community. It was great to hear about their work, and I believe we’ll be seeing more of them soon.

————-

Well, that’s it! It was a great conference, with a wide range of speakers from across the prediction market spectrum. As always, contact me with any questions or clarifications.

a

** Will Speck – FT.com **

The FT.com partnered with InTrade to create a realistic market. One of the things they found with their market is that there needs to be a balance between both long term incentives and short term incentives. They had prizes for both, but different behaviours could be driven by a focus on one or the other. The participants on their site (FTPredict) saw the game as “trading,” and not “predicting.” This brings an important distinction when it comes to talking with users, and something marketers should be aware of.

Will saw that prediction market data could be a good tool for the FT audience. FTPredict was seen as a compelling, content-based marketing tool for the Financial Times. There was quite a bit of interest in the project, and they will be doing more contests in the future. While it’s US-only right now because of the legal work required for the contest rules, that could potentially change in the future.

** Emile Servan-Schreiber – Newsfutures **

Emile’s lecture was focused on Newsfuture’s competitive forecasting software. It’s something that Newsfutures has found quite popular among certain corporate clients recently. While not a prediction market by definition, it is related and serves the needs of some of their particular clients better than a CDA model.

He stepped through the Competitive Forecasting model, which lets users enter a high-low spread of where they believe a metric will occur. (ie, sales of widget X between 10000 and 11500). To win points, the forecast must capture the final value. The smaller your spread (more precision), the more points you can win. However, if your spread doesn’t contain the value, the users will lose more points with a smaller spread. Finally, it rewards early users by accumulating points constantly over time.

Competitive forecasting has been popular for variables where there are few participants, are busy/high-level users, people are not familiar with trading, or have few training opportunities.

Finally, Emile finished his presentation by talking about bet2give.com. It’s been pretty successful so far, but Emile mentioned that building a trading base has been a challenge, as their typical play-money traders aren’t interested. That said, they’ve been getting a completely new trader base interested in the site, and have seen some good growth since introducing the site.

** Henry Berg – Microsoft **

Henry Berg leads the Information Markets Group at Microsoft. It was started in 2006, and right now is about 4 people. Their focus is in prediction markets, estimate contests, and other types of information markets.

The original prediction market effort at Microsoft was led by Todd Proebsting in Microsoft Research. This led to the development of the Information Forecasting Exchange, which was in place from 2003 through 2006. Todd is now an advisor to the Information Markets Group.

Henry showed off the tool that Microsoft has built, and is calling PredictionPoint. Their goal was to lower barriers to participation. To do this, they moved to a betting paradigm, as most people are more comfortable with betting. Their user interface lets participants explore bets without typing and without finalising a trade.

In addition to prediction markets, they have what Microsoft calls “Estimate contests.” These are similar to HP’s BRAIN algorithm, but based on a quadratic scoring rule.

Microsoft is actively expanding their efforts in this space, and have run a few dozen markets in the past year or so. A little unusually, their markets are always a bundle of securities (typically 2-8 contracts) that are tied together. Instead of trading on the value of a metric, they trade on which band the metric will fall. (ie, less than 10k, 10-11k, 11-12k, 12-13k, more than 13k) [Chris Hibbert has talked about bands, scaled claims, etc. and this concept here.]

The demo was very interesting, and I hope they eventually talk a little more about the results they’ve seen from their markets.

** Jesper Muller-Krogstrup & Oliver Pedersen – Nosco **

Nosco was started about 18 months ago, and I first met them at the Vienna Prediction Market Summit last year. Jesper and Oliver are clearly polished presenters and experienced in prediction markets.

Nosco provides both a prediction market software package and consulting services around that package. They have built three specific products around prediction markets:
- A news exchange
- An idea exchange
- An information exchange (traditional prediction market)

The News Exchange system has been used by TV2, a television station in Denmark. As stories are written, contracts are immediately created. In this case, the predictions are attached to the stories and pushed out as content alongside the story. They reported that though it was a bit of a struggle to interact with writers at first, it did improve. The system has 21,000 users and over 2,000 markets.

The Idea Exchange system is being used by Danske Bank, and kicked off quite recently. The bank’s goal is to catalog employee innovation, with the goal of eventually accumulating 1 million+ ideas. While currently in trial stages, they have already exceeded initial targets.

Finally, they also have their Information Exchange, which is Nosco’s standard prediction market. In their experience, they found three main drivers in the public markets:
- competition structure (it’s more fun to compete with friends)
- simplicity
- update & feedback (short time to market)
They also found that it’s easier to attract people with one BIG prize, instead of a series of smaller prizes, as a BIG prize simply generates more publicity.

When talking with people in Denmark, Nosco specifically avoids discussing “prediction markets.” Terminology, particularly in a country such as theirs, is very important. A “prediction market” isn’t a buzzword, and people often form mistaken impressions of what they’re doing because of the two words alone.

It’s also clear that Nosco has a very talented graphic designer and web designer on their team. Their screenshots, their graphics, and their presentation were all very polished and tied together nicely. It was a very informative presentation!

————-

That’s the end of part 2, and only a few more speakers remain to discuss. Again, it was a very valuable conference, and I will be posting the last of my notes tomorrow. Comments, questions, corrections? Please comment below or click here to e-mail me.

a

The following are my first section of notes:

** Bernd Ankenbrand – gexid **

Bernd’s presentation focused on the work he did with Nokia on prediction markets. Nokia used prediction markets (using the gexid software) to forecast KPI’s, and had about 120 participants. (KPI = Key Performance Indicator.)

One note from Bernd’s presentation that I found interesting is that Nokia first defined and communicated what the different KPI’s meant on an internal site. Despite the importance to management, the average employee didn’t necessarily know which KPI’s were being measured or what they were. The project led to a unique learning experience for many of the participants.

He did a lot of research on the participants, and I wanted to mention one specific correlation. The heaviest traders on the site were those that most valued anonymity and monetary incentives. While this may seem obvious, it is important to catalog.

When it came to forecast accuracy, Bernd did say that it wasn’t terribly good. But what was really important is that compared to the existing forecast, the prediction markets were an improvement. This is something James Surowiecki has mentioned before: prediction markets aren’t a crystal ball, but they are almost always better than any existing forecasting method.

Finally, congratulations to Bernd for finishing his PhD!

** John Delaney – InTrade **

The week before the conference, John e-mailed the Prediction Markets Google Group to ask what questions people at the conference would want him to answer. His presentation addressed both what was e-mailed to him on the group as well as via personal messages. I’ll address the important/interesting points here.

It’s obvious that John wants InTrade to be much more successful than it has been, but he still clearly believes that InTrade can still be very successful. He mentioned that one key in their business is market makers; a few of their former market makers left, and that had a significant impact in the categories in which they participated. The new gambling laws in the US has had a significant impact on their business, but John is optimistic about the long-term potential of the industry in the States.

Regarding InTrade and TradeSports, John was clear that the companies are separate legal entities, with different management and employees. That said, he said that like a divorce, there are still connections between the two that take some time to replace. They are also looking into ways that individuals can submit contracts to trade on the exchange, similar to what you can do at Inkling. Clearly this would still be fairly tightly controlled, and judgement of the contracts would be done outside of InTrade by an independent entity.

John is disappointed that InTrade hasn’t grown more than they have. At the same time, he seems to be very optimistic about InTrade because their employees are still very motivated and morale continues to be strong. He also said he would fly to the US, but it would need to be for a good reason.

Finally, John addressed the infamous North Korea missile market. He was asked, “Was it a mistake?” He said both No and Yes. It wasn’t a mistake in that the market was judged according to a strict interpretation of the rules. At the same time, it was a mistake in that they didn’t handle the PR issue particularly well. His lesson learned was that they simply need to be incredibly careful regarding their market definitions.

** Mat Fogarty – XPree (formerly EA) **

EA produces approximately 120 games per year. While they forecasted quality of their products internally, there was little accountability for the forecasts. This made for poor forecasts, and so executives didn’t really know which products truly stood out, and which didn’t. This caused them to regularly fill their supply chains with products that just wouldn’t sell in those volumes, a money-losing proposition.

Mat managed to pitch the CEO, and received good support. From that, he managed to drive the project through. What he found was that he got support at the two extremes of the company; the executives liked it because they were being told better truth, and the lower-level employees liked having a forum where they could tell the truth unfiltered. Mat found the most resistance from the middle management in EA.

EA’s first prediction markets were run on product quality, using Metacritic scores to cash out the contracts. (Metacritic produces well-regarded benchmark scores for the industry by aggregating and scoring reviews.) Prizes such as game consoles were given periodically, based on a lottery weighted by a users net worth, and each week the top trader received a stein.

How well did they work? Well, the previous error rate on games was 6.6%, and prediction markets nearly halved it, to 3.5%.

Mat also provided some interesting demographic information on traders, but my generalisation of it is that the more junior people were more likely to register, to trade, and to be profitable.

Finally, Mat introduced his new prediction market company, Xpree. Their software is really focused on maximising participating in a prediction market by making it tremendously easy for users and administrators to participate. He showed some early screenshots of sliders and other new user interfaces which essentially remove the standard trading interface for ease of use. They have started using this software within EA to start predicting ship dates for products and product sales figures.

Finally, they talked about incorporating a wiki into the process. A user could short-sell a contract, and then be directly to a wiki where they could explain why they traded the way they did. It will be interesting to see how this is implemented and utilised. My opinion is that if people have to sign their name (even user name) to wiki comments, it might be difficult to get them to tell the truth.

** Leighton Vaughn Williams – Nottingham Trent University **
Leighton gave a great presentation that explained the various boom and bust cycles of betting and taxation in the UK from World War II onward. He has done a lot of work in the past decade with the government to reduce the taxation on betting.

In 2001, tax law on gambling was changed from a turnover tax to a gross profits tax. This encourages a low-price, high-turnover strategy, compared to the previous high-price, low-turnover strategy. For the betting public, tax was effectively axed. From a betting turnover of £7billion in 2001, this rapidly grew to £53billion in 2007. Though tax rates were cut, revenue stayed neutral and an entire industry exploded in growth.

Finally, Leighton really stressed the Journal of Prediction Markets. If you aren’t already a subscriber, please sign up! An digital subscription is only £30 a year, and if you want a paper copy as well it’s only £49 a year. You can sign up for a free trial here: http://www.ingentaconnect.com/content/ubpl/jpm, or contact Christopher Woodhead for more details. If enough interest in the Journal cannot be sustained through subscriptions, the future of the publication may be in doubt. It’s academic, so they aren’t necessarily looking to turn a profit- just break even.

Additionally, they are always looking for more articles, and they don’t necessarily have to be strictly “academic.” Interesting practical studies on prediction markets, including corporate prediction markets would also be very welcome.

————-

This has already been much longer than I originally thought, and I’ve got quite a few more speakers to write about. Please check back soon for more…

a

This should be an exciting event, and a lot of great people will be both speaking and attending. Please click here for more details. This is the current list of speakers:

Bernd H. Ankenbrand – The Knowledge Laboratory
Dirk Glienke – Nokia
Dr. Michael Gebauer – gexID
John Delaney – InTrade
Mat Fogarty – Xpree
Professor Leighton Vaughn-Williams – Nottingham Trent University
Will Speck – FT.com
Emile Servan-Schreiber – NewsFutures
Henry Berg – Microsoft
Jesper Muller-Krogstrup – Nosco
Oliver Bernhard Pedersen – Nosco
Peter Norman Sorensen – University of Copenhagen
Gunther Gadler – Prokons
Peter Gollowitsch – Prokons
and myself – Mercury Research & Consulting

I look forward to seeing many of you next week!

a

Brent Stinski- Media Predict

Brent talked about his prediction market company, Media Predict. They had extensive coverage of their launch in the New York Times and other media outlets. He reported that over 5000 people signed up on the site, though didn’t go into details. Though the initial press focused on Project Publish, the idea where people compete to predict which book will be published by Simon & Schuster by the contest entries, that is not the sole focus of the site. So far, he’s had a promising start.

Ricardo dos Santos- Qualcomm

Ricardo discussed how Qualcomm has used prediction markets internally, in what I would call “Idea Selection.” In new business development, he is always seeking new ideas and new businesses for Qualcomm. They developed an internal process to seek out new ideas, rate the best ideas, and then a prediction market to determine the best of the best ideas. They had hundreds of people participate, and had good results. It was my impression that they are still experimenting with the technology, but have so far had positive responses from their trial.

Christina LaComb- GE Global Research

GE started this project back in 2005, and is using prediction markets for “Idea Selection.” In their case, employees propose ideas which are turned into the contracts to trade. The results of the market order the ideas, and they’ve experimented with what to do with ideas from there. (For example, automatically fund the top idea, or send the top 5 ideas to business managers for further development, etc.)

Steven Sachs- CNBC

Steven couldn’t discuss much of what CNBC is doing, as it’s still in development. It sounds like they are building a public-facing market, and have run internal markets to demonstrate the technology. I look forward to hearing more about it as they get ready to release a product.

———————–

Panel Discussion, moderated by Alex Kirtland

Mat Fogarty- Xpree (formerly EA)
Fortune Elkins- Misys
Brent Stinski- MediaPredict
Ricardo dos Santos- Qualcomm
Steven Sachs- CNBC
Dave Perry- ConsensusPoint

———————–

Robin Hanson-

Robin gave a quick addendum to his earlier talk, where he focused on the cost-value space of a prediction market. He described an evolution from betting markets, which have negative cost (aka profit) though little value to an organisation, to future prediction markets. Fully evolved prediction markets will certainly have a cost to operate, but the output could have tremendous value to a company.

Me-

It was my turn to speak next. I spoke about my research that was published early this year in the Journal of Prediction Markets. Please contact me if you’d like any copies of my slides.

James Surowiecki- author of “The Wisdom of Crowds”

First of all, James is a fantastic speaker. (It was rather humbling to speak just prior to him.) He spoke about prediction markets in a broad sense and made a number of excellent points.

To start off, bureaucracies and hierarchies and bad ways to move information around in an organisation. At the same time, collective intelligence is still quite counter-intuitive. As humans, we have a deep-seated impulse to find the one person with the answer. (This explains the proliferation of prediction market sites that then seek to sell the predictions of the market leaders, which can lead to disastrous results. ~Jed)

What James came back to is that prediction markets need to be measured against other alternatives for information and decisions, and not absolute accuracy. If they provide an organisation a better way of making a decision, why not use them?

His last points were on the topic of leadership and decision-making. Again, humans tend to associate the concept of leadership with the concept of decision-making. Historically, a strong leader doesn’t put too much stake in his follower’s thoughts and opinions; there’s a reason he/she leads. However, John Craven perhaps took the best or at least most unique approach to this problem when it came to the challenge of finding the Scorpion submarine, lost at sea. He developed the methodology which used the intelligence and experience of many professionals to develop the Bayesian map of where the Scorpion could be on the ocean floor. Craven however led the implementation of that methodology through to a very successful conclusion, finding the sub just a few hundred metres away from the expected location, out of thousands of square miles of ocean floor.

Again, James was an excellent speaker, and I highly encourage you to go hear him if you ever get the chance.

———————–

So this is the end of my notes on the New York prediction market conference. I’d like to make a quick reminder that you can subscribe to this blog via e-mail, either by entering your e-mail address in the sidebar to the right or by clicking on this link.

a

First of all, congratulations to Dave Perry and ConsensusPoint for assembling so many interesting speakers and attendees into a beautiful venue (the Tribeca Grand Hotel in New York.) With people like Robin Hanson and James Surowiecki anchoring the morning and afternoon, it was bound to be a great event.

Here are some of my notes from the day:

Robin Hanson-

Robin gave a fairly standard introduction to prediction markets lecture that some may have seen at other events or downloaded from his website. It was a good overview of the topic.

The question and answer period was the most interesting part with Robin. He was asked about manipulation, and provided some fairly convincing answers that manipulation shouldn’t be a worry (at least with the correct incentives.) Robin described the situation in terms of sheep and wolves. Sheep aren’t that knowledgeable; they are trading for any number of reasons, and are the “noise” in the marketplace. Wolves take advantage of that, and consequently they look for markets with lots of sheep. With better information, the wolves will easily have plenty to “eat.” The net result is that those noisy markets are accurate markets.

Another concept he talked about was creating a “fudge” account. Let’s say you want to weight one set of traders more than another, or simply want to “move” the forecast in one direction or another; create a “fudge” account to conduct those transactions. If after the account has been running for a while and it’s positive, you’ll know you’ve done a good job fudging. But if the fudge account is negative, you don’t know more than the market so just stop fudging and leave the market to itself. It’s a great idea, and fairly easy to implement.

Dave Perry-

Dave talked about ConsensusPoint and prediction markets in general. One of the more exciting clients they are dealing with is BestBuy, which is rolling out prediction markets to every single employee, all the way down to the hourly workers in the retail stores. BestBuy is making a strong push for prediction markets, and it will be exciting to hear about the results. (I’m very bullish for them.) That they developed an internal DVD to help promote the markets showed the dedication they have to the project.

Dave also showed off a demo site with their prediction market software. The newest development is that they have now implemented three different trading interfaces: Simple, Standard and Advanced. The Standard is their traditional interface, where you choose to buy or sell a certain number of shares or a dollar value at which to purchase shares. The Advanced interface shows both the buy and sell order books so you have more information when making a trade. The Simple interface just asks questions about how a person views the contract; ie, do they “believe it is likely” to occur. They are then led to select how many shares or dollars to spend. While it doesn’t quite match the ideal world I’ve discussed before, it’s a great step towards accommodating people with different backgrounds, who may or may not understand “trading” on a financial market.

When discussing BestBuy, they had a couple of instances where the prediction markets were particularly valuable. One was where a forecasting team was trading against their official forecast, demonstrating the power of anonymity in the markets! The second case was where a potential problem showed up by a dropping price in the market, even though the executive dashboard for the project was “green” across the board; the market showed the problem a couple of weeks before the official reports made their way to the executive suite.

Finally, Dave stated two great reasons why prediction markets are used. One, is that it pays people for their knowledge of their company and their industry. Two, it also pays people to “shut up.” Markets are self-selecting and very equal; a person can be a wallflower in a conference room and not be heard, but they will be heard on a prediction market. The influence of the stereotypical loud, brash, know-it-all is greatly minimised.

Fortune Elkins- Misys

Fortune is very passionate both about prediction markets and her company, Misys. In her company, as in most others, “uncertainty costs money.” Fortune has been successfully testing prediction markets at Misys, and has found that they have been working quite well. She’s also had some interesting results, where the person that made massive winnings by (correctly) betting against the crowd was an outsider to the project in question, but someone with extensive knowledge of the company itself. It shows that anyone can be successful in a market, so long as they use the knowledge, experience and resources they have within the company.

Karim Tarwahi- MyCurrency

Karim is a former derivatives trader that has founded MyCurrency. One of their first products is a website (homepredict.com) that attempts to forecast housing prices by individual house and by zip code. This functionality has the ConsensusPoint software at its core, though through a totally custom-designed interface. One of the things they want to do is provide a forum where real estate agents can prove that they know what they’re talking about, and then leverage that reputation. Again, while not strictly a prediction market, MyCurrency is using forecasting as a key component of social networking and reputation building.

—————-

So those are my notes and thoughts on the morning portion of the conference. I hope you find them helpful and informative. Please contact me if you have any questions, and I’ll be posting the notes from the afternoon portion of the day later this week.

a