Mercury's Blog

In my discussions with potential prediction market customers, I’ve been surprised by one particular application that seems to resonate with many executives. That is the “RAG Status.”

For those of you unaware of the acronym, it simply stands for Red-Amber-Green. It is used extensively in management “dashboard” reports to serve as a quick and efficient status marker for a project or element of a project. Instead of having to read a few sentences or paragraphs and digest them to understand the status of a project, a manager can simply glance at a colored dot.

This is the general scenario from upper managment’s perspective:

• If it’s Green, everything’s okay.
• If it’s Amber, I’m concerned and asking questions.
• If it’s Red, it’s a BIG problem and I am actively trying to solve it.

Project managers creating these reports approach it from a different perspective:

• I’ll mark it Green if I can deal with the problems.
• I’ll mark it Amber if I’m having problems and it’s likely my boss will find out about them.
• I’ll mark it Red if I’m way out of my depth and the situation is probably unsalvageable, and I’ll probably be able to blame someone else.

It is in the reputational interest of most project managers to de-emphasize the problems they’re having in order to save face and avoid unwanted upper management intrusion. However, it is in upper management’s interest to have a true accounting of the difficulties in any given project, so they can understand and properly plan for worst-case scenarios.

The result? A project starts in the Green, slides into what should should be Amber status long before it is ever reported as such, and becomes Red status before the project manager is willing to admit it. In the end the entire team (team members, project manager and upper management) wastes unnecessary time and resources fixing problems that could have been avoided at much less cost had they been identified earlier.

In my discussions on prediction markets, upper management recognise that markets are an excellent way of getting around the poor reporting incentive structure that currently exists. They know they’re being misled by project managers, but aren’t effectively able to do anything about it.

How prediction markets can be used to provide a real RAG status

The first step to providing an alternate RAG status is developing an understanding of what matters in the project. Is it meeting milestone dates? Is it meeting milestone figures, such as number of users? Is it something else entirely? This is then what the prediction market should measure.

I believe that in most cases the market should be structured in a binary, 0-100, probabilistic, “Will the project meet milestone [X]?” Employees, now incentivized to tell the truth on the actual status of the project, will generate a probability that the milestone will be achieved.

From here it is simple: instead of generated a biased Red-Amber-Green colour for a report, management will receive a 0-100% probability of success. If they specify in advance what probabilities are equivalent to the Red-Amber-Green status, prediction market forecasts will create a richer, more informational report that is still very easy for management to digest. (For example, Green = 100% – 75% forecasts, Amber = 75% – 50% forecasts, Red = 50% – 0% forecasts).

Summary

Prediction markets can remove the poor incentives between project managers and upper management. Where project managers can lie or obscure the truth with RAG status reports, when those reports are based off of prediction market forecasts the information is significantly more valuable. Upper management typically realises this, and then just needs to be sold that the costs of project overruns are much more significant than the costs of running the prediction market.

(Some of the most interesting prediction market case studies have been completed using this very technique. I would recommend watching the video of Todd Probsting of Microsoft at the Yahoo confab from back in December 2006. The streamed video can be found at this link.)

This entry was posted on Monday, July 14th, 2008 at 12:13 pm and is filed under General, Implementation.