This is a very common trap that decision-makers fall into. Find out how.
I will begin with two analogies: “Nail production” and the “cobra effect”.
The famous “nail production” analogy
A company announced to its employees that they were going to assess their productivity by the number of nails manufactured. The result was a large number of useless small nails; then, the managers came up with a different metric: the weight of nails manufactured. The result was few but heavy, again useless nails”.
This story itself could be a manufactured one; but, it is a good introduction to the concept.
However, Charles Goodhart, a British economist who served on the Bank of England monetary committee articulated the concept and first wrote about it in 1975.
The following sentence is the common reference used by experts to explain Goodhart’s law.
“When a metric becomes the target, it fails to become a good measure”
Charles goodhart; source: https://www.oxfordreference.com/view/10.1093/oi/authority.20110803095859655
I found another interesting analogy to the concept.
Cobra effect
This story relates to India’s British colonialism era. At that time, the rulers wanted to get rid of the cobras prevalent on the streets. As a strategy, the rulers offered financial rewards to those who brought dead cobras; it was an instant success. However, with time, people started to breed cobras to claim more money. When the rulers found out their folly, they gave up the reward. As a result, they experienced more cobras on the streets because people released their “harvest”. (I do not know how far it was true).
Let us take another metric now:
Number of calls per hour
Suppose you want to expand your service and you decide to give incentives to your employees at the call centre based on the number of calls made per hour. And you also set a target: 10 calls per hour. Now, the metric becomes the target. If you do not add another related metric/s about quality, your employees will achieve the measure at the expense of the quality of the call because they might be tempted to end a call within six minutes disregarding meeting the callers’ needs.
However, I find, that Drs. Gregg Fonarow and Boback Ziaeian expand it a little further in their editorial comment to the Journal of the American College of Cardiology. They say that this could happen when the metric is attached to financial incentives.
Metric attached with incentives! What incentives?
Here, the keyword is the incentive. Goodhart’s law comes into play when the metric becomes the target and it is attached with incentives – usually, but not always, with financial incentives.
Hospital readmission rates
Drs. Gress Fonarow and Boback Ziaeian capture this law in their editorial to the American College of Cardiology about hospital readmission rates with the following sentence;
“Metric of performance attached with financial incentives does not serve its original purpose.”
Drs. Gregg Fonarow and Boback Ziaeian
According to them, sometime back, healthcare researchers observed higher readmission rates associated with some selected conditions of heart failure within 30 days of discharge. They wanted to reduce the 30-day hospital readmission rate by giving financial incentives to residents as well as financial penalties to the management of their hospitals that failed to meet the target.
As expected they achieved the target, however not without unintended consequences.
A group of researchers led by Dr Ankur Gupta in 2018 reported at JAMA Cardiology that a hospital readmission reduction program was associated with reduced 30 – day and 1-year readmission rates but at the same time associated with increased 30-day and 1-year mortality rates.
The law can come into play not only with financial incentives. Consider the following example.
Number of research papers published
In the academic field, the number of research papers published is the standard metric in the advancement of science, economy, or any other academic discipline.
However, if the ultimate goal of this metric is either the expertise of the subject or the technical know-how of the paper publishing, I think, Goodhart’s law does not come into play. It matters only when we consider this metric as a measure of personal academic advancement because it may lead aspiring researchers to play around with data sets to show statistically significant results and spin with the writing of abstracts and conclusions.
That means the operation of Goodhart’s law depends not only on the metric when it is attached to the incentives – either financial or otherwise – but on the target or goal that is expected to measure.
This is an introduction to Goodhart’s law. Those who have explored the concept further have come up with its variants. One such expert is David Manheim; those interested can listen to one of his interviews through the following link:
The interview: https://www.mixcloud.com/rationallyspeaking/rationally-speaking-240-david-manheim-on-goodharts-law-and-why-metrics-fail/