Circulated 13 April 2023

There is a fetish for setting performance targets in, among other places, the education and health system. This note draws on economists’ experiences to caution about their use, and illustrates how this has mattered in the health system.

The foundation principle is Gilling’s Law which states ‘how the game is scored determines how the game is played’. Set out by Don Gilling (an accountancy professor from Wellington), a practical illustration is that by raising the value of scoring tries, rugby became a much more open, high trying-scoring game.

The next step is Goodhart’s Law which in its general form says that when a variable is targeted, its relationship with other variables changes. Charles Goodhart is a monetary economist and originally applied his insight to attempting to target the quantity of money to control prives. What he, and others at the Bank of England, found was that the past econometric equations relating the two broke down when they tried to use them for targeting.

However, the result is quite general. It arises as follows. If you characterise a system by a set of equations and add one further equation, the system will appear to function differently despite the underlying equations not having changed. Typically, econometric equations (which are just a sophisticated form of observation) are simplifications of the complexity of the system and the apparent relations get changed.

(A comment for economists. There is talk of ‘the death of the Phillips Curve’. We would expect it to appear to break down if the management of the economy began using the curve in their settings, just as has happened with the money to prices equation.)

Gilling’s Law applies to the way the ‘game’ is ultimately won or lost. It is looser than Goodhart’s Law (i.e. the ‘shapes’), which is about a particular variable which is used to determine the game outcome. (Scoring tries does not in itself determine the game’s outcome, it is scoring more tries and conversions than the other side – and not incurring too many penalties to offset a better try record.)

The next step is about when an intermediate target is established with the target as only an indicator on the way to the ultimate target/score. For example, waiting times in the emergency department of a hospital are not, ultimately, the object of the exercise. But the longer they are, the more likely that object (the wellbeing of the patients) will suffer.

Gilling’s Law and Goodhart’s Law warn that any such target is likely to be corrupted. Patients in ambulances are held outside the point which is triggered by the target definition. They wait just as long but are not recorded in the target measure.

Or in the case of where the target is the number of surgical operations, the tendency is to do many simple ones rather than a single long complex one.

It is not surprising then, that intermediate targeting does not have a good record and is often dropped. That does not mean that we should not record the statistical indicators upon which the intermediate targets are based upon and evaluate them intelligently to see if we can remedy poor performance. But the implication is that there should be many more indicators, rather than mechanically judging performance on a handful of indicators in the way that those with a target fetish do.

The Gilling-Goodhart lemma is that intermediate targets are likely to be corrupted.