Evidence or Blind Faith?

If you wanted to close the gap with Australia, wouldn’t you look at why the gap exists?

Listener: 6 March, 2010.

Keywords: Growth & Innovation;

A meta-analysis brings together all the existing research studies, pooling their conclusions to get an overall one that is more precise and revealing than the individual studies. John Hattie, professor of education at the University of Auckland, goes a step further. His book on how students achieve, Visible Learning, brings together over 800 meta-analyses in a kind of meta-meta-analysis, based on over 52,000 studies and involving over 200 million students (although some will appear in more than one study).

The study of educational achievement is outside my competence (although as a social statistician I have to know something about meta-analyses). But as an economist, I am extremely envious of the number and depth of these studies. Some years ago, I reviewed the evidence on the link between unemployment and health: about 100 studies. Each seemed potentially methodologically flawed, but collectively they were so overwhelming I concluded a long period of involuntary unemployment was highly likely to be very bad for your health (and the health of your partner).

In contrast, I have never found much research evidence on the effect inflation has on health. Some people claim inflation is bad for you (and clearly it mucks up market price signals) but where’s the beef? This is an example of “ideology”: one may believe it to be true, but there is no strong evidence to sustain the hypothesis, and it may be false, in which case the ideologists ignore the contradictions. Belief in the harmful effect of unemployment on health is far more scientific, although scientists are always trying to refine the analysis.

In comparison with the connection between unemployment and health, much of economic belief is closer to ideology than science. This is well illustrated in the 2025 Taskforce report Answering the $64,000 Question: Closing the Income Gap with Australia by 2025. (What a dreary title; one has to brace oneself to read it.)

The Australian economy has been growing faster than New Zealand’s since the late 1960s, and Australia’s per-person income has been higher since the mid-1980s. Because of the open labour market across the Tasman, New Zealanders can migrate to Australia for the higher income (although they might not if they were more concerned with other aspects of their quality of life; we seem to do better at educating our children). The 2025 Taskforce was established to address what could be done about the economic gap – it seems uninterested in the quality of life.

Having explained the potential problem, the taskforce demonstrates that indeed there is an income gap. The next step is surely to gather the evidence explaining why the gap has occurred. Astonishingly, the report is almost totally bereft of such evidence. The complete lack of reference to scientific studies might lead a visitor from Mars to conclude that no serious research has been done on the problem (and would be an irritation to a scientist hoping the taskforce would use some of its funds to create an inventory of the research).

Instead, it makes the desultory claim that none of the differences it identifies explain the income gap – without looking at them in any depth or at any alternative coherent explanations. Yet with no evidence to explain the past differences, the taskforce announces its policies are necessary to address it. How do we know its solutions will work? Trust it. This is ideology bereft of evidence.

After a review of the evidence, what strikes me is that 15% of the fall relative to Australia was during the Rogernomics Recession when the taskforce’s policies were being pursued. Surely the taskforce needs to explain why prescriptions that seemed poison then won’t do similar damage in the future. But that involves a tougher scientific standard than the taskforce expects of itself.

In the end, the report reads as if the taskforce wants to move society in the direction that Roger Douglas and Ruth Richardson and their friends wanted. The gap with Australia is an excuse to impose its policies despite there being no evidence they will address the gap. The electorate told them in 1990 and 1993 (and in favouring MMP) what it thought of that.