Policymaking is too often based on what someone thinks is a good idea.

Listener:  5 December, 2013

Keywords: Governance; Growth & Innovation;

The Prime Minister’s chief science adviser, Sir Peter Gluckman, has called for the greater use of evidence-based policy formation. It arises out of his medical background, where there has been an increasing demand for evidence-based treatment.

Surprisingly, a lot of health and medical treatments are not scientifically proven. Very few alternative medicines have any efficacy but even a considerable number of standard treatments provided by doctors are not assessed as robustly as you might think; they are evaluated for safety, but effectiveness is a different matter. Even the most up-to-date doctor, when faced with a particular condition, often has to use judgment based on incomplete evidence.

I was struck by this some years ago when evaluating a new diagnostic instrument that cost, literally, millions of dollars. It was in fashion and everyone wanted one. But we could not get any research that showed it led to better health outcomes. Instructively, those selling the instrument did not bother to answer our queries. There were bits and pieces of research that showed that it might work, but I was reluctant to argue for one for every hospital.

As it happens, time has shown its effectiveness and most big hospitals now have one, although no doubt it is still sometimes misused. Understandably, doctors tend to be cautious and order tests that may not be necessary, even though they are of considerable expense to the taxpayer (or as an economist would put it, the resources could be better deployed elsewhere).

If the health sector has not got it quite right, public policy is worse. How often does one observe the implementation of policy that seems to be based on what somebody thinks is a good idea – or is in his or her private interest – but where the evidence for its efficacy is thin or even contradicts the likelihood of the policy having an effective outcome?

The classic example was the imposition of Rogernomics. There was plenty of evidence for the need for market liberalisation in the early 1980s. But those who drove the changes went far beyond what the evidence indicated. The outcome was that the economy stagnated and the extremist policies failed and had to be reversed. Yet moderate market liberalisation has worked much as was expected. (Of course, we do not give much respect to the social scientists who correctly warned at the time that the contradicting evidence was being ignored.)

Ironically for Gluckman, one of the areas where evidence-based policy hardly exists is how we treat science and innovation. This is not to criticise the science; I don’t have the expertise – although I am in awe of the ability of geological scientists to see miles below the ground and millennia back in time.

Nor am I arguing that “science and innovation” is unimportant. It is what comes after the “therefore” that troubles me. Rather than a careful explanation of how science and innovation articulate with our social goals, all we get is what amounts to “therefore we should spend lots of public money on me”. (Recall the Rogernome’s “market liberalisation is important; therefore make changes that benefit me.”)

I am not even saying we are spending too much on science and innovation. It is a question of direction and priorities. I suspect a careful analysis – almost an oxymoron in policy – would give greater priority to developing a science-literate population, to improving our ability to import and adapt foreign technologies, rather than pretending we can be a great centre of widespread original research and, reflecting New Zealand’s competitive advantage, to shifting the balance from physical-based sciences to biological-based ones.

Those are my interpretations of the evidence, but what relevance has that in science policy? If the advocates of the current science and innovation policy ran their scientific research the way they advocate policy, with an almost total neglect of the evidence, they would be terrible scientists. Gluckman’s call for evidence-based policy is welcome.