In the search for correlations, are economists forsaking rigorous standards for sloppiness?
Listener 12 July 1997

Keywords: History of Ideas, Methodology & Philosophy; Regulation & Taxation;

Nobel prizewinning physicist Steven Weinberg recently wrote “the existence of a common standard of judgement leads physicists, who are no more saintly than economists, to question their own best work.” He was referring to Alan Guth who, having discovered the universe went through an inflationary phase early in its creation, nevertheless tested his own work to see whether the hypothesis was wrong. As Karl Popper says, be your own sternest critic. Is it fair to imply economists are less rigorous?

Consider the paper on taxation and growth by American Gerald Scully, commissioned by our Department of Inland Revenue, and quoted by a number of politicians and political lobbyists. It purports to demonstrate that the optimal level of taxation in the economy is about 20 percent of GDP, claiming if that is exceeded, economic growth is inhibited. For the average member of the public or politician the procedure by which this number is generated is a deep mystery. Those with an anti-public sector bent have seized upon the ratio, claiming that the reason for our poor growth record is that taxes are too high.

The study is very sloppy, not even defining the key variable, and subsequently misinterpreting it. It covers two broad periods. The 1930s and 1940s when tax rates were low and economic growth rates were high, and the later period when the relationship was reversed. It should be easy to get some sort of correlation between high tax rates and low growth rates. In fact Scully does some really odd technical things, which suggests that the relationship is not a very compelling one.

But suppose one got a good correlation. Under the Weinberg-Guth-Popper test a scientist would want to ensure there was no alternative which gave a better explanation. Otherwise one is vulnerable to the post hoc, prompter hoc fallacy, assuming because two variables are correlated one caused another. If one believes that roosters cause the sun to rise, the fact that roosters usually crow before dawn is presented as compelling proof. But Scully and, indeed numerous econometric studies widely quoted by politicians, seem oblivious to the problem.

If there is a negative correlation between the numbers of economists and the economic growth rate it does not follow that fewer economists would generate an economic boom, even if those who want to eliminate economists might seize on this finding. (I have not actually correlated economists with the growth rate, but I have no doubt that I could get the result, for there are all sorts of tricks one can do to boost a sagging correlation if one is desperate – or unscrupulous – enough.)

So even if higher economic growth rates were associated with lower average tax rates, any reasonably proficient economist can think of a host of testable explanations to explain this correlation has nothing to do with causation. Scully is simply satisfied with the correlation.

But he does an even more extraordinary thing. He ends up incestuously correlating the same variable with a transformation of itself.

I worked this all out while soaking in the bath. (No, I did not leap out and scare the horses.) Scully announced that the optimal tax rate was 20 percent of GDP. So did my bath devised method, even though it does not use the economic growth rate at all. It simply depends upon the arithmetic of the tax rate. If Scully had been investigating relationship between the optimal number of economists and the tax rate he would still have got the optimal tax rate as 20 percent! Despite an innocence of elementary econometric analysis, the article is to be published – so much for economist’s scientific pretensions.

One of the standard ways of testing the robustness of an estimate is to apply it to a shorter period and see whether it still works. (By “standard” I mean in the sense Weinberg uses the term.) When I applied the Scully approach to the last ten years, the optimal tax rate became 57 percent. But you would be as foolish to argue for 57 percent as for 20 percent. This is loony tunes.

Politicians have a right to be very angry about the Scully work. They used it in good faith, but it has made them look like proper charlies, totally ignorant of economic science. I wont name the gulled, but I will list at the end of the year those who continue to make a fool of themselves by using the Scully results. The irritation to economists is that it has wasted their time (and in my case relaxation in the bath), to deal with this level of incompetence, for which, it is said, the Department of Inland Revenue paid $94,000. Or rather you and I, dear taxpayer, paid for this hogwash.

And if the Department is not responsible for the conclusions of the study, it is responsible for its quality. If this is its level of quality control in the tax system, it is no wonder the slick willies advising the wine-box players have taken the Department, and all other taxpayers, for a bath.