Instrumental voodoo

Instrumental variables can, on the relatively rare occasions when they are compelling, be a powerful trick to keep up your sleeve. What I find strange though is how often undeniably clever people lose their grip on common sense when they decide to instrument. 

Here is a link to a short article reporting a cross-national macro-level study which purports to show  that private schooling produces better academic results not only for those that buy it but for those left in the state system. Of course private schooling is highly selective and all sorts of unmeasured and possibly unmeasurable traits are correlated with it. Light-bulb moment, let's instrument. 

The instrument chosen by the authors is the % of Catholics in the state's population in 1900. The argument is that the Catholic church was/is the main provider of non-state education. So following the standard instrumenting story we are invited to believe that % Catholics in 1900 (interacted with an indicator of whether Catholicism was the state religion) is related to average PISA test marks round about now, through and only through its effect on the proportion enrolled in private schools. 

I have no idea whether this makes sense for most of the countries included in the study, but it strikes me as absurd in the case of Great Britain. Since at least 1902 most Catholic schools  (at least those that were of relevance to the majority of the population) were to a large degree maintained by the state and subject to the same regulatory regime. In other words they were not in any very meaningful sense private schools. Maybe the main results still come out if you drop GB, but there are times when I have a lot of sympathy for the generalized scepticism of the institutional comparativists about the wide and shallow approach. 

It's never bad to invest a little time learning something about the cases you chuck into your regressions. Of course that might mean  publishing less (and knowing more) and we wouldn't want that would we?