Popular Posts

Caveat Emptor

The opinions expressed on this page are mine alone. Any similarities to the views of my employer are completely coincidental.

Monday, 11 May 2015

Pollster that got it right

Obviously there is a lot of soul searching going on in the polling industry right now given what  looks like a rather serious failure to get close enough to the target that mattered - the percentage voting for the  two largest parties. I've no more insight into what went on than anyone else. The industry has set up an inquiry to be chaired by Professor Pat Sturgis. He's a good man for the job and that gives me some confidence that  if the answers can be found they will be. We'll all await the verdict with interest.

Meanwhile, here's some food for thought. Damian Lyons Lowe CEO of Survation revealed that they conducted a  poll on the 6th of May which was very close to the actual result. However, they felt it was so out of line with all the other polls that they didn't offer it to the Daily Mirror.

A couple of things strike me about this, one trivial, the other possibly more significant. The trivial thing is that clearly I don't entirely understand the polling industry's business model. I assumed that pollsters carry out polls for clients ie nobody asks any questions unless someone is paying.  One interpretation of  Lyon Lowe's account is that  I'm wrong about this: "We had flagged that we were conducting this poll to the Daily Mirror as something we might share..." suggests that the Mirror was not formally the client for the poll.

Much more important is the implication that the polls, or at least what we the punters eventually see of them, are not independent probes of the underlying reality. In this case we have a poll judged too far out of line to enter into the stock of publicly available data and thus the precision of what we actually get to see is over estimated.

I wonder - and I hasten to add I don't know the answer to this question - how far this is just the tip of the iceberg? In polling what is observed is not what you get, by which I mean the raw data has to be weighted to produce estimates that, hopefully, approximate what you would have got had you been able to sample from the target population. Weighting is unavoidable, but inevitably it involves judgments about things that, by definition, are unknown. Now here is my thought. I wonder about the extent to which pollsters are looking over their shoulders at each other when they make weighting decisions? Polling is a commercial activity. If you are certain you are right being an outlier might be good for business. If not, in a crowded market place where reputation really matters, staying near to the average might look like an attractive strategy.

No comments: