Statistricks part 3: how they lie to you with polls

Are you diving into the data, or is the data diving into you?

WHAT’S MY TRIBE?

The appeal to us plebs is obvious. We can’t get enough of other people’s opinions, whether our response is to nod sagaciously or spit out our tea.

VIEWS AS NEWS GETS VIEWS

If polls are a novelty gift for the hoi polloi, they’re a godsend for newspapers, struggling as they are with dwindling resources, and for rolling news channels with endless airtime to fill. No time-consuming investigation, photography, writing or planning required — the pollsters take care of it all for free, right down to the covering press release with its own ready-made headline finding. And the public lap it up.

BLUNT TOOLS

As a matter of fact, there are two. The first is that polls are low-quality information.

TORIES, TROLLS AND TERGIVERSATORS

Even if you do somehow manage to round up the perfect microcosm of humanity, there are further obstacles.

MARKET INTELLIGENCE

While they’re passable diversions for punters and convenient space-fillers for papers and news channels, no one ever went on hunger strike to demand more polls. This constant drizzle of percentages and pie charts has not been delivered by popular demand. It’s a supply-side increase, driven by the people who really benefit from it.

OFFICES OF STATE

It cannot have escaped your notice that the worlds of business and politics have been growing ever more closely intertwined. There’s now so much overlap of personnel between Downing Street, big business and the City (the incumbent chancellor, who arrived via Goldman Sachs and hedge funds, is just one of dozens of MPs and ministers with a background in finance), such astronomical sums pouring into the Tory party from industry barons, and so many Tories moonlighting as business consultants, that you might be forgiven for thinking that the two spheres had merged.

“Opinion polls are a device for influencing public opinion, not a device for measuring it. Crack that, and it all makes sense” — Peter Hitchens, The Broken Compass (2009)

Loath as I am to quote the aggressively self-aggrandizing Hitchens, on this occasion, he may have stumbled across a point. A number of studies (pdf) have looked into this phenomenon (pdf), and while the findings aren’t conclusive, they all point in the same direction: people can be swayed by opinion polls.

WHAT CAN WE DO?

Surveys are — or were, originally — designed to present a snapshot of the popular mood. But even the most fair-minded, honourably intentioned, statistically savvy pollster, using the best possible methodology, can produce a poll that is complete and utter Cheesicles.

  • Who commissioned the poll?
  • Who carried out the poll?
  • What was the sample size? If it’s much less than 2,000, ignore it.
  • What’s the relative standard error? (A measure of the confidence in the accuracy of the survey. If Labour are leading the Tories in a poll by 36% to 35% and the RSE is over 2% — as it is on samples of less than 2,000 — then they may not be leading at all.)
  • What were the questions?
  • What was the methodology?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store