How to construct a bogus survey

Statistics are often used by newspapers as the basis for a story. People are far more likely to agree with you if you tell them that they are on the side of the majority. This is why bogus statistics are so effective in moralising comment pieces. It’s a lot easier to say, “hey, most people agree with us” than convince someone with facts. By bogus, I don’t mean data fabrication  (though that happens too), I mean rigging the questions to get the answers you want. Here’s how:

Divide and Rule
A poll that just went up on the Telegraph’s website has the following responses:

Options one, two and four all appear sensible. Option three sounds outrageous. Unfortunately, you can only pick one answer, this splits the rational vote. Those of a right wing disposition will most likely shun treatment, legalisation and equality (which are apparently all mutually exclusive) and therefore gain an unfair advantage on option three.

The Leading Question

In psychology and law we have a phenomenon called “leading questions“. Evidence is inadmissible if the witness is given a hint of the answer in the question.  I don’t know about you but I’d call a headline titled “Drug gangs controlling parts of British cities” a leading question. Placing the survey within an article also restricts your sample to people choosing to read that article.

The Nullified Answer & The Catch 22

If you look closely at option one in the Telegraph’s poll it actually includes two answers which are normally considered divisive. The US have lobbied internationally to stop treatment efforts in the hope that making drug use more dangerous will prevent it. A number of presidential candidates actually believe blocking treatment is a good form of prevention. Likewise, very few that support treatment will choose this option because the poll is rigged so supporting treatment means also signing up to prevention. In short, next to nobody is going to click this option but this stat will come in handy when lobbying against treatment.

The PR firm

Yesterday I highlighted a bogus poll in the Daily Mail, that purportedly found that “a quarter of young British women are dating at least three men at once“. After a couple of emails I’ve managed to get them to give me a press release. The research was done by a PR firm hired on behalf of a high street restaurant. Sound Fishy? The press release they sent me has lots of information about the restaurant’s “rich food and beverage heritage”, their “juicy burgers”, “tender ribs” and “hearty steaks” but none of the information normally associated with a study (such as the sample information, methods or the actual questions that were asked) is present. Apparently the research is ongoing so I can’t get access to the data, though they’ve assured me I can have it when it’s finished, I’m not holding my breath. PR firms are great for hiding information and deflecting bad publicity, in fact that’s what they are for.

The Reporting

Results from the Daily Telegraph Survey as of 28/02/12

Now you’ve got your bogus results you can make up your shock-horror headlines as you see fit, here are some examples we could use:

• 92% OF BRITS OPPOSED TO TREATMENT OF DRUG ADDICTS
• ONLY 8% OF BRITS WANT TAXPAYERS’ MONEY SPENT ON TREATING DRUG ADDICTS
• 56% OF BRITS OPPOSED TO LEGALISATION
• 39% WANT TOUGHER SENTENCES FOR DRUG USERS

The greatest part of the plan is that you never have to worry about getting caught because you don’t have to explain how you got your dodgy data! Welcome to the PR industry.

Reference

Tajfel, H., & Turner, J. C. (1986). The social identity theory of intergroup behaviour. Psychology of intergroup relations , 7-24