BBC, BNP and Statistics
By @edent
· BBC bnp reporting statistics · 5 comments · 600 words · read ~393 times.
There are 10 types of people in the world. Those who understand statistics and those who don't. ±8.
Ever since I took GCSE and A-Level statistics, I've had a healthy appreciation for the way they are presented to the public.
I vividly remember my grandmother shouting at the television one night. The news presenter had said "20% of people polled - that's nearly a quarter..." Before she could finish, my grandmother loudly interjected, "Nearly a quarter? It's exactly a fifth!"
The way people see, hear and understand statistics is very complex. It's a rich mine of study for economists, neuroscientists and anthropologists. Given its complexity, shouldn't we me making it as easy to understand as possible?
I'm going to show you a few poorly presented statistics from the BBC News site. I pick on the BBC only because it's the site I read most often. Other news services are just as bad.
BNP
There is much wailing and gnashing of teeth at the apparent bump in the polls the fascists got following their Grand Wizard's leader's appearance on Question Time. Here's how it's reported.
BNP support in poll sparks anger
The opinion poll carried out after Mr Griffin's appearance found 22% of voters would consider voting BNP in a future local, general or European election.
Two-thirds of the 1,314 people polled by YouGov for the Daily Telegraph dismissed voting for the party under any circumstances, with the rest unsure.
When asked how they would vote in an election tomorrow, the proportion supporting the BNP stood at 3%, up from 2% a month ago.
However, more than half of those polled said they agreed or thought when asked if the party had a point in speaking up for the interests of "indigenous, white British people".
I don't want to comment on the validity of the poll - but the way in which its results are presented make it really quite difficult to gain a quick understanding of the situation.
The stats are presented in four different formats.
- Numerical: "22%"
- Verbal: "More than half"
- Fractional: "Two-Thirds"
- Relational: "x% up from y%"
This means that within 4 paragraphs the user has to switch their mental model of how the statistics are represented 4 times. It makes it hard to easily compare the statistics on offer. It leads to confusion and, ultimately, ignorance of what is being reported.
In the above example, what percentage of people were "Unsure"?
Off the top of your head, is two-sevenths greater or less than 22%? By how much?
Political Party X has seen its support drop by nearly a third to just over 40%. What was the party's original level of support?
These head-scratchers become easier when expressed in a common base - or when set out graphically.
Florence Nightingale is credited with creating the Pie Chart in order to explain to non-statistical civil servants the problems she faced in the war. While Pie Charts have some usability problems, there's no reason why a suitable graphical representation of the numbers can't be shown. It allows those less versed in numerical reasoning a chance to evaluate the data presented.
The joke at the top of the page is based on the classic Binary Joke. It relies on your confusion between base-10 and base-2. When you mix and match different ways of expressing statistics, you might as well be mixing binary, decimal, and hexadecimal. When reporting on statistics, clarity is the key to comprehension.
I find it interesting that humans are not wired up for understanding statistics as elegantly as they can innately for all sorts of other evolutionarily-advantageous things (interpreting facial features, detecting deceit, basic game theory, accurately throwing objects etc) - many of which are 'mathematically' far harder.
One would expect that those of our ancestors who could juggle risk, reward, prediction and assumption the most effectively would have been those who'd reproduce most effectively.
I'd say it's the syntax.
But there are also plenty of famous, simple examples, told in anecdotal form, that most humans completely fail to grasp. Monty Hall's boxes, base rates in medical diagnoses etc etc
Looks like they have managed to trigger just about every cognitive bias that we have, all in one paragraph - never mind error margins and the effects of attitudinal versus behavioural data in polling situations.
Andrea Trasatti says:
My take is that the way it is presented inherently influences the way you see it. If I say "more than half of the respondents said X" you will be convinced it's true, if I say "2 out of 3 said X" you will think the number is way too small. It's politics masked like numbers.
cyberdoyle says:
I like the way reporters quote statistics too. My favourite is the famous BT quote that '99.6% of the UK is connected to a DSL enabled exchange'. This seems to say that everyone can get broadband, yet 3 million households and businesses can't. they may be connected to an exchange capable of broadband but I say: 'half the country (40%) 😉 can't get a decent service'. Tit for tat. Lazy journos don't bother investigating, they just quote OFCOM or BT. Bad use of stats IMHO. chris
There's also the nebulous statistic which (all) mobile companies use - population coverage. You can only cover a few major cities and have excellent population coverage, but your geographic coverage will be rubbish. And, as no one live on the train lines you can get away with not covering them.