Tagged: stats

Surrey Police and the Case of The Misleading Pie Charts

Surrey County Council have sent every household in the county a booklet explaining how our council tax is being spent. Within it is a highly political comment from Kevin Hurley, the newly elected Police and Crime Commissioner.

He presents a pie chart showing how the police force spend its money. Take a look at it and ask yourself this question: what percentage is spent on "Employees".

Surrey Police Pie Chart

Please use this poll to record your guess - answers at the end of this blog.

Pie charts have a long and noble history. They were popularised by Florence Nightingale and were hugely effective in helping politicians understand the causes of death among soldiers during the Crimean War.
Nightingale-mortality

As we understand more about the human brain and how we perceive shapes, it is becoming clear that pie charts are ineffective for representing complex information.

2D pie charts can still serve a useful purpose in limited circumstances. The real problem is with 3D pie charts. As far as I can tell, these abominations were popularised by Microsoft's Excel charting software.

3D charts distort the view of the data in such a way that it becomes increasingly hard to understand the information being presented. A picture being worth 1000 words, allow me to demonstrate:
GraphJam3d

So, just how bad is Surrey Police's Pie Chart? In an extremely scientific study of asking half a dozen people, they all guessed between 75% and 85%. That's quite a wide range considering it's a multi-million pound difference.

On the opposite page to the pie chart is this summary of spending.
Police Spending

In slightly more readable format, it is:

Category £ %
Employees £181.70 81.9%
Premises £8.00 3.6%
Supplies £27.20 12.3%
Transport £5.00 2.3%
Total £216.90 97.7%

A few interesting things to note here.

Firstly, how do we calculate the percentages? The total spend isn't mentioned in the report (£216.90). If we use that, "Employees" accounts for 81.9% of spending.

If we take into account the gross expenditure (£207.70) the figure jumps to 87.5%.

Secondly, if we do assume that we're using the unreported total spend - there is at least 2% missing. Some of which can be explained by rounding - but I wonder what the rest of the money is spent on.

Given the above, I don't think the provided pie chart allows Surrey residents to see an accurate view of how their hard earned money is being spent.

Hopefully, this side-by-side - of the above data - will show you how 3D pie charts distort data and end up misleading their audience.
3d 2d pie chart side by side

With this overlay, we can see the distortion much more clearly. The smaller sections of the chart look disproportionately larger.
Pie Charts Overlayed
It's time to announce a zero tolerance crackdown on dodgy data representation.

What I've Learned From A Crazy Month of Blogging

Well. That was an intense NaBloPoMo! I published a blog post every day in November - as I have for the last few years - but this was unlike anything that went before. I had over 50,000 viewers in a single day due to one of my posts, got hit by reddit and HackerNews, and even got asked to do some paid blogging!

I started this month hoping to average 1,000 page views per day. This was so I could hit the (pretty arbitrary) milestone of half a million page views.

This is what my November looked like...

Which means my total stats since 2009 are...

So, let me take you through what I learned.

When You Help Others - You're Really Only Helping Yourself

I've been badgering my wife constantly to write on her blog. I managed to convince her to partake in NaNoBloMo and she has done marvellously. I've been seriously impressed with her writing and her dedication. It has been great seeing her struggle with the challenge and having it pay off so magnificently.

People Are Stupid

I just wasn't able to put this adequately into words until my wife blogged, but some people really are stupid. I've been told that my arguments are invalid because (in no particular order)...

  • I haven't calculated something to N decimal places
  • I have an obvious anti-Apple bias
  • I have an obvious anti-Android bias
  • I used hyperbole
  • I mistakenly claimed something took X months, when it actually took X+1 months

In short, people seem to ignore the bigger picture, find the smallest and most inconsequential mistake, and then use that to hang an entire argument. Predictable, I guess, but a little depressing.

I could try to write everything in formal language, perhaps written in pure predicate logic, and illustrated with examples backed up by no less than 9 separate sources - but I have the feeling that would be a little dull to read. There must be a middle ground somewhere.

You Only Tell Me You Love Me When You're Drunk

I hate being told I'm wrong. Especially by anonymous commentators. By contrast, I love being told I'm right. Especially by anonymous commentators.

It's a weird experience to see strangers praising and damning you in - so it seems - equal measure. Bad reviews stick around in your brain far more than the good ones.

Interestingly, when I've tackled the anonymous people saying I'm an idiot, they've either apologised straight away - or run away.

Bandwidth

May 12, 2012 held the record for the busiest day - 17,186 views thanks to this article about the SIM-less Phone.

My ecosystem blogpost got 37,776 views on November 23rd. The 24th of November saw it get 51,928 views!

Which was nice.

Luckily, my blog is heavily cached and has gzip compression turned on - but even still, I started getting alerts from my host that I was edging close to the limits of my agreed bandwith. So, I bought some more.

My hosting provider - Vidahost (affiliate link) - stayed rock solid even at the height of the traffic. They were incredibly quick to respond to my questions and even gave me some free bandwidth while I was waiting to see if the traffic would continue growing.
You can use the discount code "edent" to get 10% off your order with them.

Tomorrow Never Knows

I was completely stunned by the posts which "made it" and those which fizzled into obscurity. I thought both the one about the HackerNews Effect and Why Don't Amazon Sell ePubs would do rather better than they did.

I don't know if they contained poor ideas, weren't well written, or just didn't get promoted properly. But, there we are.

Don't Hold Back

The majority of posts were written over the last two years. One was even three years old! They'd all been sat in the draft state waiting for me to be happy with them.

The Smuggling USB sticks post was my first big "hit" of this NaloPoMo - it got 22,553 views in a single day! Yet it was first written in 2010 after the BPI threatened to sue me. I'm not really sure why I sat on it for so long...

So, the moral is either "publish those posts before they get too old" or, alternatively, "Let those old posts mature like a fine wine."

Forward, Never Backward

I'm not sure if I can keep up with Richard Herring in blogging every day - although I do have a few posts lined up for December. It's been a fun - and slightly stressful - November, so perhaps it's time to take a short break.

Thanks for reading!

What's The Front Page of HackerNews Worth?

One of the things that jollies me along during NaBloPoMo (where I have to write a blog post every single day in November) is seeing that people are reading my blog. I like watching the visitor counter tick gently upwards. I also love to see people discussing, arguing, and commenting on the posts I write.

When I started this month, I looked at the blog's statistics and decided I wanted to get 30,000 views in the month of November. I normally average 600 views per day. So, how to get that up to 1,000 per day?

I had two main strategies (other than writing interesting and engaging content.

  1. Share my posts on social networks - letting my friends know that I'd published something.
  2. Submit my posts to external sites - use HackerNews or Reddit to share my writing.

I used the JetPack Publicize plugin to send my content to Twitter, Facebook, and LinkedIn.

For external sites, I thought that Y Combinator's HackerNews was the best destination for some of my posts.

My first post - about smuggling USB sticks - was submitted at 1300GMT on Sunday 11th November and, fairly quickly, made it to the front page as readers upvoted it. Near as I can tell, it remained on the front page for 24 hours.

SD Card Stats

So, 216 points and 93 comments on HackerNews resulted in roughly 30,000 extra visitors.

You can see a graph of the performance throughout the day on Hacker News Rankings.
HN Stats USB

My next successful post was about the antics of Helen Goodman MP and her self-proclaimed inability to use the Internet.

I figured 1300GMT on a Friday was a good time to submit again. It catches UK workers having or returning from lunch - and it's 0800 on the East Coast of the USA, so catches commuters there.

The post stayed on the front page for roughly 12 hours before dropping off.
Goodman Stats
The blog got 112 points and 103 comments. It netted roughly an extra 12,000 visitors.

Again, you can see a graph of the performance throughout the day on Hacker News Rankings.
HN Stats Goodman
The majority of these visitors came directly from HackerNews - but people obviously started sharing the posts on Twitter, Facebook, and via newsletters.

Once the fuss had died down, these were the sites which sent my blog the most traffic.

Referrer Stats

Now, not every story I've submitted has done so well. Most have never even troubled the front page. I obviously haven't yet hit on the winning formula of decent content, inflammatory headline, and serendipitous timing.

As an extremely rough metric, every hour on the front page of HackerNews was worth around 700 page views. It's too early to say whether that has lead to a pronounced increase in my regular visitor numbers.

I don't have adverts on my blog, although I do have Amazon affiliate links. They netted me a total of £4.04. I also encouraged a bunch of people to join the Open Rights Group, which has put me in the running to win a MaKey MaKey

A special "thank you" to my hosting provider - Vidahost (affiliate link). They stayed rock solid even as I received over a month's worth of traffic in a single day. You can use the discount code "edent" to get 10% off your order with them.

NaBloPoMo - Stats

As we enter another NaBloPoMo - where I try to write a blog post each day in Novemeber - I thought I'd take a look back at how this blog has developed.

On Friday, October 30, 2009, I switched on WordPress statistics so I could get a better idea of what was popular on my site. My average traffic back then was 80 visits per day. Not bad for a backwater blog.

Since then, my writing has been getting better (I hope), my content has become more interesting, and I've had several stories which have spread far and wide.

The blog now gets ~600 visits per day.

monthly blog stats

What I find interesting is that it's some of the older posts which get the most regular traffic. This post about getting Android to run on Nokia phones is often in my "top ten" list - despite being published over three years ago.

May 12, 2012 was the busiest day with 17,186 views thanks to this article about the SIM-less Phone.

Arbitrary Milestones

I'm not sure how accurate the stats are, but since they were set up (two years after this blog started) I've had 470,034 views in total.

So, my goal is to get to the half million mark. Hopefully by the end of NaBloPoMo or, more realistically, by the end of 2012.

Here we go!

Facebook's Mobile Adverts - Real Stats

Facebook has been getting a lot of criticism for its lack of mobile revenue. A fact it tried to hide from its IPO. Much ink has been spilled, but is it really necessary for Facebook to worry? Here's a quick case study.

Facebook has, in its infinite wisdom, decided that I would be interested in adverts for cancer. Or, perhaps, AXA have decided that 30 something males are a prime market.

The creator of the advert was Equator's Fiona Dow who, judging from her bitly profile just loves posting about cancer.

As I have mentioned several times before, bitly links are a great way to (unintentionally) share your stats. If we look at the clickthrough stats for this advert, we can find some interesting nuggets.

Here's the biggie - referrers shows where people where when they clicked on the link:

In total, nearly half of all clicks came from the mobile site.

Perhaps this is why Facebook hasn't jumped into bed with any dedicated mobile advertiser? It would seem that users are equally willing to click on Facebook's "Sponsored Stories" on mobile as well as web.

(This assumes that AXA targetted both platforms equally).

The only fly in the ointment is, you guessed it, AXA don't have a mobile friendly site.
AXA non-mobile site

I've been banging on about mobile-friendly advertising for years - and still advertisers don't get it! Not everyone has an iPhone. Not everyone who has an iPhone is on WiFi or 3G. Not every one want to have to pinch and zoom. If the first impression you're giving your customers is that you don't care about their needs - don't expect them to stick around too long.

So based on this datum, Facebook users are willing to click on mobile ads - all it now requires is Facebook to show them at appropriate times and advertisers to create mobile-dedicated campaigns.