Should You Trust App Store Reviews?

@edent7 comments · 350 words · read ~1,645 times.

If you've ever released an app - or, indeed, any creative work - you know that one of the worst things you can do is read its reviews.

Oh, yes, sure - it's important to listen to your customers, it's vital to act on their feedback, and you should always take their suggestions seriously. But when it comes to reviews... well... perhaps it's best to ignore them. Start reading them and you'll find that they follow the exact same pattern as the bottom half of the Internet - vain, vexatious, and vicious.

If you're an average customer, reviews can be worse than useless for determining whether an app is any good. It can be hard to determine if a negative review is the fault of the app - or the user's phone. Is the reviewer the sort of cretin who complains that a £0.99 app doesn't have the same polish as a £40 video game? Has the reviewer misspelt even the most basic words? These are all warning signs that a reviewer should not be trusted.

Yet, even knowing all these things, it's easy to be seduced by a negative or positive review that sounds plausible but is what we in the industry call "total unmitigated bollocks."

So, gentle reader, I present perhaps the best app review in the Google Play App Store. Do let me know if you've spotted a better one!

How do we counter this? Is there some objective method by which we can tell if an app is any good? Can we create a series of real world metrics?

How many crash reports an app has sent.
Whether people uninstall the app quickly after installation.
The number of time the app is used per month.
Average session length when the app is used.
If users reinstall the app when they move phone.

Those are just some sketched out ideas - I'm sure there are many more criteria to pick from.

User reviews have their place - but they're no substitute for accurate and objective metrics.

7 thoughts on “Should You Trust App Store Reviews?”

2012-12-28 10:35

M P (@right_mr) says:

I guess you are trying to highlight something with that scrrenshot however I would urge you to use one for a non generic app . Flash player is very generic ... maybe pick bbc iplayer !
Reply
1. 2012-12-28 10:45
  
  Terence Eden says:
  
  I'm not quite sure what you mean by "generic". I've used that review because it's an excellent example of someone who doesn't understand what they're talking about. If you've got a good screenshot of a review - do please leave it as a comment on this post.
  Reply
  1. 2012-12-28 14:09
    
    Kevin says:
    
    Rolls of Thunder, some interesting reviews: http://frickingnutz.com/files/rot.PNG
    Reply
2012-12-28 19:21

Jules says:

And the Indie Marketplace reviews are poluted by other Developers denegrading their competitors. Its a mess, that registered Developers should not really be allowed to review competitor apps and Games.
Reply
2012-12-29 17:55

Sucker Sink says:

Are you serious? Developers with half a brain will take a look at a number of reviews and comments, filtering out the crap and taking note of good ideas and genuine complaints. Users can also see if most reviews are positive or negative before taking the plunge. the operative word there is MOST. There will always be stupid or ill informed reviews, that is life. Even the so called professionals aren't immune from being biased, idiotic, being paid, or just plain clueless on occasion. Whether or not a reviewer has the greatest command of their language or simply slipped a few keys while typing should NEVER be used as a reason to judge what may be VALID points raised. The important thing is do you understand what they were saying? Not the manner in which they said it. To get some of the metrics you suggest would require extra monitoring via googles Play app, which would neither provide timely or even accurate results for a variety of reasons. This is simply because of the nature of android and the plethora of devices it can be installed on. Not mentioning the variety of conditions they operate under. It would also be reliant on continuous data or extra storage requirements for caching before uploading on some very limited devices, causing their users to pull their hair out further in some cases. Bravo for thinking about the issue, but I don't agree.
Reply