»
S
I
D
E
B
A
R
«
Ratings “J Curve”
December 20th, 2006 by philip.wilkinson

We’ve always said here at Crowdstorm HQ that rating products or doing reviews involving five stars is a waste of time. Now there is actual evidence coming out to support our hypothesis, starting with this post from the Bazaar Blog, entitled “the ratings J Curve”.

What these guys are saying is that they find the average rating always ends up around 4.3 and that there are more 1’s than 2’s and far more 4’s and 5’s than the lower ratings. Their chart implies that people are more likely to share positive experiences and often go towards the 4 or the five at the end of the scale.

Actually, what you might find across the rest of the web is a peak around the “1″ level where people want to comment if they have had a very bad experience with a product or the “4″ level where they are pretty happy but never quite know what the “5″ really means and assume it is perfection.

Either way – it’s a system with flaws and a skewed data set for sure…

Nic Brisbourne also wrote a really good post on reviews – how useful are they? on his equity kicker blog, which is well worth a read.


4 Responses  
  • Nicolas writes:
    December 20th, 2006 at 7:08 pm

    Really interesting… As reviews are an important part of the Crowdstorm site, do u think you can bring some innovations to avoid the 4.3 effect, or to display reviews in a better way (when you have too many reviews on very popular products, for instance) ?

  • Nic Brisbourne writes:
    December 20th, 2006 at 10:27 pm

    Thanks Phil. Good post. I agree that numbers are pretty useless. The way you add the status of the reviewer to the equation is very powerful.

  • philip.wilkinson writes:
    December 21st, 2006 at 11:28 pm

    Good question – basically it is so subjective and so hard to work out where a product sites on the 5 star scale. For example, I have a nice Philip’s LCD 37″ TV and if you asked me how to rate it then I would wonder whether it was a 4* or 5* as I would assume 5* meant perfect and there is the odd niggly thing sometimes which gets to me. So, really I would probably say 4.5 – but it takes me a while to try and even get to that. Likewise, I have a pair of bluetooth headphones that are pretty bad but not terrible – do they get 1* or 2* or nothing or something in between?!

    One of the things I remember from asking people survey questions was that the most important details we asked were whether someone would recommend that product to a friend. Ultimately if you can get them to do that then they are brand advocates and you can safely say the product is a good one from their perspective. If they don’t have anything to say and/or would not recommend it – then you don’t put your name to it. Easy.

  • » Product reviews: are they really useful? ____nicolas leroy___ writes:
    January 1st, 2007 at 7:53 pm

    [...] The first article – published on the Crowdstorm blog – confirmed the opinion I had on reviews: they are great for SEO as they bring some UGC (user-generated content) to shopping sites, but are not that useful to choose a product. I had a natural thinking that consumers are eager to post reviews when they are not happy or post too positive reviews to be really useful. The article from Crowdstorm explored the “ratings J Curve”, a pattern that has been recently observed and identified by Bazaar Blog: [Crowdstorm] We’ve always said here at Crowdstorm HQ that rating products or doing reviews involving five stars is a waste of time. Now there is actual evidence coming out to support our hypothesis, starting with this post from the Bazaar Blog, entitled “the ratings J Curve”. What these guys are saying is that they find the average rating always ends up around 4.3 and that there are more 1’s than 2’s and far more 4’s and 5’s than the lower ratings. Their chart implies that people are more likely to share positive experiences and often go towards the 4 or the five at the end of the scale. [...]



Leave a Reply

»  Substance: WordPress   »  Style: Ahren Ahimsa