How do you rate user ratings?

… doesn’t have the best rating score, but it has the best rating responses so should be more accurate.

I’m puzzled by the shopping lists ranking views.  For example, at I searched for a product, got the list, then I set the display to order by “Best Rating”.    Now I get a bunch of stuff and the top two items are

Item# #Reviews #Excellent #Good #Average #Poor #Very Poor
foo-1 2 1 1 0 0 0
foo-2 171 112 26 7 9 1

Doesn’t that seem odd? The first item is listed first, yet it only has two reviews, whereas the second item has a lot more reviews. True, the first item has no negative reviews.  Is that why it’s listed first?   Doesn’t sound correct to me.

I searched for an explanation on the site, but did not find one.   Yet, I don’t see an alternative.  My gut feeling is that the second one should be first.  It doesn’t have the best rating score, but it has the best rating responses so should be more accurate.  Isn’t this covered in Statistics 101?

I’m sure there are nice algorithms or frameworks to make this more useful.  Then again, maybe not. I’ve searched and I don’t find any definitive answers. Yet, there should be. How do people rate user ratings? Gut feel only?

Another example, I searched for a book on Amazon, the new Lee Child’s “A Wanted Man”. The user ratings were:

1 star 2 star 3 star 4 star 5 star
actual 174 184 268 312 462
Normalized 12.4 13.1 19.1 22.3 33

Just based on the ratings score, without reading the feedback, is this a “good” book? Note, here are a few statistics measures, though be wary, my statistics 101 was not recent:

Avg median mode var stddev
20 19 5 star 69.3 8.36

Using the SurveyMonkey computation we get a Rating Average of 3.5. This is a 4 Star rating. Seems the Survey Monkey approach is to the use the vote count of each each cardinal star as a weight. This gives an expected value computation.

Here are a few references on this that I hope to read one day:

Further reading

One thought on “How do you rate user ratings?”

Leave a Reply

Your email address will not be published. Required fields are marked *