1.09.2010

Allow me to express my doubts, and why you can't trust all the reviews you read (Discussion post)


I was browsing through new releases on Amazon the other day. In particular, I looked up a few books that I'd seen around the blogosphere that had peaked my interest.

I read a few reviews on Amazon, looked at the ratings.

And I got to one book's page and I was just like, "What is this?"

I won't mention the book name or author name because I don't think that'd be quite right, but something struck me as a bit off about the reviews for a particular book. I shall hence forth refer to it as FISHY BOOK (FB).

  • This FISHY BOOK has only been out for a 2 months or less.
  • It already has around thirty reviews.
  • All of the reviews are five star ratings. Not a single 4-star, 3-star, 2-star, 1-star review in sight.
  • All of the reviews are by "REAL NAMES" reviewers.
  • And yet, comparably, FB it was ranked around 550,000 in sales.

Okay, so maybe I'm weird, but I notice things like this. And maybe I'm boring you but this thing is GNAWING at me and I don't do well with gnawing and I have to get this off of my chest.

It just doesn't add up.


At least when you compare it to stats for other books. The bolded stuff is what's important, you can skip to that if your bored by the stats.

Take Book #1:


  • Release Date: August 2009
  • Sales rank: ~ 42, 000
  • Number of reviews: 22 (5 star=7 ratings 4 star=6 ratings, 3 star=3 ratings, 2 star=4 ratings, 1 star=2 ratings.)

This Book #1 has perfectly legitimate statistics. It's exactly what I'd expect from a book that's selling pretty well (42,000 isn't too shabby), has been out for 5 months or so, and has merits.

This Book #1 is ranked about....500,000 HIGHER than the Fishy Book (FB) in sales, has been out 2 months LONGER, and yet has much less reviews.

This is the case for pretty much any book you look over and compare to FB.


If you don't believe me, just start digging around Amazon. Get a feel for how the number of reviews, the sales ranking, and the release date correlates for a book.

Take Book #2:

  • Release Date: October 2009
  • Sales Rank: 30,000
  • Number of reviews: 15 (five star=12, 4 star= 2, 1 star= 1)

So Book #2 was released at a similar time to FB, has nearly half the reviews, and about half a million better in sales ranking.

Only 15 reviews.

Obviously people like this book, the reviews are pretty much favorable.

The point is, I've NEVER in my life seen a book with double digit reviews that had every single one a 5-star rating. It just doesn't happen.

Take Book #3:


I found one that has around the same amount of reviews:

  • 33 reviewers for Book #3.
  • ~30 reviewers for FB

Out of 33 reviewers, only THREE had included their real name (9%). On the other hand, this Fishy Book (FB) had 100% reviewers with "real names"

So that doesn't add up either.

And finally, the most fishy thing of all: 97% of the reviewers for FB have never reviewed anything else. The only book they've ever reviewed is FB. Compare this to Book #3 where only TWO, or 6% of the reviewers had never reviewed anything else.

So, I don't know if my stats helped to show what's gnawing at me. I always, always, always, without fail look at sales ranks, number of reviews, and release date when I look up a book on Amazon. So I'm pretty attuned to what a book's stats should look like.

What I've been getting at this whole post:

I think there's something not right here with these reviews.

But I don't know if this is just a giant fluke. I could be completely and utterly wrong. I mean, Amazon does have some safeguards: you have to purchase an item before you can register for a reviewer account.

But to be honest, what I suspect is that the majority of reviews for FB were written by one person, or people that were specifically told to give five-star ratings. Also, once I'd read some of the reviews, it seemed like someone was trying FAR too hard to make them seem different--spelling errors, triple exclamation points. But the tone and sentence structure seemed very similar to me.

So the point of this post is that you need to be careful about reviews you read online by anonymous people. Any kind of review, just not book reviews.

I'm not saying book blogger reviews--that's a different matter. It's one thing to read a review on a reputable book blog and another thing to read a review by an anonymous person on Amazon or Ebay.

The other point is that I do very badly with gnawing suspicions.

Boring post? Probably, haha. Sorry about that.

But if you made it through, I'd love to hear your thoughts on this: do you think there's something fishy going on here? Or is it just a fluke?

And lastly, do you trust all the reviews you read? Why or why not?

24 comments:

Book Monster said...

Yah, I enjoy reading variety reviews of one book, then all the reviews being good of that one book. I think that one person did it to the FB.

pirate penguin said...

Interesting case you've got there o___o!

It DOES seem a little fishy but...it could also be true. But I can see why you'd be suspicious-it doesn't add up nicely. Maybe it IS a fluke, maybe it's not. As for the reviews I read, I don't ALWAYS trust them. I have more faith in the reviews I read from book bloggers. Also, it depends on taste too. I know what genres I like so if a blogger has a similar taste in reading, I'll feel more inclined to trust their opinion than someone whose taste runs in a different direction. But ultimately, it's my gut feeling about a book that makes me either want it/ignore it. The reviews help but don't make me decide. :)

Emilia Joyce Plater said...

Super interesting post :D

It does sound very fishy. Is there are a reporting system for these kinds of things? Ha. Well, in the end, hopefully any made-up reviews won't count for too much when it comes to sales. I'm someone who does look at Amazon reviews to see whether a book is good, but really, if they're all 5-star and nothing else, I'm not going to take them very seriously. I usually check out what the 3- and 4- stars have to say.

Allison said...

I agree--that does sound super fishy. I love reading reviews on Amazon, but I trust blogger reviews more. I've heard stuff before about people trying to promote books surreptitiously, so something like this doesn't surprise me. It does disappoint me, though. I'd love to be able to trust that everyone's being honest when they're reviewing books, but unfortunately, that's not always going to be the case.

Allison said...

Oh, one more thing. When I am reading reviews on Amazon (or any other website) I like to read the 5- and 4-star reviews first to see what the people who loved it say and then the 3-, 2-, and 1-star reviews to see what peoples' complaints are. I think it's pretty helpful to see what some people didn't like about the book in addition to everything that people loved. And it's especially interesting when you get contradicting reviews, one saying the writing is spectacular and one saying it's elementary, for example. Everyone's taste and opinion aren't the same, and that many all 5-star reviews is definitely suspicious.

ninefly said...

this isn't the first time Amazon has manipulated ranking scores with their own biases, which is why I tend not to read reviews on commercial websites (or at least not let them weigh heavily on my judgment)
plus, for some reason almost EVERY book now has a "bestseller" tag on their covers, it's like the "best" vs. "better" debate in advertising
I usually look to "community" websites, like goodreads, that have many reviews that span the spectrum of member opinions
another thing is that all reviews are completely subjective, I know for a fact that I may like a book that other people will hate, but I still rate it high for the reasons I put in my review
it's really a matter of finding a reviewer with similar tastes or who lists specific reasons (rather than the vague "it was a fun read"/"it rocked" ones) for their ratings, I guess

Alyssa said...

Great post, and not boring at all. Though now you've made me super curious what FB is.

As far as trusting reviews you read online... I think it depends on what it's reviewing. Things like books or movies could go either way for me. Those types of things at least one person will always hate it, and one person will love (although not everyone will post a review saying their opinion). Then again, with things like oh, digital cameras for example, I pay more attention to reviews. With equipment type stuff I find usually if you have bad reviews it's less opinion based and more based on the fact that the item (whatever it is) has issues/doesn't work/breaks easily/etc. So I think I trust those kind of reviews more. But on books? Frankly, I trust my own opinion above all others. Even if the book has over 100 people telling me to read it, if I don't think it sounds interesting I won't. Gut instint over group instint I guess.

Becky said...

Oooh, a thoroughly interesting and eye-opening post. It sounds very fishy. I certainly do not believe every review I read any more whether it is a book blog or Amazon one or whatever. Take my experience with Swoon. I read a few highly complimentary reviews. I ordered it. I read it. I hated it. Now I'm highly suscipious. I only buy on the recommendation a few book bloggers that I trust completely. I still think my Swoon experience could just be an anomaly though because it is still getting great reviews. it is just me that thinks it's rubbish.

Laura McMeeking said...

I enjoyed this post! I, too, am an avid review checker, and it does seem pretty odd that all the reviews were 5 starred. I might be able to reconcile that on a piece of electronic equipment or furniture, but even then, items usually have at least one 4 star if it's that great. This is the very reason why I read reviews on other sites and blogs. However, even then, I realize that reviews for books are extremely subjective, as I see when I recommend books to friends, and they hate them. :-) All this is to say, very interesting! I will now be looking at rankings, release dates, AND reviews!

Angie said...

There was actually a post like this on AW a while back - someone had noticed that a book had all five stars and the sales numbers were really low or something. Like you said most likely one person wrote them or maybe the author asked all their friends to give reviews for the book. Seems kind of disreputable that an author would do that, but I have heard of it happening before. I don't normally look at reviews on Amazon. I take book recommendations from friends, other writers, books that have gotten awards, etc.

WilowRaven said...

Personally, I do not trust Amazon reviews. Not all are fishy but I've heard enough to be suspicious. I seen reviews that have obviously been written by the author. I've known reviews who have had their review removed because it was unfavorable (1 or 2 stars). I've seen people, who seem to work for the publisher, review one of their own books.
It's a shame because there are a ton of great reviews out there - people who write honest reveiws - but Amazon has been tainted for me.
I rely on my fellow bloggers and book loving sites such as LibraryThing and GoodReads.
Not a boring post at all! Great topic!

Donna said...

The only reviews I trust are made by people with similar tastes to me. I'd have to read through a lot of reviews on a book to actually have them be comparable to the book. Like if a slew of reviews for a book is negative (from different people), then chances are that book isn't great so I'm going to skip it. If they're all positive, I'll wait to see what one of my trusted reviewers has to say before I pick it up. If they like it, chances are I will too.

As for Amazon, it looks like it's either one person that's taking it upon themselves to attempt to pump up a book which isn't uncommon. There are a lot of psychos like that out there, and they'll probably rip your throat out for saying anything less than stellar about the book their obsessing on. But it's not uncommon either for some authors to have people go out and review their books to bolster the looks of it either. If they'll buy their books in bulk in order to up their chances of landing on the NYT Bestseller list, then they'd be willing to load Amazon reviews as well.

Kirthi said...

Wow, you do mention something pretty fishy!

choco (In Which a Girl Reads) said...

@Book Monster: I agree, variety is better. And I'm leaning the one person (the author) having written all the reviews

@pirate: I do trust book blogger reviews a lot more than I do any other type too :)

@Emilia: Haha, I don't know if there's a reporting system. This book definitely tops the list of most suspicious. And I like to check out the 4 and 3 star as well :)

@ Allison: I like to read the whole spectrum of reviews as well. And contradicting reviews are really interesting. I do think this is a case of a person trying to promote their own book

@Alyssa: That's a great point! I do think equipment is a lot more objective. But at the same time I've heard about like...company employees writing reviews for products so I have to be wary of that as well.

beth said...

I am very selective of reviews on Amazon--I don't trust many of them; I usually just check out a couple of the 4 and 3 star reviews (I ignore most 5 and 1 star reviews).

But I DO read a lot of online blog reviews. There are a few blogs that if that reviewers recs the book, I will get it almost immediately. And there are a few blogs that I don't read any more because I just can't trust them. One blog in particular, the reviewer would give fairly decent reviews to most books, averaging around 80-90%...but for the five or so authors that the reviwer chatted with CONSTANTLY on Twitter, she always gave them 100% reviews. It was so obviously biased that I immediately quit following that blogger.

choco (In Which a Girl Reads) said...

@ninefly: That's a great point, I think it's mostly the commercial websites that you have to be wary of. Because who knows who's writing the reviews and what incentives they have? At least with book bloggers or goodreads you know that their opinion is valid, but for Amazon or something it's hard to pick out whether it's a falsified review.

@Becky: Oh, Swoon! I think I've seen lots of positive reviews for that book on the blogosphere, but this is one of the times Amazon helped because there were some negative reviews there to balance it out.So far I've had good experiences with book blogger recommendations. I usually try to make sure the majority of reviews I read of it are positive.

@Laura: I do agree about the subjectivity of book reviews. And definitely the all 5-stars was what set the alarm bells ringing for FB. And I'm glad you'll be looking at all three :)

@Angie: Wow! I didn't see that thread, I wish I had. Maybe it's the same book? Of course, it could be an entirely different book--I wouldn't be surprised if there are false reviews for a lot of other books as well. And I checked up this author's website, and from what I could read of them, he/she would top the list for authors who I could believe actually wrote their own reviews/got false reviews.

@Willow: I couldn't agree more. There was this period were I was pretty trustful of reviews on Amazon, but I think now I'm pretty cautious around them. I do think fellow book bloggers and friends are the best route to go for book recommendations :)

@Donna: Wow! I didn't know that sometimes they buy their books in bulk. That's crazy! But I guess I can believe anything. And I agree, there are some pyschos--I'm thinking of one thing I read in particular a week or so back about a author who went BALLISTIC over negative reviews on Amazon, calling the FBI and everything over it.

And I agree that trusting people with similar tastes (whether they like the same books you do) is a great route to go :)

@Kirthi: Haha, I'm glad you agree :D

Jenn (Books At Midnight) said...

Not boring at all; great post! And I definitely think something fishy is going around here, I've NEVER seen an all 5-star ratings book either. What I'm guessing is that it's either one person doing the reviews or the publicist/author paid people to give it 5-star reviews?! Crazy. But that's why I always look at the high ratings and low ratings when browsing on Amazon.

And I usually trust sites with a variety of ratings. I know some sites I read give good reviews on almost everything, but they're definitely not all 5-stars. Those sites I sort of trust, but I usually browse around a bit more to see others' opinions. I'm still sort of finding my way around, though if I know a blogger has similar taste as me, I will follow his/her recommendations. And you're one of them! :D

pepsivanilla said...

That is definitely fishy. Very interesting post!
I rarely, if ever agree with the consensus on a book's Amazon page, so I skim the reviews occasionally and then ignore them and buy the book if it sounds interesting to me :)

TerryLynnJohnson said...

Well, I certainly admire your tenacity at looking into this. Good for you. And thanks for bringing it to people's attention. I understand you don't want to name the book, but wish there was a way of calling these things out.

Justine said...

I don't really read the reviews on Amazon, I read them at Goodreads...but I'd have to say that this is odd. And your discovery on this has made me interested.

When I look for books to read, I go to the blogger who has the same taste as I do...and then I borrow the books from the library since I mostly don't buy them. I think that's the best way...:D

Cleverly Inked said...

I never trust those. Thats why their are people like you I can go to for a review

choco (In Which a Girl Reads) said...

@Beth: Oh no. I tend to trust blogger reviews, but it really hadn't occured to me that some bloggers who know authors are giving them higher ratings. But I don't really trust blogs that give every single book a high score. But that's definitely something to think about as a blogger; if you know the author is going to read the review how you're supposed to handle it.

@Jenn: Awesome, I'm flattered that you trust my recs :D I'm glad that you also think something's not right, the 5-star ratings must've been tampered with.

@Pepsi: I'm glad you found it interesting! Haha, I was worried it was boring. I think the skimming and ignoring is the right way to go if need be :)

@Terry: Thanks! I would name the book, it's just that in the possibility I'm wrong I could be making false accusations and I don't want to get involved in some sort of scandal or something, haha. I'm going to look into it though if I can report it anonymously. Because the false reviews really irk at me.

@Justine: I agree about going to a blogger with similar taste! And I try to utilize my library as well but I always, always end up getting fined, lol :)

@Cleverly: I'm glad that you're aware of it, and thanks a lot :)

Shannon O'Donnell said...

Not boring at all, Choco - fascinating actually. You really did your homework for this one. I use Amazon all the time and have never thought about this issue. You make some eye-opening points here. Great post! I'llnever look at reviews the same way. :-)

5peasinapod said...

I too read the lesser stars to see what they didn't like about it and if I would care about that issue or what ever it is that brought the rating down.

Post a Comment