Skip to main content

Amazon and the Democratization of the Review

Paul Laity at The Guardian has a look at the recent paid review scandal on Amazon and what it might portend for the future of “traditional” reviews. Perhaps not surprisingly, even though the piece opens with a headline proposing that reader reviews might be “killing off the critic,” and even though it admits that only a tiny fraction of reader reviews are corrupted by payola or family-or-friendola, it concludes with a smug self-assuring pat on the back that, no, professional critics’ reviews are more reliable than Amazon reader reviews because they’re all professional and stuff.

I’ve seen this sort of thing before, in partisans of professional encyclopedias waxing smug because a study showed Wikipedia to have more factual errors per page than Encyclopedia Britannica. (Never mind that the Wikipedia articles were on average longer than Britannica’s so there were actually fewer errors per text length—and the errors were corrected as soon as they were noticed, while Britannica’s had to wait for the next edition.)

So Amazon reader reviews suffer from the tragedy of the commons. So what? Maybe they’re not up the standards of a “professional” review, but there sure are a lot more of them, including for thousands of self-published books most “professional” reviewers will never touch. Anyone with a lick of reading comprehension and discrimination will be able to separate the wheat from the chaff on his own anyway.

Frankly, I think people don’t give Amazon reviews enough credit. Really, we should be marveling at the democratization of the review. We don’t have to rely any more only on what the “experts” think. With Amazon, we can hear from anyone. Literally anyone, whether they bought the book at Amazon or not. Amazon doesn’t care if I bought it from them, or Barnes & Noble, or a used bookstore, or even haven’t bought it at all but want to say what I think about it.

You hear people calling from time to time for Amazon to disallow reviews from people who didn’t buy the product from them, but Amazon has so far been wise enough to realize that would be a suicidal move—just as Jeff Bezos was wise enough not to listen to the people who were horrified that the site allows negative reviews as well as positive. Much as people like to fear that it is, Amazon is not the only place an informed shopper might buy something. (Indeed, DeepDiscount is frequently cheaper than they on videos and offers free shipping with a lower minimum purchase besides.)

Restricting reviews only to people who bought something on Amazon would really cripple the ability of many products to get any reviews at all—and the more reviews any product gets, positive and negative, the more informed a decision any buyer will be able to make. The negative reviews give the positive ones credibility, and vice versa. In fact, one study suggested that Amazon reader reviews are, on average, as good as professional reviews.

And if some people leave one-star reviews of items they will never buy based on principle, who is to say such a review is any less valid than a five-star one of someone who liked it? The five-star reviewer is explaining why the work is valuable to him, and the one-star reviewer is explaining why the work is abhorrent to him. If it’s abhorrent based on something that leads him not actually to buy it, is that abhorrence really any less legitimate than the delight of someone who loves it? (Though, to be fair, there’s kind of a thin line between this sort of behavior and slacktivism.) Frequently the sort of people who leave one-star reviews don’t feel they can make their opinions felt effectively in any other venue. And anyone who actually bothers to read the reviews will soon be able to tell for himself which are “legitimate” or not.

When you think about it, where before Amazon was there any such review forum which attracted reviews by and reaches so many people? Even Laity admits that the majority of these reviews are most likely legitimate—and there are so many more of them than all the professional reviewers in the world could hope to publish. These reviews are a great resource for people looking at buying from Amazon, or for people looking at buying a particular item from a local shop or third party website. You can almost always find reviews for any work you want to consider, whereas professional reviews could range from fairly common to none at all. And if the reviews aren’t “professional,” they’re at least written by people who actually care about the product one way or another, rather than being paid to consider it.

Image from xkcd.

Similar Articles


Karl August 30, 2012 um 7:50 am

I’m a great believer in customers' reviews, but I also believe that professional critics can make enormously valuable and unique contributions. When they do their job right, they can put a book or other artwork into context. They can educate their readers about the field that the artwork falls within. They can teach their readers to think in a more informed way about the individual work being reviewed, and help them to understand the strengths and flaws of that work.

In short, a good critic is a professional educator, rather than just a random person reporting on their individual reactions to a book or whatever.

I heartily recommend Daniel Mendelsohn’s recent New Yorker essay, "A Critic’s Manifesto":

Conrad August 30, 2012 um 9:31 am

The problem is that the professional critic – professional not as in being paid for reviews but in "independent and critical" – is a very rare person as what passes for criticism in the major venues is as varied in quality and I would say less trustful on average than Amazon’s collective review judgement.

Money still talks and while the big players do not "buy" positive reviews as in the straight "I pay you 1000$ for a batch of 50 reviews", they buy them in subtler ways, so morally it’s like the chicken thief who goes to jail (here "jail" being the internet opprobrium on independent authors who tried to get exposure by straight out buying reviews) and the million dollar embezzler who is feted out by society as an expert (the big venues review departments)

So I completely agree with the article above and see all this Amazon review issue as a huge meh and wonder why people do not follow for example Salon’s story about corruption in the big venues more.

Guardian’s piece linked above has quite a few comments detailing other examples of perceived corruption (eg co-founder of famed literary prize gets own book praised to the skies in the Guardian and similar venues, people find it quite bad and so on)

MJ Jeffs August 30, 2012 um 11:48 am

An interesting perspective. In the spirit of the democratization of all things on the internet, I feel compelled to point out that you have incorrectly referenced the tragedy of the commons. Amazon is not a finite resource that is being depleted. Wikipedia defines the term quite nicely here:

michael mckee August 30, 2012 um 2:32 pm

What the recent news coverage on the subject of reviews misses entirely it the personal responsibility of the review reader. Yes, there are shill reviews, paid reviews, friend and family reviews. One aspect these share is that they are pretty obvious, at least to anybody who has taken the time to critically read reviews.

Reviews are subjective. That may seem obvious, but so few people seem to read reviews as if they are. Sure, there are some reviewers who’s opinion so often matches my own that I trust them. Two big name reviewers, David Pogue of the N.Y. Times and Walt Mossberg or the Wall Street Journal, get reportedly big salaries. IMO those salaries are deserved because the reviews these people offer are reliable.

So, who else is reliable? Read some reviews on Amazon and find out. Amazon offers enough information for us to make some informed guesses. First is the reviewer’s rank. Second is a listing of other reviews the reviewer has posted to Amazon. If a reviewer has only posted a couple of reviews, that might send up warning flags, especially of those reviews all gave 5 star ratings. Amazon lists the items that any reviewer has reviewed. Look at that before trusting a reviewer. That reviewer may be a shill or part of a group who have pledged to all rate up each other’s books.

Another red flag we should look for is how many five star rating a new book has and how early in the review list they occur. If a book gets a number of excellent ratings right after, or even before, publication, we can pretty much ignore those as marketing.

I reviewed on book on story plotting that I gave three stars. Mine was the 15th review and the first to offer fewer than five stars. I reviewed the book two weeks after publication. I received five immediate negative votes for my review. That raised red flags for me and should have for most people reading reviews. I offered what I felt were legitimate criticisms of the book’s content. I’m pretty sure that the author had a cabal of mutual support reviewers who tried to discredit my review. That’s not just vanity on my part saying so. I’ve received over 3,000 votes on my reviews and 95% of them are positive. That’s the only time I’ve ever received that many immediate negative votes for a review.

The moral from this is to look at the positive and negative votes a review gets. If the review seems to be honest and criticizes specific points, whether you agree with those points or not, then pay attention.

Specific points is another element to look for. A review that say’s, "Full of wisdom and humor," isn’t specific. It may or may not be true of a book. That’s beside the point. What is important is whether or not the review looks like it is specific to the book or not. Most reviews aren’t. That’s mostly because most people don’t go to the effort to analyze a book. That’s okay. Those kind of reviews are usually not polished or particularly well written. It’s the review that is slick, vague and sounds too good to be true that is probably a shill.

Does the review mention any negative points? No book is perfect. I recently reviewed a book by Tad Williams, that I noted seemed formulaic. The book hasn’t been released yet, so I won’t mention the title. I still gave the book four stars and noted that I very much enjoyed the story. Most legitimate reviews will note negatives, even in a generally positive review.

How did I get a pre-release novel? I’m an Amazon Vine program reviewer. I’m offering this as my credential to review reviewers. Being in the Vine Program means that twice a month I, along with all the other Vine reviewers, am offered a list of products to review. We get these products for free. In exchange for the stuff, we put up reviews on Amazon. I’ve recently reviewed a camera, a bottle of shampoo, Scrivener for Dummies, Kindle screen protectors, and deer repellent, among others.

How does one get to be a Vine reviewer? Amazon does not tell us their secret formula for choosing. It does seem that being a top 1000 reviewer for a while is part of it. The number of reviews and length of time reviewing are also probably important, as are the number of positive votes reviewers receive from other Amazon customers. And, having read reviews from other Vine reviewers, I’ve come to the opinion that, as a group, Vine reviewers tend to be both relatively impartial and accurate. If we look at reviewers or groups of reviews we can quickly learn to recognize people offering legitimate and skilled reviews, too.

Bottom line. Gleaning useful and credible information from reviews is a skill that can be learned. It isn’t difficult, but, as with any skill, requires some attention and practice. People who don’t bother acquiring those skills will waste money on books and other products.

As a Vine reviewer I periodically get offers of books to review. I don’t accept any unsolicited books for review. Please don’t ask.

Stumbling Over Chaos :: Linkity and earworms for everyone! August 31, 2012 um 3:02 am

[…] “Amazon and the democratization of the review.” […]

Write a Comment