Amazon has been fighting fake reviews since at least 2012. They have deleted scads of reviews, banned paid reviews, filed suit against several batches of fake review perpetrators, and even forbidden authors from having any type of relationship with reviewers.
And it’s still not enough. Fake reviews are still being posted, mostly due to an underlying problem.
Buzzfeed published a story earlier this week that delved into the dark underbelly of the paid review ecosystem that has sprung up to manipulate rankings on Amazon. This massive system continues to operate despite Amazon’s best efforts:
The systems that create fraudulent reviews are a complicated web of subreddits, invite-only Slack channels, private Discord servers, and closed Facebook groups, but the incentives are simple: Being a five-star product is crucial to selling inventory at scale in Amazon’s intensely competitive marketplace — so crucial that merchants are willing to pay thousands of people to review their products positively.
To be fair, Amazon isn’t the only retailer with a fake review problem; earlier this week Valve announced it had permanently banned a game developer from the Steam marketplace after finding out that developer’s employee had posted fake reviews.
This problem exists in all marketplaces, and not just Amazon, but Amazon is the only retailer who has a reputation for banning innocent reviewers, thereby punishing innocent authors.
Right about the time that Buzzfeed was publishing its article, I was reading FB post after FB post from authors whose reviewers were getting banned from posting reviews.
Elena Page wrote about how she lost one of her first fans: "Amazon emailed her last week and accused her of leaving biased reviews on Amazon. She was banned from ever leaving reviews on Amazon again, and told she was not allowed to argue her case or refute the claim in any way. All the reviews she’s ever left, have been removed."
Other authors chimed in with similar reports, including one who noted that their books had lost most of the reviews even though none of the reviews were fraudulent.
Amazon has an automation problem. This company uses bots to run almost every aspect of the Kindle Store from detecting fake reviews to checking the formatting in ebooks and finding fraud in Kindle Unlimited, but the bots don’t work very well. This leads to situations where innocent authors are punished because Amazon’s bots think the authors are scamming Kindle Unlimited while the actual scammers continue to operate at a massive scale.
The under-performing bots are also in part to blame for the ongoing problem with fake reviews on Amazon.com.
The simple fact is Amazon can’t tell which reviews are fake, and which are forbidden by policy, and they’re not the only ones.
A lot of the recent stories about fake reviews, including the Washington Post article from last month and Buzzfeed’s piece, have relied on a company called ReviewMeta. This one of two startups that have developed algorithms to detect fake reviews. The other is called Fakespot, which has its own algorithms that work in essentially the same way.
These two companies will tell you that they can identify which reviews are fake with a high degree of certainty, but the thing is, they rarely agree and that raises serious questions as to their accuracy.
After the Washington Post reported on the fake review problem last month, David Gaughran followed up with a post that pointed out how ReviewMeta falsely labeled his books as having fake reviews.
Okay, this doesn’t look good. And if you look down the page it shows each of the products they have assessed that led to this overall brand trustworthiness score. You can see many of my books have “failed” in the eyes of ReviewMeta and “Unnatural reviews detected” has been appended to several of my books. Crikey.
You can click on each product and see how it came to that determination, and the supposed evidence for each component of that decision. Again, I stress, this transparency is truly commendable.
But this breakdown also reveals the faulty assumptions that led to these incorrect determinations about my reviews. And it’s not just my reviews, of course. These simplistic calculations affect most authors. (You can search for your own books here.)
You can find ReviewMeta’s analysis of David here. While you are at it, you might also want to take a look at Fakespot’s analysis.
The thing about these two companies is that they identified some of David’s books as having fake reviews, but not all. The other thing about these two companies is that they didn’t agree on which books had fake reviews.
One thought Let’s Get Digital was tainted, while the other passed the book. The same goes for Liberty Boy, and other titles.
The disputed analyses call into question the very idea that fake reviews can be identified by algorithms. As David put it:
The sad thing about all of this is that Amazon does have a fake review problem, one which is compounded by Amazon deploying a fake review detection algorithm that seems about as accurate as the one from ReviewMeta, perhaps for similar reasons too. Which means that authors innocent of any wrongdoing get genuine, organic reviews from bona fide reviewers removed every day and the scammers and cheaters with fake reviews keep getting away with it.
This problem is only going to get worse before it gets better.
