HarperCollins, PRH Adopt the Flawed Lexile Book Rating System
Pop Quiz: Would you trust a book rating system that told you that Sports Illustrated for Kids' Awesome Athletes! was a more complex read than Jane Eyre, Huckleberry Finn, or To Kill a Mockingbird?
If the answer is no, then congratulations. You’ve just proven yourself smarter than many US publishers.
A new press release crossed my desk today which shared the news that a number of publishers, including HarperCollins and Penguin Random House, will soon be joining Macmillan, S&S, Scholastic, etc in providing Lexile scores for their children’s titles.
Originally invented in the 1980s, Lexile is a system for assigning a numerical reading score to books. It’s pitched as a better alternative to the more holistic reading level assessment historically used by teachers and librarians which replaces an expert’s opinion with a mechanical calculation. Lexile’s creator, MetaMetrics, uses a proprietary algorithm which analyzes sentence length and vocabulary of the text of a book and come up with a number between zero and 1,600.
The scores can be matched up to grade levels, more or less:
Having been adopted as part of the Common Core standards, Lexile is having its day in the sun. Publishers are adopting it left and right, schools are using it to buy books, and everything is peachy.
Or at least it would be, if not for the fact that Lexile is – at best – flawed.
While it is indeed possible to guesstimate the reading level of a text (MSWord and other office apps have a similar feature) the algorithm used to generate a Lexile score frequently results in scores that simply don’t make any sense.
For example, the Lexile score for Sports Illustrated is actually higher than the one for Jane Eyre. With a score of only 870, To Kill a Mockingbird is only a fourth-grade read, while a children’s book like Mr. Popper’s Penguins (with a respectable Lexile score of 910) is deemed more complex. Raymond Carver’s Cathedral, on the other hand, scored a weak 590, about the same as Curious George Gets a Medal.
The New Republic reported on Lexile last fall, and they posted this handy graphic which illustrates some of the more questionable scores.
I confess that I have not read most of the books listed, so I can’t comment on their difficulty, but any system that would rate the first volume of Lord of the Rings as being less complex than Mr Poppers Penguins is not one I would trust.
And I’m not the only one; even the creators of the Common Core admit that Lexile is flawed:
As the Common Core Standards Initiative officially puts it, “until widely available quantitative tools can better account for factors recognized as making such texts challenging, including multiple levels of meaning and mature themes, preference should likely be given to qualitative measures of text complexity when evaluating narrative fiction intended for students in grade 6 and above.” But even here, the final goal is a more complex algorithm; qualitative measurement fills in as a flawed stopgap.
And yet it’s still being adopted widely and "recognized as the global standard for matching readers with texts", which is not at all an exaggeration.
While researching this story I found many graphics which explain how to use Lexile scores to match books up with the various grade levels.
Even Scholastic posted one.
I’m sure you’ve heard of New Math, right? Welcome to New Reading.
To be fair to the publishers, they might only be adopting Lexile scores because teachers and librarians asked for the scores. And some of the teachers and librarians might not have a choice; use of the scores might be mandated by their school board or principal.
But even if that is the case, I would much rather see the publishers explain why they weren’t using Lexile by pointing out the issues to everyone who asked. If they really see themselves as the bulwark of book culture then this is exactly the kind of thing that they should be defending against.
image by JD Hancock
fjtorres November 3, 2014 um 3:54 pm
Just for fun, I looked up Asimov: his books score in the low 800’s. 790 for THE GODS THEMSELVES.
Which, on the one hand is obviously a reflection of his lean accessible prose but…
Come on! You don’t give THE GODS THEMSELVES to a fourth grader!
Nate Hoffelder November 3, 2014 um 4:04 pm
Wow, I struggled with that book in high school. I honestly can’t tell you if I finished it or not.
fjtorres November 3, 2014 um 4:11 pm
It’s a great book, arguably his best, but it is not a trivial read.
Any book that starts at chapter four, switches mid-stream to another universe, and deals with variable laws of physics *needs* lean prose but that does not make it a kids book.
It is just plain silly to judge a book by its accessibility.
Nate Hoffelder November 3, 2014 um 4:11 pm
You can also give the first volume of LotR to a 5th grader. It scored 860:
fjtorres November 3, 2014 um 4:25 pm
Do that and child protective services will come knocking on your door. 😉
I was 16 when I read LOTR…twice in one week.
Haven’t been able to stay awake since…
Oyster Goes Social for Discovery | Digital Book World November 4, 2014 um 8:05 am
[…] titles. The system organizes content by reading level according to its complexity (though opinion is divided as to how well). Related: Learn How the Educational Standards Are Reshaping Children’s Publishing […]
Lexile scores | Making Book March 15, 2018 um 10:47 am
[…] length and vocabulary, which is what Lexile measure, though they are quite up-front about this. The Digital Reader calls the whole thing into question because of some admittedly fairly surprising results. Still a […]