As a subscriber to The New Yorker, I often find myself thinking about several of the pieces that they publish. One just came out this week in the February 14-21 issue, called "The Order of Things." Written by Malcolm Gladwell, the article starts off by noting how ranking systems such as that employed by Car and Driver can distort data when certain variants are either not taken into account or have too much weight placed upon them. After establishing this, Gladwell then proceeds to tackle the infamous U.S. News and World Report College Rankings that is published annually. Viewed as a sort of benchmark, this ranking system is often cited by university promotional staff as reasons why their college ought to be chosen over competitors, despite the same sort of distortions that take place within the data collation and analysis.
Gladwell's article reminds me yet once again why book review-related systems, whether they be Amazon star ratings or what Goodreads employs or even those 5, 10, or even 40 point scales utilized by some online reviewers, ought to be taken with quite a few grains of salt. One size-fits-all systems are ill-suited for heterogeneous entities, whether they be types of cars, university core subject foci, etc. This is especially true for books, whose elements often do not fit into nice pre-fab squares (one only had to look at the interminable debates over how to define "fantasy" and "science fiction," not to mention other genres and subgenres to see the futility on display there).
It has always baffled me to see many readers comment that they value ranking systems, considering how obviously flawed they are going to be. Can one rate, for example, a work by Rabelais with one by Sartre? Or a book written by Gene Wolfe compared to one by Karen Tei Yamashita? Should one even attempt to "rank" at all when doing a review?
I would argue that due to the wide variance between stories that having anything approaching a single-entity review system, particularly that which employs "rankings," is going to be a pointless exercise that detracts from other, more important aspects of reviews, which deal with the reviewer's interactions with the text and how well engaged that reader was with that particular text. Comparing Umberto Eco to George R.R. Martin makes as much sense as comparing a piranha to a cobra. Both might bite you in the ass and kill you, but the methodologies of each are too dissimilar to rank one higher or lower than the other.
Gladwell's article reminds me yet once again why book review-related systems, whether they be Amazon star ratings or what Goodreads employs or even those 5, 10, or even 40 point scales utilized by some online reviewers, ought to be taken with quite a few grains of salt. One size-fits-all systems are ill-suited for heterogeneous entities, whether they be types of cars, university core subject foci, etc. This is especially true for books, whose elements often do not fit into nice pre-fab squares (one only had to look at the interminable debates over how to define "fantasy" and "science fiction," not to mention other genres and subgenres to see the futility on display there).
It has always baffled me to see many readers comment that they value ranking systems, considering how obviously flawed they are going to be. Can one rate, for example, a work by Rabelais with one by Sartre? Or a book written by Gene Wolfe compared to one by Karen Tei Yamashita? Should one even attempt to "rank" at all when doing a review?
I would argue that due to the wide variance between stories that having anything approaching a single-entity review system, particularly that which employs "rankings," is going to be a pointless exercise that detracts from other, more important aspects of reviews, which deal with the reviewer's interactions with the text and how well engaged that reader was with that particular text. Comparing Umberto Eco to George R.R. Martin makes as much sense as comparing a piranha to a cobra. Both might bite you in the ass and kill you, but the methodologies of each are too dissimilar to rank one higher or lower than the other.
Posts
7 comments:
It's only ever going to be an approximate guide, which is why I never bother with half-stars and so on. But they are useful as a shorthand, so people have a rough idea what kind of review to expect.
Nobody expects you to be able to compare Sartre with Rabelais on a numerical scale, but crap writing exists.
Neither a piranha or a cobra have a purpose, they are the blind product of evolution. Books by Umberto Eco and George R.R. Martin, on the other hand, are the product of their authors and have a deliberate purpose. And I don't see what the problem is with comparing to things of the same type with deliberate purposes, even if those purposes are radically different. I agree with you on the unhelpfulness of ratings. One can certainly rate a work by Rabelais with one by Sartre - but why would you want to?
As an aside, I think it is interesting to compare literature (where ratings are relatively new) to film (where ratings are well established). The genre divide is a lot less noticeable in cinema and critics happily deploy the star system against Akira Kurosawa and Rob Schneider alike.
That's why I don't rate. You're either rating against an arbitrary bar of perfection (one that only you can see) and will skew appropriately, or...
Or you are rating a book or movie against how well it accomplished what you think it set out to do and how it compares to similar novels or films. In that way could you reasonably rate, say, Bring It On 4 or 5 stars and Yojimbo only 3 or 4.
Either way, it's fictitious.
I don't think it's ridiculous to rate commercial fiction; you just need to determine How Well It's Doing What It's Supposed to Be Doing. Nonetheless... I'd rather an in depth analysis than stars. And there's always going to be a lot of books for which a ranking system will prove worthless.
On a related note, there's still that funny story about the game reviewer who hated Deus Ex: http://www.rockpapershotgun.com/2010/06/29/tom-chick-the-man-who-hated-deus-ex/
I find the idea of scoring any creative effort a gross oversimplification. I do not add scores to my reviews for that reason. If you need a quick summation then simply read the last paragraph of a review and that should provide the highlights.
The best anecdote I can provide is Rotten Tomatoes. Due to the number of reviews, I can pull out enough common threads to decide if I want to watch a movie. Even if the movie is scored low, the context of all the reviews is a better estimate of my prospective enjoyment than the rendered score.
So, what I am trying to say is that every reviewer has a different perspective. Few match my own. But a well written review will let me make up my own mind. Blindly following a trite review score would never give me this insight.
I really like ratings. I think they're a good way to sum up your thoughts in a nutshell. Do I take that and compare it to other works reviewers have "rated"? Not always, unless it's actually similar. You really have to take it all in, the rating plus the review for the rating to actually have merit.
I also know I'm pretty lazy and don't always read every single review unless it's a book I'm really interested in. So, a rating for me says, here's a ballpark for what this one, single reviewer thinks. Whether they had a good time or not so much. Not that this book is definitely .000043 stars better than the last.
I've stopped visiting most book-reviewing-only blogs in the past few years, so I don't know how prominent this kind of ranking is...but I absolutely agree that it's silly to place much (any) emphasis on this kind of a system.
I do like Amazon's stars for other products--tech hardware or toys for my kids or whatever. I'll check out both the 5-star and 1-star ratings especially to get a sense of the product's quality. But for books? 1-star reviews of a book I enjoyed can be entertaining in a way, but I'm not gonna choose what book to read or not based on an average star rating (nor other random number systems out there).
Post a Comment