|

Product Ratings Are Overrated

I read a lot of reviews–electronics reviews, full-size car reviews, off-road product reviews, fishing gear reviews and, of course, RC reviews. When I write a review for RC Truck Stop, I often think of many of the different reviews I’ve read. Some are examples of getting it right and others are examples of, well, missing the mark. I like to think that a RC Truck Stop review is as good as you’ll find in this segment, but that isn’t what this Commentary is about.

Keep it Under Your H.A.T.
All you need to know about RC Truck Stop reviews is that they cover a vehicle or product like a hat–or I should say as a H.A.T. And, that is because our reviews are Honest, Accurate and Thorough. That is our simple philosophy behind creating truly professional reviews. We’re honest with our opinion. And, not just kinda sorta completely sugar-coated “honest.” We tell the truth as we see it. We do due diligence, fact checking and this thing called research so that our reviews are accurate. We write articles with our audience and the product’s intended user in mind, and we thus make sure we cover every important aspect so that the review is thorough. We encourage everyone who writes reviews–for us or anywhere–to use the H.A.T. standard.

One hot topic of debate I’ve had with my peers has been the use of ratings. You may have noticed that we currently don’t use ratings in our reviews. Here’s why:

The most common rating system is a numerical system such as 1 to 5 or 1 to 10. The higher the number, the better score. But, doesn’t matter if it’s stars, check marks, grades–it simply doesn’t work for RC. Let me explain. Because the writer isn’t there with the reader to put ratings in context, they are actually more confusing than helpful. Okay, let me explain some more. Say an author tests a monster truck–the latest-and-greatest monster truck. Because it’s the latest and greatest, he gives it a 10 for jumping, and it probably deserves it. Here come the complications. Let’s say the next vehicle reviewed is a sport-level 1/8-scale truggy. It’s not a full-race pro-spec truggy, so the reviewer gives it a 9 out of 10 for jumping. So, does the monster truck jump better than the truggy? It got a 10 compared to the truggy’s 9. Nope. Even a sport truggy will jump better than a monster truck–they just do. So, the ratings don’t compare well from vehicle type to vehicle type. Think about it. Even the greatest short course truck is, at best, a 7 compared to a stadium truck. It gets even more complicated. What happens when the next latest-and-greatest monster truck comes out and it jumps even better the previously tested? Is it an 11? The Spinal Tap of monster trucks? Ratings almost instantly become obsolete in RC because the technology changes so fast. The last generation of 10’s is this generations 5’s. Now, consider the often-rated durability. I scratch my head when a vehicle is given less than a perfect score and no breaks, parts failures or excessive wear is reported in the review. I know the author thinks he’s providing a “real” review, but it makes no sense. In addition, how does one 9 for durability compare to another 9 since vehicles aren’t tested in the exact same manner? The point is it is next to impossible to really come up with a rating system that is fair and actually informative. In RC, ratings are given with good intentions, but the reality is the “information” is pretty useless.

What is the solution? The key is to define expectations and report on how the vehicle meets or does not meet those expectations. That puts everything in context. The expectations are based on the intended audience and properly formed via the reviewers unbiased position and experience.

When are ratings valuable? Ratings make perfect sense and are very useful when doing shootout or guide type articles. If we, at RC Truck Stop, were doing a shootout between three 4WD short course trucks, we’d certainly include a ratings chart. Because the ratings are only in the context of that hypothetical article and that testing, it works.

For the record, I give this article a 7 out of 10.

Similar Posts

8 Comments

  1. Matt, there was a reason why you were the EIC of the best RC magazine out there. Don’t know why you left, don’t care. You… are the reason why I have RC TRUCK STOP liked on Facebook, and bookmarked on my computer. Keep up the great work.

  2. While I almost always check out the charts and read the captions first, you might have changed my opinion. What you say makes sense, but I think there is a way to include ratings even for R/C

  3. The first thing I look at in a review is the ratings, but I see what your saying and think your right. With a new vehicle coming out or new versions of vehicles coming out monthly the ratings dont make to much sense.

Leave a Reply

Your email address will not be published.