SFF Network has been fortunate enough to be able to do a few product reviews since we started the site earlier in the year. In fact, we’ve actually been doing a lot more component testing in the past month or so, with a pile of forthcoming reviews now in the pipeline, and a few of these already published.
To be honest, though, this push of reviews is somewhat unexpected for us, especially since we originally planned on solely reviewing products of interest that we bought personally (or just happened to come across). That was certainly enough to get us going up to now, content-wise, but more recently we’ve actually started to hear from some companies, and reach out to others, as traffic to the news site and forum have both grown. From this, we’ve now found ourselves at a point where we’re already receiving samples, provided from various companies, specifically for review purposes – a very exciting development, as it enables us to publish component reviews that we otherwise wouldn’t be able to.
Anyway, I mention all of this because, as is the case for many technology websites, our reviews have quickly become an essential part of the content we publish, and drive a lot of the traffic and interest in our site…
…And yet, ever since I can remember, I’ve held a strong, deep-seeded dissatisfaction with the way the great majority of product reviews are done today in industry, with ours being no exception. Too often, product reviews are strongly tinted with the preference and priorities of the reviewer, clouding how well a component might work for my own purposes. Most of the time, they don’t tend to be critical enough, glossing over seemingly minor points that actually matter a great deal to some. Frequently, in fact, this is underscored by a lack of quantitative data to answer my questions, though more and more I’ve seen the opposite problem – a glut of charts and tables that completely bury the testing data that’s actually relevant. And, worst of all, the very format of reviews, and their desire to distill themselves into rating scales and other arbitrary systems, doesn’t help at all when trying to make comparisons, and often makes the process more difficult, not less.
I get the sense that I’m not alone in feeling this disappointment, either, even if most people don’t think about or vocalize that feeling very much. But, perhaps most importantly, as an editor and contributor to a fledgling publication, I’ve found myself feeling somewhat responsible for voicing my thoughts, now that I’m in a position to do something about it. It’s easy to complain as a seemingly powerless reader, but now I have no excuse – if I’m not critically evaluating how we make reviews here at SFF Network, any silence on my part is effectively an endorsement of how we do things. And I don’t endorse how practically any publication or individual does reviews right now.
Up to today, my involvement in generating any of SFFN’s reviews has been limited mostly to my day-to-day editing duties. But I feel compelled to capitalize on our opportunity to publish, and have felt for a while now that it would be an interesting and beneficial exercise to really think through how product reviews in the technology space are created – from format and testing methodologies, to ethics and writing styles. Consequently, over the next week or two, I’ll be doing exactly that: writing a series of pieces that will look at what product reviews are meant to do, how the ones created today fail, and how we can possibly – just possibly – do better, for those who seek guidance in building their systems.
For years, I’ve tried to imagine what the “ideal” review might look like – the sort of review that can be the most helpful to the greatest number of readers – and I think I’m at a point where I can begin to flesh that out. At least, with the help and input of our community at SFF Forum.
Everything I have to write about this wouldn’t possibly be able to fit within a single post or article, so instead I’ll be putting up my thoughts in a multi-part series, with updates every few days or so, starting with this one. They will originate on a dedicated thread on SFF Forum, so as to encourage discussion and dialogue, but I will publish each part of the series to SFF Network as well, for those who’d rather just read along.
The progression of these parts will look something like this:
1. Introduction (this post)
2.1 “What is a review?” (What are reviews meant to do?)
2.2 “The Problem” (What’s wrong with technology reviews today?)
3.1 “The Deep Dive” (Analysis & critique of common review formats)
3.2 “Lessons Learned” (A high-level summary of what’s generally right and wrong with existing product reviews)
4.1 “Building a Better Review” (Rolling up all the previous sections, outlining a format and style of review that seeks to address the problems with more traditional reviews)
4.2 “Looking ahead” (How SFFN will incorporate these ideas, taking in feedback, etc.)
During the time these go live on the forum thread, I encourage all readers and community members to give feedback, and engage in some community discussion with respect to product reviews and related topics. As I continue to put thoughts into words, I’ll be participating in that discussion, and getting a better sense of some of the positive experiences and frustrations that you all have with reviews, that I may not be thinking about.
Also, it’s worth mentioning that, while SFFN’s reviews will themselves change in response to this work (and your feedback), we won’t be holding back on publishing reviews we’re already working on, so don’t worry about a dearth of content 
We may retroactively edit past reviews to whatever new format we implement down the line, but for the most part any changes in writing or process are meant to be forward-looking. At this point, having this discussion and bringing about these changes to our review process is more a personal pursuit of mine than anything else.
To conclude, this topic may seem like an unusual one to spend a lot of time and effort thinking over, but to me it is one of the most important parts of tech journalism. Product reviews by publications – especially large ones – hold incredible influence and sway, and have the capacity to empower millions of users to make better buying decisions, in a way that tangibly improves their lives. Us as individuals couldn’t possibly find the time, energy and resources to thoroughly vet a field of competing products, and discern which is best for us specifically, but journalistic entities can do this for us, and this has a huge impact on all of us as enthusiasts. It makes or breaks companies and products, and it sets the standards that those companies compete against for our dollars.
Simply put, product reviews have the potential to give us, as consumers, direction. And I want to see how far that we, at SFF Network, can really take that.
Hopefully, across this editorial series, we’ll begin to find out.
