Excuses, Excuses

Yes, participation in reviews--especially comparative reviews that pit vendors' products against those of their competitors--carries risks. "A bad review can cast a shadow over not just the product but the company as a whole," vendors tell us. "It's not worth the risk." The reality, though, is that our lab tests mirror the pounding their products will take in your real-world environment. Wouldn't a vendor rather find out about and address potentially fatal flaws before readers buy its products?

Another vendor roadblock we often hit: "What if you don't test the features we've worked so hard to improve?" We develop our test plans for each review months ahead of the publication date, and we share the gist of those test plans with vendors so they know what they're getting into--no surprises. We're also starting to share our test plans and other article information online, giving you an opportunity to participate in our planning (see blog.networkcomputing.com/webserv.html for information about our upcoming Web services platform review).

"But what if you discover a problem with the product that we didn't know about?" We don't show vendors our test results and reviews before an article is published, but we do give them a heads up if we find a problem they can readily solve. We're not out to get them--we're just trying to help readers make wise choices about buying and implementing technology. If a flaw is identified and fixed before readers have to ante up, then we've done those readers (and the vendors) a valuable service.

"OK, but what if you make a mistake?" Well, we're not perfect, and our tests are complex, so we'll acknowledge that's a risk--but it's a mitigated risk. We have a staff of full-time and freelance writers, all current or former IT professionals, who live, eat and breathe technology in our Real-World Labs®, and we have more than 12 years' experience successfully reviewing products and services for publication. We're not bragging (well, maybe just a little) when we say nobody does it better.

Last but not least, there's the "We can't afford to give you our product for an extended period of time" response. Granted, the resource issue is very real, especially when it comes to major hardware products (10 Gigabit Ethernet switches, for instance). Vendors need to keep enough products in the pipeline for potential customers, and for trade-show exhibits and customer demos. Vendors also need to make a product manager and tech support people available to respond to installation and integration problems that might arise during our tests.

Still, those are costs well worth incurring. Better for us to tell our readers "We had a problem with XYZ but the vendor's tech support people responded quickly and effectively and now we can tell you how to avoid that pitfall," right?

Bottom line, vendors take a bigger risk by opting out of our reviews than by opting in. They stand to alienate potential customers by their absence, and that's bad for business.

How can you help your vendors see the light? Show them this column, and tell them you want to see their products reviewed alongside those of their competitors. Let them know you need more information--not less--to make a smart buy, especially in an emerging product category. Then, stand your ground.

Mike Lee; mlee@nwc.com