I've been involved with Network Computing since its third issue. The engineering computing center on the University of Wisconsin in Madison campus was my old stomping grounds for a little over 10 years, so I know a little bit about real-world testing. While I was at the UW, we had Cisco write us custom patches for routers, we brought in print servers that couldn't even boot on our network, we worked with Novell to solve the issue of multiple frame types on a single network (it didn't initially work when NetWare 3.0 was released), and we worked for months with Novell to fix the initial versions of NetWare for Macintosh, and much, much more. If you think real-world testing isn't useful, then I respectfully submit the question, why do you think vendors beta-test their products?

In his column, Hall points out the repeata bility of the tests done in ZD labs. I t's a fair point and one that I agree with. The bugs that ZD labs misses on Monday are the same ones that they'll miss on Friday. It seems to me that perhaps there is a misconception about what we, and others, call real-world testing. First of all, it's worth noting that not a single one of Network Computing's technology editors has a journalism background. They all have engineering and information systems backgrounds. They've all worked in real computing centers and have supported real users. The value here is obvious: They can take testing to a higher level.

We run bench tests on virtually every product we evaluate. If we bring in an ATM switch, we ram packets down its throat until it pukes. It's an obvious first test to run. Large labs do this, too. We go the extra mile and put the product to work in a production environment. Indeed, what will happen when we do this isn't always known, but it's an important step. If the product uses TCP/IP, we really want to see what will happen when it meets a large IP -based network, like those found at a university. If it's an e-mail product, we want to know what happens when you try to migrate 10,000 users and their mailboxes into the system (a likely occurrence) at a large energy company. It's the unexpected result that makes the test interesting.

Even more important about the way Network Computing tests is that we do it in an environment where people have to live day-in and day-out with the types of products we're testing. They provide us with insights on what makes a good test, and they provide us with the environment to carry out those tests. Maybe you don't care that a server and NOS can add and delete 5,000-user accounts in a single day (the way a university might), but don't you feel a little better knowing that product was tested with more thought than just setting it up and shoving packets or users or e-mails or Web clicks down its throat?

In the end, I'll stack our Real-World Labs reviews up against anyone's in the indus try. Taking products into the real world is the value add that you've come to expect from Network Computing, and it's a task we take very seriously. If you just want benchtesting, then read the weeklies. If you want to know if a product works for real, read us. But you already knew that.

Art Wittmann can be reached at awittmann@nwc.com.