InformationWeek 500 Conference -- September 14-16, 2008 Registed Today!

IMMERSE YOURSELF:

ON THE WIRE

Solving Those Hard(ware) Problems

by Bill Alderson and J. Scott Haugdahl
Q: Our network architecture generally can be described as severalrouted Ethernet segments spread across multiple buildings within a cityblock. There is usually one or two segments per floor, and most groups havetheir own departmental NetWare server. For some reason, a few of the departmentalservers are noticeably slower than others, despite having the same hardwareplatform and operating system. There are about 100 to 150 users per segment.Strangely, it does not seem to matter how many users are active on a "bad"segment. Even off-hours usage with a few active users is slow. Our Category5 wiring passes all cable tests with flying colors. Very few collisionsor CRC errors are seen on any of the segments. We've tried reloading theoperating system and even isolating the segment to just the server and asingle user, but to no avail. Help!

Bill: The last time I saw a wiring system pass with flying colors, I couldn'thelp but notice that the wiring closet contained an eye-popping assortmentof red, bright yellow, blue, orange and green Category 5 cable.

Scott: Sure beats that boring gray color.

Bill: Our customer seemed to have checked out everything in the networkat the bottom and the top, but what about that stuff in the middle?

Scott: So, once again, it was time to break out the trusty ol' protocolanalyzer.

Bill: Going into the analysis, we had no idea if the slowdown was hardwareor software related.

Scott: After capturing and analyz ing a few traces, it didn't take long tosee that a workstation would frequently ask the file server for the samepiece of information two or more times in a row before getting a responseback fro m the server.

Bill: Since there weren't any replies from the file server when this happened,we classified this problem in the general category of a transport retransmission.(With NetWare, transport retransmissions are actually handled by the applicationlayer.)

Scott: Now there are a zillion reasons why we could be experiencing transportretransmissions.

Bill: Kind of like trying to figure out the ambiguous "Abort, Retry,Ignore?" message whenever we encounter a DOS problem.

Scott: Fortunately, we did have the benefit that the server was on the samesegment as the users, thus localizing the problem.

Bill: The workstation packets in question were transmitted without error,so we wondered why the server never saw them, even with only one activeuser.

Scott: Since every 10BASE-T port is a repeater, a device plugged into ahub port is really attached to its own repeated segment. So one thoughtwas to plug the analyzer into the same segment as the server.

B ill: This would violate the 10BASE-T rules, of course, so we decided totrek to the server location and attach four additional ports to the serversegment by means of a mini hub.

Scott: We attached this break-out box of sorts via a microtransceiver tothe original server drop, giving us the four additional repeated ports inwhich to tap our analyzer and server. We reanalyzed and the problem disappeared.

Bill: Armed with this new discovery and the fact that the server could alwaystransmit packets but not always receive them in the same manner, we suspecteda problem with the receive side of the 10BASE-T transceiver built into theserver's adapter.

Scott: It seemed like the server worked well with the short cable pluggedinto our mini hub, but not the longer run to the 10BASE-T concentrator.

Bill: On the oth er hand, the microtransceiver on our mini hub worked justfine with what had been the server cabling segment back to the concentrator.

Scott: Our suspicions were confirmed wh en we yanked (OK, gently removed)the mini hub from the circuit, put an AUI-to-10BASE-T microtransceiver onthe AUI port of the server's adapter, downed the server, changed the driverto activate the AUI port, rebooted the server and observed that everythingworked well.

Bill: Not only that, but overall performance more than quadrupled.

Scott: Moving along, we discovered the same problem on another segment ona different floor.

Bill: So we exclaimed, "Ah ha! Let's do the 'microtransceiver on theserver trick' again."

Scott: Only this time, it didn't work, and the problem persisted.

Bill: So we probed further into the server and discovered that a proprietaryadapter was supplied by the manufacturer of the platform. Therefore, wecouldn't easily swap in some other vendor's adapter.

Scott: We could, however, try a "new and improved" version ofthe adapter that was recently released by the platform vendor.

Bill: It was improved all right, since sw apping in the new adapter revisiondid the trick.

Scott: So, in the end, we had the same manifestation of a problem, but withtwo different solutions.

Bill: Thank goodness we didn't start with a situation where the server waslocated on a segment several routers away from the user.

Scott: Then we probably would have been hopping mad trying to isolate theproblem.

Bill: But, seriously, when troubleshooting these kinds of problems, checkboth ends, and, if necessary, whatever lies in between.

Scott: And never assume that the same problem has the same solution.

Bill and Scott are principals of Pine Mountain Group. They can be reachedat otw@pmg.com. Portions of the actual trace files from selected columnsare available via Pine Mountain Group's Home Page ( http://www.pmg.com ).

April 15, 1996