
By Philip Carden
Managing service provider relationships is becoming a much more complex and time-consuming part of the network manager's role. An integral part of managing those relationships is measuring the service providers' performance against baseline metrics. This may sound like a monumental task, but the good news is that a structured approach to baselining your service provider will result in a better-defined and better-controlled relationship.
We offer you a five-step methodology to achieving baselining success (see "Baselining Your Operations," on page 174). This methodology is not strictly chronological: The data needs analysis, and instrumentation and reporting steps are typically iterative. Also, the control step may not be completed last. For example, management may present you with an pre-agreed SLA (service-level agreement) with which to measure performance.
That said, we'll follow the process, with a particular emphasis on how service analysis and service-level metrics drive engineering decisions, as well as tools selection in the data needs analysis and instrumentation and reporting steps.
Service Analysis With the evolution of the network into an increasingly complex entity, the number of service alternatives has increased dramatically, both in terms of technology and contractual relationships. Your company may have relationships with several service providers for reasons of fault tolerance, geographic coverage, varied classes of service offerings from different providers (one carrier is used for the frame relay backbone, another for Internet access) or just to keep vendors on their toes. The all-important first step is to determine which services need to be baselined.
Although the variety of service types is broad, for the purposes of developing a baselining strategy, services can be divided into three metric categories:
· Bandwidth Services. These are used primarily for connecting sites into your corporate network. You own the equipment on either end, and the service provider charges you only for the pipes in between--whether they are leased lines, frame relay or ATM. Service-level metrics are easily defined and objective.
· Managed Services. The service provider owns and operates CPE (customer premises equipment) in addition to providing bandwidth. For example, the service provider might offer both the routers and the frame relay cloud that connects them. Service-provider performance depends on more than one link in a chain of technologies, so metric definition is a little more complex.
· Application Services. The service provider offers an application-level service, such as Web-hosting or messaging. The service provider has substantial control over end-user performance experience, so service-level metrics based on round-trip response time may be reasonable.
Service-Level Metrics For bandwidth services, service metrics are relatively easy to define. For leased lines, we're concerned primarily with availability (we want them to be available nearly 100 percent of the time--the actual percentage will depend on your business). For frame relay and ATM, we need to augment availability metrics with some performance metrics, such as CIR (committed information rate) or latency.
For managed services, we still are interested in availability and performance. However, the type of metric used for performance will be a little different; the service provider will control more than one element of the overall network solution, and the performance metric needs to be network response time-oriented. Unlike bandwidth services, where the metric is usually very specific and measurable, managed services may need to use a metric that doesn't quite measure actual performance, but rather approximates it.
|