Although its load-testing reporting functionality is limited, SOAPtest does an excellent job of functional and regression testing, and makes failure analysis a breeze. And the ability to view entire Web services transactions at the protocol level, apply XSLT transformations and validate XML (among many others) more than make up for the product's less robust reporting aspects.

In addition, SOAPtest can test any Web service. I tried it out with services developed using Microsoft .Net, Apache SOAP, Cape Clear Software's CapeConnect and The Mind Electric's GLUE and encountered no problems.

Wash...

I downloaded SOAPtest from the Parasoft Web site, installed it on a Red Hat Linux 7.2 machine in our Real-World Labs® in Green Bay, Wis., and started it up with only one glitch--the shell script needed to be modified so a single environment variable pointed to the installation directory. After that, it was smooth sailing.

SOAPtest requires JDK or JRE 1.3 or higher; I used JDK 1.4. The user interface is not as intuitive as other Web service testing tools I've used (see "Empirix's FirstAct Promises Greater Things To Come for Evaluating Web Services"). To help you get acquainted with the software, Parasoft will have a tech support staffer walk you through a live demonstration before you download an evaluation version. I was able to muddle through basic testing without the demonstration but would have saved some time if I had let tech support go through it with me.

In my first set of tests, I used SOAPtest as a client, evaluating several publicly available Web services as well as services I've developed for testing this type of products. A test wizard is available for SOAP service testing. It is surprisingly simple, requiring only that you enter the location of the WSDL (Web Services Description Language) file of the service you want to test. SOAPtest retrieves the WSDL file, parses it and builds a test from the file. If you don't use the wizard, you'll need to fill in the appropriate information--RPC router, operation and the like--and add each method you want to test.

Good News Can test both servers and clients. Extensive output options. Java-based, runs on Windows and Linux. Can test a wide variety of Web services implementations, including .Net, Apache/ SOAP, CapeConnect and Borland Inprise. Bad News Reporting of performance results not granular enough. Interface not intuitive enough.

At this point you can customize the data being sent by designating parameters. You can let SOAPtest generate data automatically based on the data type of each parameter--the product supports a wide variety of primitives and complex data types. Or you can customize the test data manually or through the use of a function, which can be written in JavaScript, Python or Java.

Here's where the product really gets fun. You can specify many types of output to validate the Web service you are testing (see screen at left). I added an output type for HTTP traffic that would display the entire transaction--request and response --down to the HTTP protocol level.

Output can be displayed in SOAPtest or directed to a file, stderr or stdout. If you'd prefer less information, the request and response can be deserialized and displayed or redirected to a browser

or file. Several other out-put options are available, including passing the data through a Diff function and transmitting it via a socket to some other recording service. Check out the SOAPtest manual for a complete list of the options.

Running the test is a right-click menu choice away. I performed an initial test of a Web service that adds two numbers together, and the result popped up in a dialog box. I dismissed it and then examined the traffic output (see screen at right). This feature is excellent for those learning Web services and for digging into the guts of a transaction: You can examine the results and investigate failures through root-cause analysis.

Rinse...

Regression testing is important, especially when modifying code, and using the regression-testing features of SOAPtest couldn't be simpler. The "Create Regression Control" option runs the test and saves the output as a regression control. Subsequent tests compare the results with the output created, and if the results do not match, the test fails.

But be careful using this feature in conjunction with automated parameter generation. Because the parameters are changed from run to run in this mode, subsequent tests will fail. This is only true with automated generation--if you enter the data manually from the test cases you wrote, this shouldn't be an issue. Once a regression control has been created, it can be updated or the test can be removed.

Load-testing functions appear simplistic but have some interesting facets. Distributed agents as well as your local machine can provide virtual users with which to stress a Web service. To set up a scenario (a scenario defines how many users to emulate and when), you can use a click-and-drag method to determine how many virtual users will be active and when they will be active.

After defining a scenario with a duration of two minutes, I added a "point in time" at one minute into the scenario and then grabbed the point and dragged it from one virtual user to five virtual users. This is a quick way to ramp up the load on the server. The default schedule is set to start all users at the same time and run the entire test under the same load.

Vendor Information SOAPtest, starts at $5,000. Parasoft Corp., (888) 305-0041, (626)305-9090; fax (626) 305-9048. www.parasoft.com

Profiles can be defined, each representing a different test, and multiple profiles can be integrated into a single test. You also can specify what percentage of each profile should be run during each scenario to generate a more realistic user load.

Repeat

One of SOAPtest's personalities is the ability to act as a server. You might use this capability if you've developed a client to interact with a business partner's service or for individual testing of clients when the servers aren't ready yet for integration testing. Creating a server is easy. Specify the port and root directory and then add services. I created a service with a method "square"--which simply squared the number input--and I wrote some code. SOAPtest's scripting API offers Python, JavaScript and Java as options for defining server methods. I used Python and then started the server on Port 8081.

The biggest drawback to using SOAPtest as a service is that it does not generate the WSDL automatically for the service you create. Parasoft said this capability will be available in a future release. I created a test suite for this new service and defined the RPC router, method to call and parameters manually. The SOAPtest service worked just fine.

SOAPtest exposes its APIs so you can create a Web services "router" that approximates a software content switch--and makes routing decisions based on the payload, such as XML elements. Parasoft says this functionality, which is available to just about any Web service if you know how to use the request and response headers, is not intended for production environments--yet. The company is working on a version that is scalable and more robust.

Technology editor Lori MacVittie has been a software developer and a network administrator. Most recently, she was a member of the technical architecture team for a global transportation and logistics organization. Send your comments on this article to her at lmacvittie@nwc.com.