Although a security background is an obvious criteria for using a security tool, the need for a programming background may come as a surprise. But to best use Arsenal to protect Web apps, you need to understand the basics of how the languages behind these applications (ASP, PHP and ColdFusion) affect Web security.

Bottom line: Arsenal is good for security pros conducting basic Web application testing, but the cost is high considering the lack of features.

Arsenal's Weapons

We tested a beta of Arsenal on a Red Hat 7.2 Linux server running Apache 1.3.22 in our Real-World Labs® at Syracuse University. The product is lightweight, comprising only five PERL scripts and around a dozen HTML pages.

Good News Forced browsing automates an otherwise tedious process. Rip and rewrite aids form-tampering analysis. Installation and setup are easy. Bad News Does not identify potential vulnerabilities or offer solutions. Reports lack organization. Pricey, considering the lack of features.

Arsenal is a collection of tools, so users aren't greeted by wizards or welcome screens, just a menu of features. The most powerful features are spidering, forced browsing for common unprotected directories and rip and rewrite forms.

We first used the spidering feature to find all the files in one of our commercial Web applications. Arsenal found all the application pages and logged the results in XML, making it easy to parse through the results at a later time. Further attempts at spidering the same site occasionally produced errors, and the spider often halted after the first page. We were told this would be fixed before the commercial product rolls out.

In using the product with a Web application built on more than 50 pages, we found the XML report long and difficult to interpret. There were no sorting or summary details, so scrolling through the results script looking for potential vulnerabilities was time-consuming. We expect to see better sorting features in the commercial release, but you'll probably be on your own when it comes to identifying vulnerabilities.

Forced browsing for common unprotected directories was easier. This feature works by firing off HTTP requests to a Web server for a predefined list of nearly 150 common directories, as well as for common scripts, backup file extensions and log file reports. We found vulnerabilities on all five of our randomly chosen sites. We even found a beta Web application on a major corporate site. We could have run these tests without Arsenal, but it would have taken hours to complete even one of the tasks that took minutes with Arsenal. The lists of directories, scripts and file extensions are stored in text files so customization is simple. We easily added common directories to the forced-browsing list for Microsoft Exchange and Internet Information Server administration Web pages. Although we were able to browse for unprotected directories, we ran into more beta bugs that should be fixed in the final release.

Finally, we tested the rip and rewrite feature, which automates the process of downloading a Web form, modifying the variables and resubmitting a "hacked" version of the form to the Web server. While you can use this tool to look for potentially compromised security information, such as session cookie IDs or user names being passed through hidden fields or headers, the process is arduous. We found it easier to submit a negative or fractional quantity into a shopping cart application to get a reduced price or refund than to use Arsenal to find a security vulnerability.

Vendor Information WhiteHat Arsenal 2.0, starts at $10,000. Available: July 1, 2002. WhiteHat Security, (888) 373-5004; fax (240) 220 8121. www.whitehatsec.com

The rip and rewrite process is handy for testing SQL injection attacks and hidden variable manipulation. The URL encoding and decoding feature was great for passing JavaScript and SQL through URL query strings. Base64 encoding and decoding, as well as MD5 encoding, also are offered, but they are not really practical -- only some Web sites use MD5 encryption to prevent cookie tampering and session hijacking. Although you can use the MD5 encoding tool to impersonate a session, this sophisticated attack still requires guesswork and goes beyond Arsenal's scope.

Time-Saver

Once we worked through the bugs, we found Arsenal has some basic time-saving applications. But weak reports and false positives with the forced-browsing feature lead us to recommend a more expensive and comprehensive Web security product for anyone who has more than simple development problems. However, those products have a heftier price tag of around $20,000.