Arachni - Web Application Security Scanner

[M2G]

Introduction

This month’s issue kicks off a two part series on web application security flaw discovery and prevention, beginning with Arachni. As this month’s topic is another case of mailing lists facilitating great toolsmith topics, I’ll begin this month by recommending a few you should join if you haven’t already. The Web Application Security Consortium mailing list is a must, as are the SecurityFocus lists. I favor their Penetration Testing and Web Application Security lists but they have many others as well. As you can imagine, these two make sense for me given focuses on web application security and penetration testing, and it was via SecurityFocus that I received news of the latest release of Arachni. Arachni is a high-performance, modular, open source web application security scanning framework written in Ruby. It was refreshing to discover a web app scanner I had not yet tested. I spend a lot of time with the likes of Burp, ZAP, and Watobo but strongly advocate expanding the arsenal.

Arachni’s developer/creator is Tasos "Zapotek" Laskos, who kindly provided details on this rapidly maturing tool and project.

Via email, Tasos indicated that to date, Arachni's role has been that of an experiment/learning-exercise hybrid, mainly focused on doing things a little bit differently. He’s glad to say that the fundamental project goals have been achieved; Arachni is fast, relatively simple, quite accurate, open source and quite flexible in the ways which it can be deployed. In addition, as of late, stability and testing have been given top priority in order to ensure that the framework won't exhibit performance degradation as the code-base expands.

With a strong foundation laid and a clear road map, future plans for Arachni include pushing the envelope where version 0.4.2 include improved distributed, high-performance scan features such as the new, distributed crawler (under current development), and a new, cleaner, more stable and attractive Web User Interface, as well as general code clean-up.

Version 0.5 is where a lot of interesting work will take place as the Arachni team will be attempting to break some new ground with native DOM and JavaScript support, with the intent of allowing a depth/level of analysis beyond what's generally possible today, from either open source or commercial systems. According to Tasos, most, if not all, current scanners rely on external browser engines to perform their duties bringing with them a few penalties (performance hits, loss of control, limited inspection capabilities, design compromises, etc.), which Arachni will be able to avoid. This kind of functionality, especially from an open and flexible system, will be greatly beneficial to web application testing in general, and not just in a security-related context.

Arachni success stories include incredibly cool features such as WAF Realtime Virtual Patching. At OWASP AppSec DC 2012, Trustwave Spider Lab’s Ryan Barnett discussed the concept of dynamic application scanning testing (DAST) exporting data that is then imported into a web application firewall (WAF) for targeted remediation. In addition to stating that the Arachni scanner is an “absolutely awesome web application scanner framework” Ryan describes how to integrate export data from Arachni with ModSecurity, the WAF for which he is OWASP ModSecurity Core Rule Set (CRS) project leader. Take note here as next month in toolsmith we’re going to discuss ModSecurity for IIS as part two of this series and will follow Ryan’s principles for DAST to WAF.

Other Arachni successes include highly-customized scripted audits and easy incorporation into testing platforms (by virtue of its distributed features). Tasos has received a lot of positive feedback and has been pleasantly surprised there has not been one unsatisfied user, even in the Arachni's early, immature phases. Many Arachni users end up doing so out of frustration with the currently available tools and are quite happy with the results after giving Arachni a try given that Arachni gives users a decent alternative while simplifying web application security assessment tasks.

Arachni benefits from excellent documentation and support via its wiki, be sure to give a good once over before beginning installation and use.

Installing Arachni

On an Ubuntu 12.10 instance, I first made sure I had all dependencies met via sudo apt-get install build-essential libxml2-dev libxslt1-dev libcurl4-openssl-dev libsqlite3-dev libyaml-dev zlib1g-dev ruby1.9.1-dev ruby1.9.1.

For developer’s sake, this includes Gem support so thereafter one need only issue sudo gem install arachni to install Arachni. However, the preferred method is use of the appropriate system packages from the latest downloads page.

While Arachni features robust CLI use, for presentation’s sake we’ll describe Arachni use with the Web UI. Start it via arachni_web_autostart which will initiate a Dispatcher and the UI server. The last step is to point your browser to http://localhost:4567, accept the default settings and begin use.

Arachni in use

Of interest as you begin Arachni use is the dispatcher which spawns RPC instances and allows you to attach to, pause, resume, and shutdown Arachni instances. This is extremely important for users who wish to configure Arachni instances in a high performance grid (think a web application security scanning cluster with a master and slave configuration). Per the wiki, “this allows scan-time to be severely decreased, by as much as n times less under ideal circumstances, where n equals the number of running instances.”

You can configure Arachni’s web UI to run under SSL and provide HTTP Basic authentication if you wish to lock use down. Refer to the wiki entry for the web user interface for more details.

Before beginning a simple scan (one Dispatcher), let’s quickly review Arachni’s modules and plugins. Each has a tab in Arachni’s primary UI view. The 45 modules are divided into Audit (22) and Recon (23) options where the audit modules actively test the web application via inputs such as parameters, forms, cookies and headers while the recon modules passively test the web application, focusing on server configuration, responses and specific directories and files. I particularly like the additional SSN and credit card number disclosure modules as they are helpful for OSINT, as well as the Backdoor module, which looks to determine if the web application you’re assessing is already owned. Of note from the Audit options is the Trainer module that probes all inputs of a given page in order to uncover new input vectors and trains Arachni by analyzing the server responses. Arachni modules are all enabled by default. Arachni plugins offer preconfigured auto-logins (great when spidering), proxy settings, and notification options along with some pending plugins supported in the CLI version but not yet ready for the Web UI as of v.0.4.1.1

To start a scan, navigate to the Start a scan tab and confirm that a Dispatcher is running. You should see the likes of @localhost:7331 (host and port) along with number of running scans, as well as RAM and CPU usage. Then paste a URL into the URL form, and select Launch Scan as seen in Figure 1.

While the scan is running you can monitor the Dispatcher status via the Dispatchers tab as seen in Figure 2.

From the Dispatchers view you can choose to Attach to the running Instance (there will be multiples if you’ve configured a high performance grid) which will give a real-time view to the scan statistics, percentage of completion for the running instance, scanner output, and results for findings discovered as seen in Figure 3. Dispatchers provide Instances, Instances perform the scans.

Once the scan is complete, as you might imagine, the completed results report will be available to you in the Reports tab. As an example I chose the HTML output but realize that you can also select JSON, text, YAML, and XML as well as binary output such as Metareport, Marshal report, and even Arachni Framework Reporting. Figure 4 represents the HTML-based results of a scan against NOWASP Mutillidae.

Even the reports are feature-rich with a summary tab with graphs and issues, remedial guidance, plugin results, along with a sitemap and configuration settings.

The results are accurate too; in my preliminary testing I found very few false positives. When Arachni isn’t definitive about results, it even goes so far as to label the result “untrusted (and may in fact be false positives) because at the time they were identified the server was exhibiting some kind of anomalous behavior or there was 3rd part interference (like network latency for example).” Nice, I love truth and transparency in my test results.

I am really excited to see Arachni work at scale. I intend to test it very broadly on large applications using a high performance grid. This is definitely one project I’ll keep squarely on my radar screen as it matures through its 0.4.2 and 0.5 releases.

Download

Sursa