The Difference Between a Vulnerability Assessment and a Penetration Test

[Nytro]

The Difference Between a Vulnerability Assessment and a Penetration Test

There are many views on what constitutes a Vulnerability Assessment versus a Penetration Test. The main distinction, however, seems to be that some believe a thorough Penetration Test involves identifying as many vulnerabilities as possible, while others feel that Penetration Tests are goal-oriented and are mostly unconcerned with what other vulnerabilities may exist.

I am in the latter group, and what follows is my argument for why you should be too.

Background

The impetus for this post came out of a conversation with Johannes Ullrich--the highly distinguished, Ph.D, CTO of SANS. He stated to me (read his argument here) that if a Penetration Tester simply accomplishes a given goal and doesn't continue on finding other vulnerabilities, he's done a poor job--which is where I disagree.

Language is important, and we have two terms for a reason. We already have an aptly-named security test for compiling a list of vulnerabilities (a Vulnerability Assessment) and to say that Penetration Tests should always include a Vulnerability Assessment is to confuse the matter completely.

The sole purpose of discovering vulnerabilities during a Penetration Test is to find a method of achieving the goal given by the client--nothing more.

The Question of Exploitation

One thing that always comes up in this debate is the topic of exploitation. Many are tempted to say, "If you exploit a vulnerability, it's a pentest." So in their minds it's a simple matter of, "If there's exploitation, it's a pentest."This is a confusion of the concepts.

Exploitation can be imagined as a sliding bar between none and full that can be overlain upon both vulnerability assessments and penetration tests. Although most serious penetration tests lean heavily towards showing rather than telling (i.e. heavy on the exploitation side), it's also the case that deleting all data from a database to prove that it can be done is often not desirable.

A penetration testing team may be able to simply take pictures standing next to the open safe, or to show they have full control of an AD domain, etc. without actually taking the destructive action that a criminal could. And vulnerability assessments can slide along this scale as well. There's no reason you can't simply have a vulnerability assessment where you're instructed to exploit all the vulnerabilities you find. Sure, it would a long time, but exploitation doesn't, by definition, move you out of the realm of vulnerability assessment.

The belief that pentesting is always heavy on the exploitation, and that vulnerability assessments always lack it is a fallacy. The key attributes of a VA vs. PT are list-orientation vs. goal-orientation, and the question of exploitation is not part of that calculation.

Proposed Definitions

Vulnerability Assessments are designed to yield a prioritized list of vulnerabilities and are generally for clients who already understand they are not where they want to be in terms of security. The customer already knows they have issues and simply need help identifying and prioritizing them.

The more issues identified the better, so naturally a white box approach should be embraced when possible. The deliverable for the assessment is, most importantly, a prioritized list of discovered vulnerabilities (and often how to remediate).

Penetration Tests are designed to achieve a specific, attacker-simulated goal and should be requested by customers who are already at their desired security posture. A typical goal could be to access the contents of the prized customer database on the internal network, or to modify a record in an HR system.

The deliverable for a penetration test is a report of how security was breached in order to reach the agreed-upon goal (and often how to remediate).

The Physical Analog

A good analog for this is a Tiger Team working for the government, like Richard Marcinko used to run with SEAL Team 6. Think about what his missions were: to gain control of a nuclear submarine and bring it out into the bay. So imagine that he were to be debriefed after a successful mission where he broke in through the east fence, and someone were to ask him about the security of the western side of the building. The answer would be simple:

We didn't even go to the west side. We saw an opening on the east-facing fence and we went after our target.

If the person doing the debrief were to respond with, "You didn't check the other fences? What kind of security test is it where you didn't even check all the fences?", the answer would be simple.

Listen, man, I could have come in a million ways. I could have burrowed under the fences altogether, parachuted in, got in the back of a truck coming in--whatever. You told me to steal your sub, and that's what I did. If you want a list of all the different ways your security sucks, hire an auditor--not SEAL Team 6.

Summary

Vulnerability Assessment

Customer Maturity Level: Low to Medium. Usually requested by customers who already know they have issues, and need help getting started.

Goal: Attain a prioritized list of vulnerabilities in the environment so that remediation can occur.

Focus: Breadth over depth.

Penetration Test

Customer Maturity Level: High. The client believes their defenses to be strong, and wants to test that assertion.

Goal: Determine whether a mature security posture can withstand an intrusion attempt from an advanced attacker with a specific goal.

Focus: Depth over breadth.

Sursa: The Difference Between a Vulnerability Assessment and a Penetration Test