We often get asked about the inner workings of Netsparker’s vulnerability scanning engine. People familiar with network and virus scanners also ask what vulnerability databases we use and how often we update them. In reality, it’s all a lot more interesting than ticking boxes on a list of known issues. Time to set the record straight about how a cutting-edge web vulnerability scanner works.

Two Sources of Vulnerability Information

When most people hear the word “scanner”, they think of software that looks for known risks. This is generally what virus scanners and network scanners do: check targets against a list of known issues, such as (respectively) malware signatures and CVE vulnerability reports. So when customers see how effective Netsparker is, their first question is often: “What vulnerability database do you use?” Well, the short answer is: “None, mostly.” The full answer is that Netsparker is an advanced heuristic scanner that also checks for known web application vulnerabilities – but let’s break this down a bit.

The Mundane Part: CVEs

The idea of relying on a vulnerability database comes from the systems and network security world, where a software or hardware bug is discovered, publicly disclosed, and added to a vulnerability database such as CVE. Network scanners, for example, work by finding such known issues in target systems. To fix the vulnerability, you simply patch or update the affected component.

Some CVEs also apply to web applications. These are bugs in widely-used products that need to be patched to avoid attacks. As one part of its scanning process, Netsparker checks for such issues based on the CVE registry and other vulnerability databases, so scans also cover vulnerabilities such as Heartbleed (CVE-2014-0160) or POODLE (CVE-2014-3566). In fact, the Netsparker security advisory program actively contributes to finding bugs in open-source packages by scanning them for vulnerabilities during engine testing. To learn how our security researchers do this, see our article on vulnerability disclosures.

Although an important part of overall security, checking for known issues is relatively easy and not terribly exciting. Things get interesting when you have to check for unknown issues – and this is when you find out how effective your web application security solution truly is.

The Really Clever Part: Heuristics

The vast majority of web application vulnerabilities are brand new issues that were introduced in new code in custom-built applications – so how are you supposed to know about them? This is the main difference between web application security testing and signature-based security checks: web vulnerability scanning is primarily about finding new vulnerabilities resulting from underlying weaknesses categorized in the CWE system. To find previously unknown issues, Netsparker uses a cutting-edge heuristic scanning engine that probes websites and applications for vulnerabilities just like a penetration tester would.

Netsparker uses a variety of advanced heuristic techniques to find all entry points in web applications and test them for vulnerabilities. This includes automatic URL rewriting to provide maximum scan coverage, automated fuzzing to generate unexpected inputs that may reveal a weakness, and proprietary Proof-Based Scanning™ technology to safely test weaknesses and provide proof that the vulnerability is real.

Because web vulnerability scanners don’t rely on signatures, their effectiveness is highly dependent on the quality and maturity of the underlying heuristic scanning engine. If the scanning engine is too eager to flag suspicious responses as signs of vulnerabilities, it will flood the user with false positives. If it is too cautious or simply not advanced enough, it will miss real vulnerabilities or even bypass whole pages, for example because it can’t deal with authentication. As an industry veteran and technology leader, Netsparker knows how to strike the right balance.

Get the Best of Both Worlds

The purpose of a web application security solution is to help the user improve security more efficiently than with manual testing alone. This goes way beyond vulnerability databases and even beyond scanning itself. To get measurable security improvements, you need a holistic view of web application security that pulls together accurate information from all relevant sources and applies it through effective automation.

Netsparker combines high-quality heuristic results from its industry-leading vulnerability scanning engine with information about known issues listed in vulnerability databases. All these vulnerability results are complemented by asset discovery and crawling information, warnings about outdated web technologies, detailed vulnerability descriptions complete with suggested remedies, best-practice recommendations, compliance reports, and more. This gives you a complete picture of what you need to fix in your web environment, so you can start getting real value from Netsparker in a matter of days.