According to the National Vulnerability Database (NVD) maintained by the U.S. National Institute for Standards and Technology (NIST), 92 percent of reported vulnerabilities are in applications, and not due to insecure networks. That’s 11 out of 12. But this doesn’t mean you can throw away your firewall or go back to using telnet for your remote logins. Considering the NIST NVD contains over 86,000 known vulnerabilities, that still leaves around 7,000 reported network vulnerabilities. Using a layered approach to security is still the best approach.
This does mean, however, that your quality assurance team must also be an integral part of your security team; that is, if you have one. Traditional security teams generally consist of a developer who specializes in application security compliance (security engineer); several security managers; and IT auditors. If you have such a team and it’s not working closely with the quality assurance team, this could be a hole in your application security.
Every web-based application and nearly all mobile apps need the internet to receive the data that makes those applications useful. This leaves the overall system vulnerable to attacks if the application is not capable of handling various types of attacks that can be leveled against it. Threat modeling is an effective method of identifying various threats and developing strategies towards mitigating them.
Testing is the only way to validate that the application or service is protected against a known vulnerability. Having detailed security requirements with matching test cases is a good way to ensure coverage, alongside employing additional negative testing.
Different kinds of attacks are used, but the most common are those that try to compromise the application by using invalid input, either by using invalid or illegal characters, or by simply overrunning the buffers used to accept that data. Wherever possible, fuzzing should be employed to flush input validation issues, and it tends to be useful when dumb and smart (adaptive) fuzzing are used in combination. You’ll also need to be concerned about testing the authentication and access-control components of the application.
NIST recommends the use of an application vetting process to determine if an application is ready for production. This process is described in detail in the NIST Special Publication 800-163: Vetting the Security of Mobile Applications. The TL;DR for the process is simply this: the vetting process needs two phases — application testing and application approval.
The QA team, with guidance from the security engineer, should be responsible for the application testing phase of the application vetting processes. They should use various tools and services to provide a report of results to an auditor, typically the security engineer. The auditor should then make a recommendation to the security manager, based on the data they have received from the QA team, on whether the application is ready for production. Finally, the security manager should either approve or reject the application’s deployment into production.
That explains the app vetting process and the critical role QA plays in the application’s security. If you are a QA engineer, what kinds of things should you look for?
First, let me start by saying that every application is different, and therefore your security engineer (if you have one) should provide you with an analysis of the application’s alleged vulnerabilities so you can target them directly.
As a general starting point, if you don’t have a security engineer to tell you where your team should get started, or you simply want to ensure that most known issues are checked on a least a cursory level, you should consider the use of a good commercial static analysis tool. I recommend commercial tools for two reasons: you get technical support, and they are more likely to be kept up to date. Veracode’s SAST or RogueWave Software’s Klocwork are a couple of great tools for statically detecting vulnerabilities in your software.
As a QA team, the best thing you can do is to educate yourselves. You need to know what kinds of threats are out there. You’ll need to stay up to date on the vulnerabilities and the possible attack vectors. For this reason, I highly recommend at least getting familiar with the Open Web Application Security Project (OWASP). Keep an eye on the OWASP Top 10 Project, as it provides descriptions, example vulnerabilities, example attacks, guidance to avoid and other related resources for each security risk.
The OWASP provides lots of great resources. Beyond all the general information about the latest and most critical software vulnerabilities, it provides an actual guide for developers, security engineers and QA engineers called the OWASP Testing Guide. It covers important topics like identity management testing, authentication testing, session management testing, input validation testing and more.
The OWASP also provides a great tool called WebGoat designed to teach web application security lessons.. It has over 30 lessons, including Cross-site Scripting, Unvalidated Input, SQL Injection, and many more that you'll find valuable.
The tools and information that OWASP provides for the community are invaluable for learning how software applications are hacked. Your security engineer should be intimately familiar with this information. If your company doesn’t have a security engineer, then your QA team should also really familiarize themselves with the security standards and regulations.
There are a couple of useful standards that you should consider, depending on what kinds of sensitive data your application may handle. The ISO/IEC 27002, jointly published by the International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC); and the PCI Security Standard, which is published by the Payment Card Industry Security Standards Council.
Both of these standards define how to secure data, particularly sensitive data. These standards must be met in order for your organization to be certified as compliant. These standards are in place to protect users from having their sensitive data stolen, and should therefore be considered the minimum amount of security you must have on your application. This compliance does not guarantee that your application is secure, but it does give you a good starting baseline.
As a QA team, it’s very important to be aware of any software dependencies your application has. So, even if you do your best at validating that your company has done what it can to secure the application(s) you develop, you must also verify that any interactions with software components your applications depends on are equally secure. It does you no good to invest heavily in application vetting if the dependencies that you don’t develop (such as TomCat, MySQL, SSL, etc.) are insecure. You must understand the implications for your own application of any vulnerabilities in dependencies, especially if you don’t control updates to these dependencies. This is where tools like JFrog’s Artifactory, Sonatype’s Nexus Repository Pro, Black Duck Software’s Protex, Whitesource and others can be used: to identify the dependencies in your software. Most of the good tools can notify you in real time when a vulnerable dependency is added to your software, or when a new Common Vulnerabilities and Exposures bulletin (CVE) impacts any of these dependencies.
Ensuring that systems are running software component versions that are most secure usually involves making sure you have the latest and greatest version on the system. If you have a lot of systems, this can be a very painful and daunting task to do manually. Plus, you can't even be sure you aren't making mistakes as you update, so you have to double- and triple-check, also manually.
This is where (cough shameless plug cough) a technology like Puppet really helps. By automating discovery of everything that's running on your systems, AND enforcing the version you choose, Puppet creates a more solid security stance for your organization, while giving you back time you can use more strategically. Like, writing more security tests for your application.
I hope this helps you and your organization get a better handle on securing your applications. If you have any questions, please put them in the comments. I'll be happy to answer, or find you an answer.
Stan Duffy is a senior SDET (software development engineer in test) at Puppet.
- You can try Puppet Enterprise in a hands-on way. Just download the Learning VM for free, and see for yourself how straightforward automation can be.