Principles of Testing Web Applications & Services

Principles of Testing

There are some common misconceptions when developing a testing methodology to find security bugs and vulnerabilities in software.


"There Is No Silver Bullet"

While it is tempting to think that a security or vulnerability scanner or application firewall will provide many defenses against attack or identity a multitude of problems, in reality, there is no silver bullet to the problem of insecure software.

The Application Security assessment software useful as a first pass to find the low-hanging fruit is generally immature and ineffective at in-depth assessment or providing sufficient test coverage.

Security is a process and not a product.

Think Strategically, Not Tactically

Security professionals have come to realize the misconception of the patch-and-penetrate model that was comprehensive in information security during the 1990s.

The patch-and-penetrate model involves fixing a reported bug or vulnerability, but without proper investigation of the root cause. This model is usually associated with the windows of vulnerability, also referred to as the window of exposure. The evolution of vulnerabilities in common software used worldwide has shown the ineffectiveness of this model.

Symantec's Internet Security Threat Report study has shown that with the reaction time of attackers worldwide, the typical window of vulnerability does not provide enough time for patch installation, since the time between a vulnerability being uncovered and an automated attack against it being developed and released is decreasing every year.

There are many incorrect assumptions in the patch-and-penetrate model. Many users believe that patches interfere with normal operation and might break existing applications. It is also incorrect to assume that all users are aware of newly released patches. That's why not all the users of a product will apply patches, either because they think patching may tamper with how the software works, or because they lack knowledge about the existence of the patch.

OWASP says It is essential to build security into the Software Development Life Cycle (SDLC) to prevent the occurrence of security problems again-n-again within an application. Developers can build security into the SDLC by developing standards, policies, and guidelines that fit and work within the development methodology. Threat modeling and other techniques should be used to help assign appropriate resources to those parts of a system that are most at risk.

SDLC is the King

The Software Development Life Cycle or SDLC is a process that is well-known to developers, and engineers. Integrating security into each phase of the SDLC allows for a holistic approach to application security that leverages the procedures already in place within the organization.

Aware of that, the names of the various phases may change depending on the SDLC model used by an organization, each conceptual phase of the SDLC model will be used to develop the application (i.e., define, design, develop, deploy, maintain). Each of the phases has security considerations that should become part of the existing process, to ensure a cost-effective, and comprehensive security program.

Many existing secure SDLC frameworks provide both descriptive and descriptive advice. Whether a person takes descriptive or prescriptive advice depends on the maturity of the SDLC process. Essentially, prescriptive advice shows how the secure SDLC should work, and descriptive advice shows how it is used in the real world.

Both have their use cases. Taking an example, if you don't know where to start, a prescriptive framework can provide a menu of potential security controls that can be applied within the SDLC. On the other hand, descriptive advice can then help drive the decision process by presenting what has worked well for other organizations. Descriptive secure SDLCs include BSIMM; and the prescriptive secure SDLCs include OWASP's OpenSAMM, and ISO/IEC 27034 Parts 1-7, all published (expect part 4).

Test Early and Test Often

When a bug or vulnerability is detected early within the SDLC it can be addressed faster and at a lower cost. A security vulnerability or bug is no different from a functional or performance-based bug in this regard. A key step in making this possible is to educate the development and QA (Quality Assurance) teams about common security issues and the ways to detect and prevent them.

Although new libraries, tools, or languages can help design programs with fewer security bugs, new threats arise constantly and developers must be aware of the threats that affect the software they are developing. Education in security testing also helps developers, and engineers acquire the appropriate mindset to test an application from an attacker's perspective. This allows each organization to consider security issues as part of their existing responsibilities.

Test Automation

In the modern development methodologies such as Agile, DevOps/DevSecOps, or RAD (rapid application development) & more (as it is not just limited to) consideration should be put into integrating security tests into continuous integration/continuous development (CI/CD) workflows to maintain baseline security information or analysis and identify "low hanging fruit" type weaknesses.

This all can be done by taking advantage of dynamic application security testing (DAST), static application security testing (SAST), software composition analysis (SCA), or dependency tracking tools during standard automated release workflows or on a regularly scheduled basis.

Understand the Scope of Security

It is important to know how much security a given project will require. The assets that are needed to be protected should be given a classification that states how they are to be handled (e.g., confidential, secret, etc.)

OWASP says the discussions should be occurred with the legal entities to ensure that any specific security requirements will be met.

In the USA, requirements might come from federal regulations, such as the Gramm-Leach-Bliley Act, or state laws, such as the California SB-1386. For the organizations based in EU countries, both country-specific regulations and EU Directives may apply. For example, Directive 96/46/EC4 and Regulation (EU) 2016/679 (General Data Protection Regulation) make it mandatory to treat personal data in applications with due care, whatever the application is.

Develop the Right Mindset

Successfully testing an application for security vulnerabilities requires thinking "outside of the box". Normal use cases will test the normal behavior of the application when a user is using it in the manner that is expected. Good security testing requires going beyond what is expected and thinking like an attacker who is trying to break the application.

Creative thinking can help to regulate what unexpected data may cause an application to fail in an insecure manner. It can also help find any assumptions made by web developers that are not always true, and how those assumptions can be brought down.

One of the reasons that automated tools do a worse job of testing for vulnerabilities is that automated tools do not think creatively. Creative thinking must be done on a case-by-case basis, as most web applications and services are being developed in a unique way (even when using common frameworks for development).

Understand the Subject

One of the first major initiatives in any good security program should require accurate documentation of the application. The architecture, data-flow diagrams, use cases, etc. should be recorded in formal documents and made available for review.

Technical Specification and application documents should include information that lists not only the desired use cases but also any specifically disallowed use cases (not be used).

Lastly, It is good to have at least a basic security infrastructure that allows the monitoring and trending of attacks against an organization's applications and network, for example, IDS (intrusion detection system).

Use the Right Tools

While OWASP states that there is no silver bullet tool, the tools do play a critical role in the overall security program. There is a range of Open Source and commercial tools that can automate many routine security tasks. These tools can simplify and speed up the security process by assisting security staff in their tasks.

However, it is very important to understand exactly what these tools can and cannot do so that they are not oversold or used incorrectly.

Use Source Code When Available

While black-box penetration test results can be impressive and useful to demonstrate how vulnerabilities are exposed in a production environment, they are not the most effective or efficient way to secure an application. It is very difficult for dynamic testing to test the entire code base, particularly if many nested conditional statements exist.

In case, If the source code for the application is available, it should be given to the security staff to assist them while performing their review. It is possible to discover vulnerabilities within the application source that would be missed during a black-box engagement.

Develop Metrics

It is an important part of a good security program is the ability to determine if things are getting better. It is important to track the results of testing engagements and develop metrics that will reveal the application security trends within the organization.

OWASP says Good metrics will show:

  • If more education and training are required;

  • If there is a particular security mechanism that is not clearly understood by the development team;

  • If the total number of security-related problems being found is decreasing.

Document the Test Results

For concluding the whole testing process, it is important to produce a formal record of what testing actions were taken, by whom, when they were performed, and details of the test findings. It is wise to agree on an acceptable format for the report that is useful to all concerned parties, which may include developers, project management, business owners, IT department, audit, and compliance.

OWASP says the report should identify to the business owner or head(s) where material risks exist and do so in a manner sufficient to get their backing for subsequent mitigation actions.

The report should also be clear to the developer in pinpointing the exact function that is affected by the vulnerability and associated recommendations for resolving issues in a language that the developer will understand. The report should also allow another security tester to reproduce the results.

Writing the report should not be overly burdensome on the security tester themselves. Security testers are not generally renowned for their creative writing skills, and agreeing on a complex report can lead to instances where test results are not properly documented.

PRO TIP: Using a security test report template can save time and ensure that results are documented accurately and consistently, and are in a format that is suitable for the audience.

Related Posts

See All

Testing Techniques Explained It is a high-level overview of various testing techniques that can be employed when building a testing program. Manual Inspections and Reviews Overview Manual inspections

In India, you must visit the National Cybercrime Reporting Portal ( of the Ministry of Home Affairs for reporting all types of cybercrime. This portal is an initiative of the


Thanks for submitting!