We have recently adopted the practice of creating a Test Strategy for all our Ruby on Rails projects. This approach ensures that there is a shared understanding of verification and validation activities and is based on a template that we will be discussing at RubyConf India.
The purpose of a Test Strategy is to define the overall approach, tools, targets and timing of test activities. Additionally, a test strategy can be used to define quality and to ensure testing supports the validation and verification of the software.
In an Agile development environment, traditional test strategy documents are uncommon, as a key agile principle is to favour Working Software over Comprehensive Documentation. Furthermore, traditional documentation does not create a shared understanding of the test strategy. This is because the document authors (and perhaps the testers) are usually the only ones who actually read the document. However, the test strategy has an increasingly valuable role in ensuring quality. This is because the traditional separation of developer and tester roles is being eroded through the use of automated tests. Therefore it is important to ensure there is a shared understanding of what has been tested so that the risks can be minimised and the defined quality achieved.
Automation of tests at all levels (unit, integration, acceptance, performance, ...) enables software changes to be rapidly verified and then potentially deployed. This provides significant business benefit and is a practice that is increasingly common amongst start-up companies. However, before this practice can be reliably used, it is critical to develop a test strategy that reduces risk to an acceptable level.
Another key aspect of the test strategy is that it can help define quality for both functional and non-functional requirements. In the template we included a section to define targets for code maintainability. This is often overlooked in traditional test strategies because it is considered a development only issue. However, internal code quality can be a significant contributor to the technical debt of a system.
The document has been shared publicly as a Google Doc that can be copied and adapted. Although, the template implies using a document-style format for a test strategy, the same result can be easily achieved using post-it notes on a whiteboard. Therefore, whatever form a test strategy takes the common criteria for success are:
- Defines approach to verification and validation
- Defines quality in terms of measures and metrics
- Creates shared understanding between stakeholders