The audience for the test plan artifact may vary from organization to organization. A test plan with an internal project team as the target audience will be detailed against that produced for external regulator consumption (you may decide to keep the two artifacts similar, but many times in practice this is not the case). So, in this article, I will explain the details of a test plan and how to make the document useful and effective.
Regardless of the methodology followed within each organization, a test plan detailing how the overall product or service will be tested and delivered is a key artifact. Delivery may be a combination of internal applications only OR internal applications and third-party vendors. In either case, it is important to describe how testing will be done for internal applications, third-party applications, integration between applications, etc.
Consider the points below while describing the type of test:
-
- Scope of testing
- How new features provided will be tested
- Data requirements
2. Clarity of acceptance criteria
- Ensure test acceptance criteria are clearly defined
- Set your test acceptance based on user story acceptance criteria
- Ensure acceptance criteria are reviewed and accepted by users
3. Test environment
In many organizations, environments are built by different departments. It is therefore essential to clearly define the environmental requirements. A meeting to discuss and make sure your needs are clear is always useful:
- What systems are needed to perform tests?
- What data is needed to perform tests?
- Any other critical activity that you believe is supported (batch runs, monitoring, alerting, etc.)
Third-party product testing
If your organization uses products from third-party vendors, you’ll likely get vendor builds. In most cases, you won’t have any say in how “vendor sprints” should work.
In this case, from the point of view of test planning, please refer to the points above to define the plan first.
As a customer of the receiving organization, it is very important to ensure that the releases you have received still work as expected. There needs to be a way and criteria that releases must meet for later acceptance into the organization, as you don’t want to spend time/effort deploying releases to different environments and then finding out that the basics don’t work .
One way to handle this is to ensure that there is a predefined regression pack that can be run quickly (in hours) and that validates key features of the release.
Better yet, if such a test pack can be run on the vendor side on the same environment setup as yours. Can you, together with a vendor, decide what are the minimum quality criteria for release acceptance? These initial steps will save you a lot of time, effort and money.
Regression testing
For me, this is one of the most important aspects of testing. It MUST ensure (by validating) that the “unaffected” feature that worked before actually works. All features/user stories/requirements are tested in isolation over different sprints cannot guarantee if features that worked before will still work and therefore it is extremely important to have a regression pack that validates the key pillar at a minimum of service. You should invest in automating such a regression pack and ensure that it is run as soon as new code is posted or at least daily.
The definition of the key points above apply to most individual test phases.
