What’s your definition for when testing is done?
Personally, I’m interested in real-life stories. When is testing determined as done for your current or recent projects?
This post is inspired byJohn’s post:
4 Likes
In my current organization, there are different levels at which we decide the testing is done:
- In the test environment, we decide that testing is done when all the stories are closed and there are no major issues. In case there is any minor issue that is not blocking the user journey, it is planned for the next release or moved to the backlog
- On staging environment - We create some applications before deployment in different statuses and complete their user journeys post-deployment, again, in case there is any minor issue that is not blocking the user journey, it is planned for the next release or moved to the backlog else if all the application journeys are successfully completed we consider our testing as done on staging environment.
- On UAT Environment - We use UAT only for sanity testing. We have some high-level scenarios focusing on sanity testing, and hence, if all those scenarios are successfully executed, then we consider our testing as completed on UAT. Same process for UAT also, in case there is any minor issue that is not blocking the user journey, it is planned for the next release or moved to the backlog. Else if all the application journeys are successfully completed, we consider our testing as done on the staging environment.
Practically, we can’t have a bug-free product, so our focus is on marking the testing as completed when there is very low priority open, but still we ensure that those low priority bugs won’t cause any compliance issue.
For e.g. there is any grammatical mistake in any document, even though we may categorize it as low priority but it may bring a compliance issue if the customer notices it on prod.
So many things are being taken into consideration when we shift the bugs for future spring,t but this is our overall strategy for marking the testing as completed on a different environment.
3 Likes
When all internal stakeholders are happy i.e. ops know whats coming, the customer knows whats coming, product are happy, dev are happy, qa are happy, platform engineering are happy. To do that we have a Quality Gate, 15 minute gathering each talking through their aspects/evidence/thoughts to make sure we’re as confident as possible the customer will be delighted going through a common sense checklist.
For me, my ultimate goal of the quality gate is to not need a quality gate. Nothing should be a surprise in that 15 minute session as we should have communicated everything during the process. But, we’re not there yet, which is why its a very important session for QA to confirm we’re done. QA run the session.
3 Likes
Is “Quality Gate” something devised internally? Or a common practice in other places?
Testing is a task without end. There are too many variables to say testing is completed, finished or done.
You can say you have used up your testing time. You have tested for the highest risks. You have exectued the test cases. You have reasonable confidence in the product. When the team has tested the product. When you have UAT sign off. etc.
But, to test every combination on every device of every app and operating system is impossible so testing is done when an agreed set of parameters (defined in the context of the team, company or regulatory umbrella) is met.
I experienced them 
years ago in a v-model set-up when it was a heavily documented process. They were a very negative experience where everyone was trying to look for flaws in everyone else.
So about 4 years ago we had a quality problem with releases and the main problem was communication - sometimes test related, sometimes not. So I felt we needed some sort of pre-release check to make sure everyone was on the same page and knew what was going on. So I introduced a QG, but the objective being to make sure everybody knows what they should know before we deploy to production systems. Obviously for Agile teams, I adapted it so it was a 15 minute check list - not a heavily documented process.
So like I said I put this in place as a reaction to a quality problem.
The cool thing that happened that I didn’t plan was everyone took responsibility for quality, we would learn to explain concerns to different people so they would understand etc. it really did improve comms.
From my selfish QA management perspective, I wanted to use the platform also to “market” the test team so people outside engineering got interested what testers did and how cool it was. We since recruited 2 more people from our operations team and started their test careers.
So even though I still buy into that initial objective, the benefits have been wide reaching enough that there is no panic to end them.
I once gained agreement that testing never finishes, it stops.
4 Likes
Agree with this word 
that quality is no longer a one-time act, with current software which are getting complex day by day and rapid integration of features, continuous quality is no longer a choice, it is mandatory. So it may stop for a while, but it will again resume with some task.
2 Likes
What are examples of how and when it stops?
Testing normally stopped when the team was given a new piece of work. Testing stopped, but the management team and dev team understood that testing was unfinished.
1 Like