What do you consider when creating a test strategy?

Bringing fresh eyes to a project can be tough, thinking of how to test can be hard. So we reached out to the community on Twitter and LinkedIn, asking you to share your ideas.

Hopefully, there are some useful suggestions that you can consider and even add your own thoughts!

Here are some great suggestions:

If it’s an existing project: What have the biggest pain points been in the past? Does it effect our productivity? (e.g. flaky tests in a pipeline). What are our biggest risks? What do we definitely not want to go wrong in prod?

Appetite for risk.

What does this system/component do, and who cares about it. Who expects to benefit and who could be hurt by it, and how. How can we best use the resources available to get information that will enable them to decide whether/when it’s good enough.

Since I mostly deal with automation, my answer is specific to that, but most of it translates to general testing as well, I think…

  • Why? (what are the larger goals we want to achieve by implementing automation)
  • Who? (who are responsible for creating the automation, and how are they organized)
  • What? (what test-related activities do we want to automate, and at what level of scope)
  • When? (how is automation made part of our day-to-day work and our development cycle)
  • How? (what tools and languages best fit the answers given to the previous questions)

The resulting strategy doesn’t need to be a lengthy document at all.

I would say simply what problem are we trying to solve?

I use the Heuristic Test Strategy Model and try to think through each area. I document only those things that are relevant to the project.

What is in-scope

What is out-of-scope

What level of test is required (E2E, component, UA)

What are the dependencies

What is the risk factor

What is the timeline for test completion

4 Likes

The “template” which I use actually has the words “Who, what, when, where and why” in it, with “How” being referred to the context-based plan for each component, except on a global level (Example: a future project I will be involved with has a new technology for me, so “how” must be considered since I need to learn something before testing it."

But what I really do is consider the project environment, the product factors, the quality criteria and (on a more global level) some test techniques. If a test risk is enough to mention, it can also go into a strategy document. This is exactly the Heuristic Test Strategy Model, so I refer to that a lot on my path.

But since the HTSM shows a lot of different factors to think about (MIDTESTD, SFDIPOT, CRUCSSCPID, in addition to a few words about risks, materials, scope, and personel needs which must be taken into account before planning) plus supporting information, the NOTES for the test strategy can be pretty long.

Somehow I manage to cram most of in to a short, readable, document with lots of pictures. My latest test strategy is 5 pages long.

3 Likes

I used the MOT One Page Test Plan as the basis of my test strategy. Basically I took the headings from that article and filled out the doc with higher-level content than would apply to a plan, i.e. strategic level thoughts.

Despite 15 years in IT I’ve never seen a test strategy except for the one I created. My question to the company before I created the test strategy was ‘what is the problem for which a test strategy is the answer?’ Basically I got no detailed rationale. Mostly it was very general stuff about a strategy giving confidence in testing and ensuring a consistency of tester approach. I wasn’t convinced these points necessitated a meaty strategy document, but I created one nevertheless to keep the folks requesting it (i.e. management) happy. The document has seen little light of day since it was created, though perhaps the act of creating it did give management confidence in the test team and the folks on the team some standards they can follow (the team is not greatly experienced in testing).

So I guess my advice is (1) to consider if you really, REALLY need a test strategy and if so (2) who are your stakeholders and what do they want to get from the strategy document. If they are an inexperienced team in a smaller organization, that will be a very different document from what you may create if you are working with a very experienced cross-country team in a large organization.

I think James Bach’s advice on documentation was golden for me: define your documentation need in literally one sentence (and ask all stakeholders to do the same). It’s a very interesting exercise. For my test strategy one of the notable sentences was something like 'the documentation will give readers confidence in what the test team does, and ensure the test team members are performing testing in a consistent way".

Hope this helps :smiley:

3 Likes