Who do you write a test strategy for?

Hi folks,

I have another question, this time about strategies.

Who do you write a test strategy for?

I want to show to learners that we write for strategies for a wide range of individuals and roles and weā€™d like to know who you have written a strategy for recently or in the past. For example, I have written strategies for test managers, engineering leads and testers who report to me in the past. Who have you written one for?

As always, thank you for your contribution :robot:

3 Likes

I usually experience that a Test Strategy is a document (sigh) defining the test process, phases, and tool setup. In an outsourcing context, it is written to be approved by the customer, sometimes even become part of the contract. In other contexts, development shops, the test strategy is just the content on the wiki detailing how testing is done - with everyone in the team as the audience.

(Did I mention thereā€™s a better form of strategy - that is not a document? :wink:)

4 Likes

How do you help to define best practice and standards without a strategy document or equivalent to refer back to?

2 Likes

Indeed, i have written a test strategy doc for senior managementā€¦who were wanting to know what i was all about when joining a new organisation. I say docā€¦it is a few confluence pages, written in wiki styleā€¦but weā€™ll say ā€œdocā€ here for ease. I also use it as an on boarding tool for new team membersā€¦ Its worth being to emphasise that they will not be writing shed loads of test cases, but using exploratory techniquesā€¦that they will not be writing verbose test reports that no-one reads, but using lean documentation approaches to capture just enough detail on whats been tested and where the outstanding risks remain.

Itā€™s a high level docā€¦a doc that sets the scene on achieving higher levels of quality in a fast paced environment. It outlines the likely tooling for specific types of testingā€¦but it is a guide, a live doc which is open to change as processes and best practices change.

It is project and feature agnostic, open to be read and adopted by all.

5 Likes

During the early years of my career as a tester, i thought that a Test Strategy was written just for the test team. But as i matured , learnt from feedback and the sign off authorities , that this document was being consumed by almost all the stakholders of the project to understand what the Test Strategy and their role and responsiblities . So when i started as a test manager and started writing strategies, i try to cover the below key stakeholders :

1. Program Manager/Project Manager/Sponsors: Who are interested in knowing what the overall test strategy is that is going to be followed and implemented for the project and whether it will satisfy the objectives of the project. Governance Model - Meetings, Reports and their frequency.

2. Infra Team: Who are interested in the the ā€˜Test Environmentsā€™ that need to be stood up for the project - ST, SIT, UAT, Pre-Prod. The connectivity requirements between the different components of the system, so that appropriate firewall rules may be determined by them to implement for the project. Also, any infra specific assumptions, dependencies like tools, accesses and the timelines when they are required by.

3. Dev Team: Who are interested in the Exit criteria from DEV and Entry criteria for Test for the code to move into the test environment, for Eg. Unit tests completion and evidence shared as an Exit criteria from DEV and Entry to Test. Also, the Defect Priority and Severity definitions and SLAs for fixing, Defect Workflow, besides other Dev specific dependencies, assumptions.

4. Test Team: - My most important audience - who must know the overall test strategy , the objective of the project, test methodology to be used , the different test phases and the techniques of testing to use, the scope of testing in each testphase, the deliverables they should be working on, the overall schedule, the various stakholders in the project and their roles and responsibilities, the defect management strategy, tools to be used for looking at requirements, test case management, meetings they must prepare for, reports they must submit daily, assumptions, dependencies, entry and exit critieria etc.

5. Support/Maintenance/Security Team: who were interested in the scope of the Operational Acceptance Tests including Performance, DR, BC , Availability, Capacity , etc which are going to be executed for the project as they were going to be the team maintaining the project post go-Live.

Sorry about my long replyā€¦

3 Likes

Oh, do write the way of working down :slight_smile:
consider, though, if its actually strategic - in the sense that it explains why more than how.

How do you help to define best practice and standards without a strategy document or equivalent to refer back to?

Often we donā€™t have to.

Best practices donā€™t exist because context does. Whether or not you need explicit, written testing standards is defined by that context. If it better to define and formalise those standards, do so, otherwise do not. Test strategy is a set of ideas, not documents; we only choose to turn the ideas into documents.

The problem with codifying things is that they can be wrong, change depending on the viewpoint of the person that writes or reads them, become outdated, and then they need constant updating (which is costly) or throwing away or they begin to guide everyone towards the wrong goals.

So, in general, Iā€™d say that test strategy is an abstract concept unique in the minds of each person in a project, and at different levels. The strategy changes, adapts and grows as testers learn more about the product and the users, and they learn about risks.

So I say write down only what we need to so that we remain flexible and adaptive to the ever-changing chaos of our project. ā€œWhat we need toā€ will be different for every project, of course, and may also change as the project progresses.

4 Likes

I define test strategy as the set of ideas that guide my test design.

I cannot write my whole test strategy out because it constantly changes based on what I learn, and is too big to define in detail. There are also different levels of strategy based on the scope of the ideas within it.

So who would I write down some of my test strategy for?

Mostly myself. I write my own charters, which are expressions of the ideas Iā€™m using to guide the testing Iā€™m doing. As my knowledge of the product and its risks increases so does my understanding of useful coverage, and therefore Iā€™m likely to write down more strategic information as this happens to help keep track of it.

If Iā€™m expressing to someone why I did what I did in my testing that could be an expression of test strategy. I might do that if someoneā€™s going to read my notes later, which could be me (to see what I did, refer to it in new testing, examine it for ideas, or pairing), another tester (pairing, coaching, teaching the product, explaining a technique), or anyone who wants insight into my testing such as a product owner interested in coverage. Note that this isnā€™t just expressing what I did, but the ideas that guided why I did it.

I might communicate strategy to developers. If they know some of the driving forces behind my testing they can develop with my testing in mind. If I write down some of the questions I want answers to when I test they can use that as an indicator of risk and use it to influence how much effort (and of what kind) they put into to designing, coding and testing parts of the product.

My strategy may be influenced by the strategies of other testers, and therefore their strategies may be influenced by mine, so I could write down some of my strategy. One example is that of coverage, if Iā€™m splitting up coverage according to some coverage model I might tell other testers so they understand the scope of my intent.

I also may talk about my strategy with anyone in the team or beyond to get their perspective, input, feedback and/or consent.

Iā€™m sure there are more.

1 Like

I write a test strategy for me (and my team). I write a test ā€œplanā€ for higher ups. They are 2 different activities for me, the one is a thing us testers are iterating upon, the other is a doc covering what itā€™s going to cost the company.

But yes, being clear about intended audience will help you frame what you want to communicate and will frame where the document sits. I have often enough had senior leadership want to know how we test, so I always write a quick test plan. Because generally the higher ups want to know which tools we are using (and which ones they need to pay for.) And so the plan needs to cover stuff like security risks of the test rig and how testing hooks into the SDLC processes as well as into CI/CD. None of that really belongs in my test strategy, which I write for the use of devs and testers.

3 Likes

My approach has changed over the years, going back 15 years I used to manage a large separate test group and their we did produce strategies and plans specifically for testing, in hindsight perhaps they were mainly communication vehicles for other stakeholders though of course we also took value in the thought process of producing them.

These days Iā€™m not so keen on having them separate so its mainly whole team documents but I do occasionally get asked for a test plan again for that awareness element for others.

One of the interesting changes in my views, I remain a strong advocate that testing is not Quality Assurance but quality has come back to being the focus around a lot of the thinking about testing.

With this in mind I see a team based quality strategy with sections on testing as more valuable and allows a much more holistic view of both testing and quality and how they intertwine. This is much more for the team themselves at software development level but it also serves at that communication, due diligence and quality focus elements that appeals to customers and other stakeholders, they would also have input into quality goals that the strategy is based from.

Most of my context is new or newish products still adding new features so great opportunity to put in good practices from the start but on occasion Iā€™ll step into legacy products with a lot of technical debt in terms of regression risk. Regression risk focus tends to move it to an automation strategy but I still like to emphasise that whilst the strategy looks to stabilise regression risk we can also look to address some of the bigger root cause issues that created that risk in the first place.

1 Like

^^ for me, this response by Andrew takes away the prize ^^

1 Like

I champion testing in agile green field development teams and I write a strategy to sell testing into projects. That means the document is for leadership, stakeholders and team members.

For leadership:

  • Why do I need testing?
  • How can testing help me make decisions?
  • What reports or information do I get from testing?
  • How do I plan testing (resourcing and timings)?

For stakeholders:

  • Can testing give me confidence in the product?
  • How can I get information from the project team about quality?
  • What due diligence is taking place?

For team members:

  • What can I expect from testing / testers?
  • What do I have to do?
  • Where will tests live, where do I look for things?
  • What are the risks we need to think about?
  • What are these terminologies and what does this all mean?

For other testers:

  • Whats expected, what are we selling to the project that weā€™ll do?
  • A joined up thinking about what we need and what risks need to be overcome.
  • What championing / training do we need to give people?
  • Do we, as test professionals, have a shared vision?
3 Likes

Yeah similar to how we structure our current documentation thanks for the comments.

2 Likes