Building a test strategy from scratch

Hi,

It is the first working day of 2021 and I have started on a new project.

The project is an embedded product but also has a simulation environment which can be ran on a PC based environment.

Initially the project was a Proof Of Concept so there is not any test documentation/infrastructure available.

I have been thinking about the best way to start testing this product and have come up with the following steps.

  1. Acquire basic knowledge of the product via the system architecture and the object interfaces.
  2. Understand how to build the system and run the simulation environment.
  3. Grow my knowledge with some exploratory testing using the simulation environment.

I am interested to know how other people would approach this situation and what artifacts they would identify as being important.

Thanks in advance
Peter

6 Likes

I am interested in this too from the perspective of the only tester in an org who are looking to start work in earnest on a new product and testing has not previously been included in the definition of done (I know right) I’m looking to try and show my value by working on the strategy in advance, but I need some pointers on a good place to start. Thank you :blush:

3 Likes

Carry on as you are! But create a high level list of things you think are important to test and share this with your stakeholders (maybe don’t share the first list). This is an attempt to prompt the question “X is important to me, but I don’t see it on the list”

1 Like

I good starting point @mranderson , I like that handle , something x-files about it. Archeology almost. A bit like this new product. If it were me, I’d be doing all those things, but I would be looking at the business value, and use that to uncover the key use-cases for the product/feature/project that really matter. And focus a key use-case and test goal around that, and get it agreed, to only be testing the most-agreed-on important aspect of the feature.

I have a few good memories of “skunkworks” projects that came out of nowhere, some of them do go back there too. So for me to get some test coverage for the new project, but make sure I have bandwidth for the old job that I still, have to keep on doing. But I am envious of your new project, all the best for 2021.

3 Likes

I would split it in 2 parts normally

1.:
requirements/functionality:

  • what shall the product do?
  • what are features (must be in /should be in /nice to haves)?
  • are there any requirements?

2.:
things which are often forgotten not mentioned:

  • can it be easily installed?
  • which systems need to be supported?
  • does it worked with other components in an environment?
  • Performance needed? Disc space? Memory? Access rights? Firewalls?
  • Security needed? Encryption? Sensible data (personal or secret data)
  • Backup? Error handling?
  • Any support for maintenance needed? (Restart, Reinstall, Monitoring, Logging)
3 Likes

or potentially a Matrix reference?

1 Like

You are in the right path. With the support of the simulation environment, explore the system. But keep in mind:

  1. If the simulation is doing the right things!
  2. If the simulation is exactly mirroring the actual system
  3. If simulation has all the functionalities of the actual system.

I foresee a lot of interactions with the simulation developers and product owner of the actual system, to figure out both. This should happen before, and in parallel when you are exploring the system simulation.

Congrats on getting your hands on a new stuff in the new year! All the best!

2 Likes

Broad checklist:

  1. Have interactions with stakeholders and make a list of key use cases, functionalities, and understand system architecture

  2. Modules functionality test strategy/test plan

  3. Modules interactions test strategy/test plan

  4. System test strategy/test plan (use cases)

  5. Performance and security testing aspects.

  6. Usability testing (gui and related aspects)

  7. Automation strategy.

  8. Other components test strategy/test plan like API, microservices as applicable

1 Like

Some questions that might help with the strategy:
In what position are you working - role you were hired? What’s your experience with testing, the domain of the product?
Are you alone or are others there as well in the testing team? How much do they know about testing or about the product/project?
What’s your time? How long have you been hired to support that project? 1 week, 6 months, unlimited?
Are you located with the team or remote or offshore?
What hardware & software do you have available for testing purposes?
Can you manage the environment or do you need help from others?
What were you hired to help with?
What is your current mission? Who do you report to? How often? What information would they like to know and when?
Are there any milestones that you should consider? When are they?
Do you have access to people, code, documentations, minutes of the product discussions, tasks that have been tackled, architecture models, emails, data;
Do you know the stakeholders, or could understand what they need from such a product?
Is the product fully developed inside or using other things(systems, APIs, devices, companies, databases, etc) - how is the communication being made?
How often is the product changing? Do you get informed? Can you get informed?
Are there regular meetings about product management or features? Can you try to get in those meetings to catch the direction, meet the people, understand the inner workings…
Are there similar products on the market? What stage are they in, how do they differ or are similar, how is your product going to compete with those?

Also, some good ideas are present in the Product Elements, Project Env, Quality Criteria in James Bach’s HTSM: https://www.satisfice.com/download/heuristic-test-strategy-model

1 Like

@conrad.braam thank you for the reply. I have suffered in the past from trying to do too much. Your suggestion of trying to focus on the parts that deliver key business value to the stakholders is good one. I will try my best to keep that in mind.
I wish you a good 2021 too.

1 Like

@meowy24 thanks for your comment. In one place I worked I made many mistakes. At one point they employed a test manager who thought she could manage the testing activites without understanding the system. I now can seen that was a major mistake on her part as the developers ran rings round her. Therefore I am not starting off with understanding the key components of the system. Good luck on your project. It would be interesting to know how you get on.

1 Like

@countertestserism thank you for the reply. The idea of creating a list of key items is interesting. I’m sure that it would be a focal point for furture ideas and improvement.
Good luck in 2021!

1 Like

@meowy24. Technically it was my fathers name :smile: but as you said it is also a Matrix reference. I have some American friends who can quote the lines better than my standard Glaswegian.

1 Like

@testervenkat thank you for the reply. You have make a very good point that the simulation will be different from the target environment. Indeed today something came up to suggest that was the case! Hopefully as differences pop up they can be either resolved or documented.

I hope you have a great 2021 too!

2 Likes

@andreas7117 thank you for your comments there are interesting points there.

There are quite a lot of requirements so I think the key thing will be to indentify a priority list, as other people have suggested in this thread.

Your last point wrt maintenance is also interesting. I know last year they ran into issues with a live demo and did not have enough logging to quickly understand the issue. So there is room for improvement there. As an developer with many years experience I tend to put in a lot of logging and was surprised at the lack of it in some products.

Good luck in 2021.

1 Like

@ipstefan thank you for taking the time to put down your ideas here.

There is much food for thought here. In particular you have made a very good point about milestones. I will need to find them out!

I hope you have a good 2021.

Congrats on the new project! Perhaps this post might be interesting for you: 7 Steps to Create a Test Strategy from Scratch | Cassandra HL

4 Likes

Thank you for taking the time to comment.

This looks very useful.

1 Like