I wonder if you have any resources how to prepare a test automation strategy from scratch and how to evaluate its efficiency ?
In my previous workplace, it wasn’t that well structured from the design patterns side and it was mostly focus on the UI tests.
I will probably need to set a whole test framework from scratch in the future or maybe workign on exisiting one, so I’m curious to see your experiences there.
How you set the test automation strategy from scratch and include different tests on it easily ? I’ll appreciate if you have a shared git links on examples that could inspire.
How you adapt to the exisitng test automation framework? What kind of critical change you would recommend to be updated ?
Ps: it’s standard question, feel free to use any kind of automation you used before to answer.
@emna_ayadi : This is a very subjective question, few points from my side.
Look for application under test and choosing the correct automation tool is very important considering tool is easy to use and can easily adoptable by QA
Reusability in automation f/w and reporting will play a major role
Whenever I join a project no matter if they already do automation or not, they never have the test automation architecture drawn out. So I always start doing that because they will always have “something” and if it’s a new project, then you can do it also based on the needs of the project.
I draw a pipeline with all the environments and what tests run where, when data gets scrambled, data gets injected, anything generated, … everything that happens including reporting & logging. I also note down who are the people who reports too, who does the Project management & test management, who are the stakeholders. The picture below might help identifying what you ‘need’ to check up on.
After that you can start assessing which part needs improvements and which are missing.
Maybe here and there you can get quick wins and some long term wins but the automation landscape will definitely become more visible. Reporting might be some separate for all the automation levels. A win would be to have them all on 1 dashboard.
It will also depend on your role & project context but I often check up on/questions I ask often:
Unit test coverage & Mutation Coverage
API & Integration Tests
UI automation
How does reporting happen for automated tests?
Which tests they do manual
Which tests are done on multiple levels and why?
How is test data created?
Who creates test cases?
Which tools is used for monitoring & logging + in which environments & what exactly is logged.
For monitoring & logging => are there dashboards?
If it’s a new project, I would look at your reporting dashboard & communicate with your team members. A retrospective for example and just ask what could be improved or how they see the way of working.
I don’t know how far you can go but there is some AI-Framework out there which compares logged bugs with developers and it can indicate based on requirements if a developer is going to create a bug or not (for example: if he always forget to make a field required, the AI will tell you — but that’s taking it a bit too far XD)
Mutation Testing, I’ve never seen it on a project unless I introduced it. It has so much value for the unit tests & quality of the product. That’s a critical change I would recommend
Thank you so much @kristof for such a wise reflection and all the great points you mentioned !!!
So true, I also didn’t know it before only from you in another post in the club, thanks for recommending it ! that would be great to have such new ideas.
Great ones, my previous experience was a bit different and I want to prepare myself for the coming challenge. So all those advice are very valuable
How is test data created? I didn’t get the chance to manage data related to test “test data management”
API testing: I’m sure I’ll need it I’m learning it, started with basics I need more practice
Unit test: How can testers help developers achieving more/better unit tests if developers are not giving any importance to testing ?
I’d also enable pair testing (dev-test) and early testing, BDD can also be useful
I’ve found it useful to zoom out when considering a test automation strategy and seeing where it might fit with an overall test strategy, with information discovery and quality awareness in mind.
And for that, I’ve enjoyed using @danashby’s strategy model. I’d typically end up creating my own version of the model for the context I’m working in, like the example available on this post.
My focus on setting up a test automation strategy from scratch is to keep momentum:
ensure that I have management and team support. This means, that I have enough time and budget for test automation. These tests must be taken into account during the planning. It could be added to the Definition of Done.
learn to use the tools and development environment in a proper way. A quick fix is to copy and paste a lot of code. On the long term this is hard to maintain.
talk with the developers how to improve the code. There are always ways to improve the code. Test automation is like software development. I need to use the right principles to write good code.
make small increments in the test automation code. I like Test Driven Development: make a failing test, make enough code to pass the test, and refactor. I can observe the behaviour of my test code. It is also possible for me to automate checks in some cases.
(for example: if he always forget to make a field required, the AI will tell you — but that’s taking it a bit too far XD)