Whenever I join a project no matter if they already do automation or not, they never have the test automation architecture drawn out. So I always start doing that because they will always have “something” and if it’s a new project, then you can do it also based on the needs of the project.
I draw a pipeline with all the environments and what tests run where, when data gets scrambled, data gets injected, anything generated, … everything that happens including reporting & logging. I also note down who are the people who reports too, who does the Project management & test management, who are the stakeholders. The picture below might help identifying what you ‘need’ to check up on.
After that you can start assessing which part needs improvements and which are missing.
Maybe here and there you can get quick wins and some long term wins but the automation landscape will definitely become more visible. Reporting might be some separate for all the automation levels. A win would be to have them all on 1 dashboard.
It will also depend on your role & project context but I often check up on/questions I ask often:
- Unit test coverage & Mutation Coverage
- API & Integration Tests
- UI automation
- How does reporting happen for automated tests?
- Which tests they do manual
- Which tests are done on multiple levels and why?
- How is test data created?
- Who creates test cases?
- Which tools is used for monitoring & logging + in which environments & what exactly is logged.
- For monitoring & logging => are there dashboards?
If it’s a new project, I would look at your reporting dashboard & communicate with your team members. A retrospective for example and just ask what could be improved or how they see the way of working.
I don’t know how far you can go but there is some AI-Framework out there which compares logged bugs with developers and it can indicate based on requirements if a developer is going to create a bug or not (for example: if he always forget to make a field required, the AI will tell you — but that’s taking it a bit too far XD)
Mutation Testing, I’ve never seen it on a project unless I introduced it. It has so much value for the unit tests & quality of the product. That’s a critical change I would recommend