Our medium-sized computer software company, where I currently work as a QA lead, is currently fine-tuning our test automation approach to better fit our agile methodology. Even if we have some software tests in place, maintaining the test suite, guaranteeing thorough coverage, and striking a balance between automated and manual testing are all difficult tasks.
Here are some particular places where Iām looking for guidance and understanding:
How do you decide which tests ought to be automation and which should be done by hand when balancing automated and manual testing?
Do you adhere to any rules or standards when making these choices, particularly in an agile, fast-paced work environment?
Updating Test Suites: As the product develops, itās getting harder to keep up with our test suite. How do you keep your automated tests current and relevant in the face of frequent changes to the codebase?
Integration via CI/CD Pipelines: Our automated tests and our CI/CD pipeline are currently being integrated. Which best practices should be followed for the integration in order to guarantee efficient and successful test execution during the deployment process?
Instruments and Structures: For UI testing, we now use Selenium, and for unit testing, JUnit. Do you know of any additional frameworks or tools that work especially well for automating tests in agile teams? Furthermore, how do you manage the learning curve that comes with using new tools?
Test Data Management: Weāve had trouble keeping track of our test data. How are test data for tests that are automated managed? What methods or technologies would you suggest using to efficiently create and manage test data?
I would love to hear of your successes as well as any advice or recommended techniques you may have. If you have encountered comparable difficulties, how did you resolve them? We would be grateful for any advice on how to make our test automation method even better.
There are a lot of variations and nuance between each company, thereās never a one size fits all approach but Iāll talk about how Iād think about these questions.
When figuring out what to automate, the high level is as much as you can, the more specific answer. Automate what makes your dev teams lives easier and gives you more confidence in your deployments. Thatās really what it comes down, what can we automate to give our dev teams more confidence in making changes. Some fields may require more manual testing and some fields may not. I work in fintech, so itās mostly transferring data and calculations so I can automate most everything (although to be honest Iād still challenge any industry with the āWhyā we canāt automate something)
Is there a standard to follow? Yes, now the big industry standard that pure tech companies try to follow is the Test Pyramid - Google does a great write up on it here Just Say No to More End-to-End Tests as well as their book āSoftware Engineering at Googleā as well as Martin Fowlers āClean Architectureā. This will help with the ideas of what and where to automate for the best return on investment.
Updating Test Suites - this (generally) can be a sign of weāre to focused on the top of the test pyramid (End-To-End Tests via selenium/playwright etcā¦) UIs change all the time and lead to flakeyness or instability. For an anecdotal example - I havenāt written any UI automation in 2 years. Iāve been more focused on automating backend code and Public APIs. Once I write them I havenāt had to go back and change them, the only few times Iāve had to is when emergency changes happened. - Public APIs generally donāt change as it can lead to breaking changes to our clients so theyāre more stable compared to UI testing.
New tooling? There are so many tools out there. I would first ask what weāre trying to solve for and what benefit are we trying to bring in. There are so many tools that do so many different things. Like Postman/Thunder Client, WireMock, TestContainers, Insomnia, FSCheck, Chaos Testing, Contract Testing (Pactflow) etcā¦ and thatās now even looking at reporting/alerting tooling for monitoring.
Test Data and CI/CD Iām actually trying to solve that at my company right now with a team of people.
Edit: Welcome to the community
Double Edit : forgot to drop this here for ideas on how to assess and mature practices at a company. This group does a LOT of research and itās really great stuff DORA | Capabilities
,
in advance.