However much your team uses automation to support your testing, I’m betting the road to where you are now was hard fought, and full of myths to be busted and misconceptions getting in your way.
I’d also wadger, that job isn’t finished yet, and you’ve still got more blockers to adopting automation in a healthy way that fully supports human testing and development.
So, have at it, what are your war stories? What are the blockers that remain? Maybe by posting them here, we can realise we are not alone, and share tips on how to overcome the problems we face!
Misconceptions I’ve seen recently:
Unit tests with mocks all the things, fast is king only
Unit tests and TDD mean we reduce the need for other testing (partly true… at a stretch)
In writing it I realize that a lot of misconceptions crop up because the language around automation testing is so imprecise. Half of the reason why people can’t agree on when to write a unit test is because they can’t even agree upon what a unit test is.
Truth here, all the way! What a unit is, and how to test it, are so fundamental and yet lack so much agreement and clarity. I suspect I would get slightly different answers from each Dev I work with.
Automation test repository is a sacred land only QA are permitted no one else can’t contribute the test script
Test automation have to separated from dev repo
Use BDD and only believe that it make test automation write/maintain easier
Test automation = Every QA focus write every test every scenario in UI level
Look a bit deeper I think all four point kinda link into how people do collaborate, make the team more cross functional rather than work. on their own silos.
Or, related to the one by @theology:
Automated checks are supposed to catch many bugs. If not, then automation is not worth it.
As if regression finds many bugs in all but the worst of code bases, regardless of automated or not.
Unit tests with mocks all the things, fast is king only
I can see why you’d want unit tests to be fast if you run them on every PR or build. But at the same time you need those slow tests too, and maybe they just need to be run at a different phase of the dev/release cycle.
Code coverage tells us the quality of our software.
I recently made a pitch to stop including these numbers in a review meeting where people were showing how they went from 78.3% to 78.9% coverage last sprint. That misses the fact that the tests intermittently fail. That misses the fact that the most brittle and high risk areas have sod all automation. When ranting with a colleague he told me that his previous work had a requirement to have 95% coverage before release, so folk wrote tests that didn’t even assert anything.
Also one I saw was a job advert for a Test Engineer who can perform TDD. That would be… a developer.
The two most common misconceptions I’ve encountered are these…
1: We can automate everything. The truth is that not all tests or testing types are good candidates for automation.
2: Automation equals CI/CD. It just doesn’t… Automation is a key piece of that process, but it absolutely doesn’t take you from your current process to CI/CD by itself.
Edited to add a third item…
3: Once automated tests are written they’re good to go! So many teams and companies don’t factor in that as their product grows and changes, the automation will need to be maintained and expanded to match it.
