I recently looked into a potential QA opportunity, and they wanted to know about my CI/CD experience.
I have worked in environments that involved CI/CD, but other folks were involved with setting everything up. The details didn’t seem important to my ability to do manual testing. I worked closely with developers as I have in any other environment. I adjust my testing for any environment I’m in, and it didn’t occur to me to mention anything about CI/CD. I want to understand what I’m missing.
In your opinion, what specific CI/CD knowledge should someone have as a tester? Does it differ if they are doing automation or testing manually? Who sets up CI/CD in your environments?
What do you think testers should know about CI/CD?
I posted something on linkedin but adding a variation here as might be interesting to a more focused group.
All of the the testers I currently work with are capable of multiple roles, some directly testing related and some a bit further afield.
One of the reasons for this is that we merged some of our guilds, testing, customer support and tech/dev/system ops support. All of which sort of provide a service which was the main reason for the merge. The cross skilling element was partial cover for some roles in a very small team 5 of us to around 80 developers but also ay different times experience peaks and troughs in the need for the specific roles.
We have one pure Dev OPS, system tech support architect level.
Tester, Automation architect level, coder, tech support.
Security/penetration specialist, tester, automation engineer, tech support.
Tester, automation engineer, Quality/planning specialist, testing/people/quality coaching and training.
Support Manager/Tester dual role.
All have their primary roles which is the main strengths they want to spend the most time on.
All of understand the basics of CI/CD which I think should be the level all testers are at.
Three of the five are at the level they can design and set it up. To be clear I see this as a very different hat from tester but very useful in smaller teams particularly if you have an automation hat on quite a bit.
The desire for testers to do everything remains evident and there are pro’s and cons to that but have to say it works for us, as a small team we have a very broad range of things we can help out on. Note all have over ten plus years experience in their primary field.
Your context will likely be very different but dual rolling I’m seeing more often
what specific CI/CD knowledge should someone have as a tester?
Most of the testers that I’ve met had very little to no CI/CD knowledge. Some automation engineers could figure things out on the CI/CD flow.
I find it useful to know:
what is deployed (sub-systems versions and code changes);
where things are deployed; when there are several environments to work on, and dozens of branches to be potentially checked, things can get busy;
how to configure and trigger a specific deployment package myself so that I control what I test and where I test it;
able to identify who triggered a deployment; in large teams, one might want to know who to talk to about a particular thing related to the deployment or build;
what caused deployment to fail (able to review logs, unit, integration, security, health, other checks, static code analysis issues,…)
one might need to be able to support business, operations, finance, tech support, or other departments in building/deploying for reviewing a specific package;
exceptionally, one might even have to take corrective actions;
also useful in knowing how to integrate additional automated checks, without much support from others;
understand the technical constraints, debt, risks, and timings related to various parts of the process and highlight things getting worse, and propose ways to improve them;
There’s always feature flags or deploying things on the “test branch”
Some of your automated tests must run on specific stages of deployment so one needs to know what they are and how to configure them
From a testing and less tooling point of view an important thing to know is “how does testing work in a pipeline and what should we test?”
Testers usually act as the champions of what makes a good candidate for automation in a pipeline, so knowing when something would be good as a manual / automated test is great. Look at Angie Jone’s awesome talk “which tests should we automate” for that. Knowing how to advocate and champion good testing through the stack also helps with this (Definition of done, pairing, being engaged with engineers).
Also if you’re using CI/CD you probably need to know how to look for good enough at speed. You don’t want to be the bottleneck, so what tricks do you have for making sure you can go fast in your testing? Being pragmatic with scope, sharing information point in time, timeboxing, shifting left, using tools and testing styles that speed things up (documenting as you go?).
