Hey Everyone! I’ve looked at some previous CI/CD posts and it’s been awhile. So I figured I’d bring this back up.
I’ve been looking into how to help evolve our testing practices from a QA stand-point. We got our automated tests down pretty well but now as you can imagine those who are mostly manual testers are feeling left out and I’m looking into how to adapt their skills, processes and workloads. I’ve done a ton of research and have ideas of my own but I’ld love to here the community and other experts weigh in with their experience.
So how do manual testers participate and add value in CI/CD? With it being mostly automated, how do you foresee or have experience with more manual testing in this type of environment?
What are you looking for them to do in this environment?
Hands on testers specialise in testing to discover and learn, so exploring, investigating and experimenting with risk. Your CI/CD will no doubt carry risk so they could be running experiments there however a lot of setup is fairly out the box and those who usually set it up in my experience are fairly familiar with those risks so value add may not be that high. It is though often an area that is assumed to be optimal and that may not be the case so even as a learning exercise get your testers to evaluate it from a strengths and weaknesses perspective just as they would a product.
Then you have risks regarding deployment, did it deploy okay etc - this though I’d suspect they already do even though they may not put it into a CD specific risk category
What I would say though is that they should be very familiar with the CI/CD in place and understand how it can impact their testing. They’ll benefit from log access, if their are tests running as part of this they will benefit from that.
These days your hands on testers will also likely be looking to build locally as it gives them better access to tools to do their investigations. As they do this more even if they are not involved in automation they may start leveraging from AI more and maybe even vibe coding some tools and data to help with their investigative testing, adding this to the CI to share with others may be a low risk entry point to gain practice and familiarisation.
They are hands on because they want their testing to be targeting human biological thinking strengths rather than mechanical activity strengths. The feeling of left out may come from having to spend time on those activities that favour mechanical strengths but not having the skills to hand those over to a machine to do, they may need help with that.
For me I was an automated tester years ago but I fell out of love with it. Why? Because I felt my strongest point and the bit I enjoyed the most was finding out what to test, questioning the wider team and exploring it. Being an automation engineer I was quite isolated, automating other peoples tests that they’d done all the work on and I personally didn’t feel much buzz from the creativity as manual testers did all that cool work.
So for me, the manual testers are the perfect people to review the CI/CD pipelines and establish if what is being tested is as valuable as it could be and influence what the coverage should and could be. They can also as part of that establish where their efforts will have the most reward knowing whats being covered in the CI/CD pipelines. There certainly shouldn’t be a line drawn between the 2 approaches, quite the reverse where they feed eachother and evolve together.
I’ve predominantly been a manual tester but there’s been a few areas where I’ve liked to get involved in our CI/CD work.
From 2019-2023 I was a primarily manual tester in dev teams and my involvements included:
Setting up a job to get the latest builds deployed to our UAT-like environment.
Fixed it when I realised that one of our test results wasn’t being checked.
I’ve added some vulnerability scanners to our pipelines.
I used the code analysis tools to confirm what was suspected about where needed the most focus my testing efforts.
(Disclaimer - I learnt CI/CD during my time as a dev)
After switching to a new part of the business as a quality coach I has no experience of the pipelines, there were scattered tests and it was just awful so:
I lent on my ignorance of the new pipelines to support people solving problems. (“explain it to me like I’m a clueless idiot” had a high success rate)
I paired with someone to add smoke tests in Playwright to our deployment process using GitHub Actions (both new to me).
CI/CD, automation and manual testing are three unique skills. To get into CI/CD you don’t need to be an automation tester. There’s no reason why a sufficiently motivated manual tester couldn’t learn how the pipelines work and get involved in improving them.
That said, maybe they just don’t need to add to CI/CD. Whilst I can do pipeline stuff, IMO it is such a waste of my talents. That manual tester mindset is awesome to get analysing requirements or doing different user focused tests.
Thank yall! You have really touched on what I’ve been seeing and reading about. I was just aiming to get other points of view to combat my own bias. I’ve been a developer who created automated testing before I became an architect so I never really had the experience from a manual perspective.
The biggest things I’m working towards is shifting them left. More continuous testing practices. Focusing in on User Behaviors and Usability. I’m catching some resistance, since it’ll be new to the company for this group. So I’m trying to find ways to slowly help shift that mindset over and teach new skills day to day.
This query is very common and present in teams when CI/CD as well as automating completely matured.
Regression testing won’t need the same effort but the role of testers has changed.
Exploratory testing is where human input comes in, for while the tests are written and executed automatically, people will still be able to discover strange behavior, find edge cases, and detect user experience (UX) problems that no pipeline can uncover.
Risk and coverage guidance are some of the rare perspectives from the actual users that manual testers provide to areas for future automation and weakness or risk pointing out the standards, where the danger is.
Test Design & Review
They don’t write code, but still are able to create quality test cases, review the automated tests, and check if the coverage is in line with business requirements.
Quality Checks Around the Pipeline
The entire process of smoke testing staging builds, validating hotfixes, and conducting quick user-focused checks after deployments is still very crucial today.
Step by step, manual testers will gradually pick up some lighter skills—like API testing, basic scripting, log reviewing just enough to work alongside automation, not to replace it.
CI/CD doesn’t rule out the need for manual testing. Rather, it enables the human mind to focus on the more valuable thinking that automation is not capable of carrying out.
This is why I’d never want to be an automation engineer. I was just going through STEC and there’s a bit where Cassandra Leung says “testing informs further testing” and that just highlights the merit of having both forms.
@sharmon When I was a (primarily) manual tester in a scrum team I often had spells with spare time. As an effective “shift left” contribution I would go “Larvae Hunting” where I’d read through stories that were coming and flag up potential bugs early. I kept this up when I had chance as a Quality Coach.
That’s actually a very good question.
Feeling part of the Quality assurance team is a necessity even if you fill the role of a manual tester.
I have few good tips":
Make them familiar with the process, just like in a black-box testing. While your automation is the “black box” all the parts sounding it are “need to know”.
As a Manager, I volunteer one of the automation people to explain, show, demo how they work.
Familiarise them with the tools in use. Make a list with those tools that is being used for automation running, it needed to be in the level that they understand it.(simplify).
Get them involved in all the processes around the automation writing EXCEPT writing the code itself.
Examples:
Go to the last night automation run, check whether it passed.
Go to the last night automation run, check the failed build number.
Open an issue for the automation failed build run.
Investigate the issue
Know where the logs are created, know how to read them.
You did not detail your process, therefore it’s super generic from my end.
If there is automation reports, they can know where to find and read them.
But if there is a test writing requirements, flows etc they definitely can be a part of that too.
Not only that creates a better feeling of the person for him not to be left out but also it help them grow professionally.
Having a back-up for automation personal is also helpful in your team.
Who knows maybe one of the manual testers would want to switch to automation one day and this might make it easier for them to onboard too.
To be honest, I don’t really understand the question
those who are mostly manual testers are feeling left out and I’m looking into how to adapt their skills, processes and workloads
What are they feeling left out of? Do they want to get involved in maintaining the pipeline and writing automation? Do they feel like they have nothing left to do now?
For me, it comes down to where / how they want to develop themselves, and what the team / project needs to improve and succeed. Combining those two to work on something they enjoy and benefits others would be the best scenario. But I personally couldn’t just say what they could do now without more context, because they might end up doing something they hate, that no one really wants or needs.
And yeah I get it is kind of nebulous without deeper context. With NDAs and all that, I sometimes find it hard to explain the problem with what I can or can’t give details on.
However, we have a lot of ‘old school’ QAs who only do manual testing. Devs work their User Story, when they’re done they pass it to a QA for validation (manually checking the ACs). Now with the push for true CI/CD and speed. A lot of the manual validations have been automated, which leads to a lot of QAs with a lot less work to do and thus feeling unsure/unsatisfied of their role. So I’m trying to find ways to adapt(and train) the people in the role in a meaningful way.
