For our thirteenth session of TestBash Home 2021, @naomif takes to the stage to talk about a situation not too dissimilar to the one Ministry of Testing is in at the moment 
Naomi will explore how to get your developers involved with automation and how to set the stage for successful shared ownership of testing.
Weâll use this Club thread to share resources mentioned during the session and answer any questions we donât get to during the live session.
- What do you do when someone replies âItâs not my jobâ, yet their job now requires it?
It helps if you can get a sense of where this is coming from. For example, If they are simply overwhelmed with their current workload and deadlines, you can explain that since this is a company decision, there is full understanding and commitment from management that testing tasks will be built into their timelines like any other task and make sure that you have negotiated beforehand that this is the case. If there are other dev teams already participating successfully you can also draw their attention to them as an example that other teams are already managing to integrate it into their flow and it has helped them find and fix bugs early which ultimately saves them time. If this remains their stance, you can explain as patiently as you can muster - (as long as itâs true) - that this isnât coming from you but is a company decision and though maybe historically was not part of their job, now is. This means that itâs included in the expectations just like their other work, and their success at this will be included in evaluations and performance reviews going forward. You can empathize that you know it can be overwhelming to take on even more than they are already doing but clarify that ultimately this shift should only serve to improve the quality and increase the speed of release of their work since they will be more self-sufficient in this area and not be dependent on others to get their features out quickly. You can also ask them what you can do to help remove obstacles and help them be successful.
2. Naomi, have you had any challenges where either a lack of engagement, or negative engagement has caused you problems in developing a culture of quality?
Sadly yes - though many devs embraced this to varying extents, there were others that really struggled with dropping the boundary between âtester responsibilitiesâ or simply werenât interested. This is difficult not only from the individuals itâs coming from but also because these attitudes can be contagious. When this happens itâs important to do your best to turn things around quickly. If the issue is boredom for example, you can try to get them quickly into the code and problem solving which will hopefully be more interesting and familiar territory to them. If the issue is the blurring of the boundary between tester and dev, maybe get them to help you with problems in the test code or with planning about how to make a certain feature that they care about more testable. The testing they work on should always be about features they are working on and care about so that success at it affects them directly and immediately. Donât do it for them or theyâll lose their incentive to work on this but you can contribute as well so that youâre working on a given project together and making it clear that you have their back. Also keep in mind that some people will always be more into this than others so goal one is getting everyone trained and at basic competency and compliance but you can also respect their individual team dynamics and let them figure out who among them handles the testing most if need be once theyâve all been exposed and know how to. Maybe the people who are more interested in it will end up doing it more organically and thatâs ok. As long as the main cultural change is accepted and they arenât outsourcing all their testing tasks to a separate testing siloed team. They should be operating within the guideline that quality is everyoneâs responsibility.
- What do you see as the most common reason devs donât write automated tests?
I think this generally comes out of habit because itâs just not what theyâve typically done so it feels like an unnatural shift in focus. Also itâs mostly a different skill set and something that they need to work on to pick up - but who wants more work? This is why itâs so important to highlight the benefits that could help them specifically of what we get when devs DO go that extra mile and contribute to testing such as earlier bug detection in their feature code and more control over their releases since in this model testers are not gatekeepers.
- I work 100% remote, even without the pandemic. We have distributed team. How do we get around not having Snacks to make it festive and friendly?
Great question - this is a tough one but lucky for you since zoom forced so many people to be remote this past year there are tons of resources online right now for how to do fun stuff with distributed teams. Since you have a lot of ground to cover it makes sense that you wonât want to do a whole long bonding event here but even something short like reminding everyone to bring a snack in advance and playing a game like âTwo Truths and a Lieâ or doing a quick group bingo with options like âhas seen a coworker in person this yearâ or âhas done chair yoga todayâ etc as found in the link below can be enough to set a vibe and get the ball rolling in positive spirits.
Examples I randomly found on the internet here: 37 Best Virtual Team Building Activities in 2021
- How did you decide between training current employees vs. hiring someone new?
This was not initially my first choice since training and keeping a dedicated team was what I had
known until this point and my comfort zone; however my boss and I discussed the reality of the situation which was that it takes between 4-6months for us to get someone fully onboarded
after we find them and that we didnât have that kind of time until half the team would go out on maternity leave. We also had been burned before by investing a lot in people only to have them move on for various reasons not within our control and it was beginning to feel like an inefficient solution to keep starting from 0. We decided that in investing in our dev team, anyone who comes in would catch on that much quicker because they would have the support of everyone on the floor. We would have many more people versed in test automation and this means that we will all be able to help each other and newcomers so everybody benefits. We almost felt like we didnât have a better option- we just worked through any concerns that we had to make it work and we kept an automation guild in place to keep a testing focus, champion testing within the teams, expanding and optimizing the framework, keeping it a priority no matter what else came up and reinforcing best practices by review and by example.
- How do you ensure that the developers are writing the right tests - either for the feature work or for the risk based analysis?
This is a process that continues to evolve as the devs get more practice at test writing. We coached the developers on how to think and talk about the testing with tips such as having the conversation at the same time as they have their tech specs meeting so that they can make sure that they were planning testability into the feature since this was their first opportunity to consider the feature, how it should work and what tests they should write to verify the basic user flow as well as the concerns they had about it or how it would interact with other features. We also have test tasks included in their regular workflow (so in our case in JIRA and in kanban). For example, when a feature is introduced and a parent task opened in jira, a test automation task is opened as well on the same ticket as one of the sub tasks so that basic test automation for the feature will be included in the definition of done where relevant. The test task would serve as a reminder to create a test plan which would then be reviewed by another person - originally this was always done by someone from the core automation team but ultimately as they got more practice at this another dev could verify the test plan as well. This step ensures that we cover what is important to cover about the feature and prevents reopens of the test task further down the line for the strategy being off. Then a test would be written by whoever got the task in kanban based on that test plan. All tests go through code review by dev and peer review before merge by someone from the automation team. At this point if there is any issue with what the test is covering or how it was written it would be reopened (sometimes this already happened at the code review level but often we find that the issues in what or how the test is making an assertion is found at the peer review level which is why we havenât opened this stage and merge permissions to dev yet). We also have someone from the core automation team meet weekly with each dev team leader to plan and help strategize their automation priorities for the week. This helps them keep testing on the radar as well as helps us all stay on the same page about what features are being worked on as the highest priority. It is also an opportunity to reinforce our relationship with the dev teams by being helpful to them, helping them get rid of any obstacles in the way of their branches or figuring out coverage for areas of concern and keeping us aligned with each other.
- You mentioned something called âShare and careâ - could you elaborate please?
âShare and Careâ is a weekly presentation that the technical teams do on a rotation at our company to give each other insight into something they are working on or something interesting that theyâve spent time on or researched. Each week a representative from a different technical team presents. It is done by dev, manual qa, automation, sometimes design on UX/UI, BI and the CTO .
- If I got it right, you influenced devs to build tests without being a part of their teams. What are the challenges with that âdistanceâ and how to overcome?
It is definitely challenging to âmanage from the sideâ and you do feel sometimes like a bit of a project manager for testing when going through this process. One way to get through this is by understanding that you are not trying to be their boss or manage them- you didnât decide they should test, the company did - you are teaching them HOW to test which is a very different thing. One is being a boss when you arenât their boss while the other is being a knowledgeable colleague with a particular expertise who is helping them be successful. It also helps to build trust by establishing good interpersonal relationships with the people you are working with, asking for help, listening to them, empathizing with them and making sure that they are being shown appreciation for their efforts.
- How do you go about asking developers to start testing if they really push back against doing this?
If they are pushing back that hard my question is why? Once you understand that you can figure out the best response and whether or not dev testing is the right path forward in this particular situation. If the company decision is that it is, donât ask them - explain the new policy and make sure that it is explained beforehand by the highest level of management involved so they understand where this is coming from. Make sure they understand why they are being asked to do it and what the benefits are. clarify that it is understood that there are challenges with taking on this new focus and these tasks and therefore they will be baked into the timelines going forward. The core testing team will be there to support them and you will make sure that they have adequate resources to complete any of the testing tasks expected of them, and this ownership of their testing and responsibility for quality will very much be a part of evaluation of their overall performance just like any other responsibilities in their role. Cheer them on a ton when they succeed in positive test participation in any way. Baby steps.
- If dev gets involved in testing, who would be ensuring the coverage from end-user perspective?
Hopefully everyone:) In our case the majority of the tests that theyâre writing are UI tests which are as end to end as we get in automation. However this mainly covers the regression. New features are still verified end to end by manual QA to make sure that they are intuitive and visually as planned in every way. Most tests go out together with the new feature or quickly afterwards before the next feature comes up and the quality of the tests is ensured by early discussions during feature planning, test planning and test plan review, code review of the test and peer review of the test plus any new test has to prove itâs stability in the run through within itâs test suite so there are many chances along the way for someone to iterate on the test to improve it or flag it for not properly covering the end user experience.
- How do you see the role of QA engineers in a team or org where devs do the automation testing (they own quality end to end)?
This is a very important question that troubles a lot of testers bringing dev into the fold. At our company automation and manual QA are also separate entities who work closely together so what the core automation team or âAutomation Guildâ does is serve as quality advocates/coaches, assist dev teams with test strategy and test planning, contribute to testing and the expansion and optimization of the testing framework including stability as well as incorporating new kinds of testing (performance, api, app etcâŠ) Some also support devops or cross trained and pick up dev tasks as well. Manual QA continues to verify new features from a UI/UX perspective in order to ensure things that bots canât answer as well like is this feature intuitive, interesting, easy to use and does it look the way we intended it to cross device and cross platform.
- What made you sit on your unicorn team success for so long before bringing developers into the fold? You wanted to build trust did you not? What held you back?
Good question- it could be force of habit and a strong desire to micromanage when it comes to quality. Giving up that key to the gate requires a whole lot of trust and delegating quality to people not directly reporting to me meant that I wonât necessarily have full control over every line of code and test that rolls out. This was a lot to get over for me but I knew that I had to for the good of the company and I dealt with the anxiety of letting go by channeling that effort into training others well. I knew that just as I canât be everywhere at once, neither could my team and I would be doing my dev and automation colleagues a disservice by having quality revolve around us if we knew that we couldnât scale. Ultimately I was happy to find that we didnât lose out on sharing this responsibility. We gained from their technical and feature experience and the time we gained back eventually from sharing ownership we were able to convert into time spent on diversifying and strengthening our test framework and capabilities. I think that we had to get to a certain stage of readiness as a company for the transition to be successful and Iâm not sure it would have worked as well had we done it sooner but no regrets.
- did you encounter the discussion of independent verification (aka devs should test their own) - what did you reply?
Iâm not sure if Iâm understanding this question right but if the question is, did it come up whether or not devs should test their own code or we should have QA, our take on it ended up being that both are true. Devs should share quality by writing automated tests for their own feature code which helps them plan testability in better and shift left by finding bugs faster and lowering the cost and blast radius of bugs found since the sooner you find them the easier they are to fix and the less likely they reach a user and do damage. However manual QA is still needed because they are pros at user experience and are better able to objectively represent the user while using their professional tester chops to kick the tires and find the holes. An automation Guild in our case is important for keeping a tester mindset focus, advocating that testing stays a priority and gets into the workflow each week, expanding and optimizing the framework, and diversifying testing, visualizations of results, stability of test runs etc⊠areas where dev are less likely to naturally give attention as they are highly focused on their features. Feature quality does not contradict with feature development but general testing frameworks and other testing activities outside of their immediate features often do not align as well with their regular workflow. Therefore itâs important to keep tester involvement and their unique perspective and contributions too.
- What training did your automation developers or testers get to be upskilled to lead, assist and advocate their own feature team in quality?
Before we knew we would undertake this we did a programming course together. We did this by taking 2 hours a week in office on a rotation matched by a voluntary 2 hours a week from home. We would box out the learner for their designated learning time making sure they wouldnât be disturbed and covering any questions or disruptions for them ensuring that we each got through the material at a reasonable pace. Each week a different team member would present to the rest of us a concept learned that week and a practical application of it being used to solve something in our test code or an example of how it could be used or has been used before in a relevant situation. We also refactored the original test code as a team because it needed to be done before bringing in the devs but this turned out to be a great practical application of the programming concepts learned and made our skills that much sharper before the training. We solidified a code of conduct for code reviews to make sure that we were holding to a unified standard of what we expect from each other and our new contributors within the new framework. We also had a bit of HR team building together to make sure that we were aligned and working well together as a team before taking on training others.
- How did you measure test coverage with UI tests?
Our measure for this is not exact but qualitative based on the features and platforms that our company offers in the context of the test suites that weâve built to cover them as well as bugs caught by test automation in the context of the amount and types of issues that we see in production (thankfully these are scarce and generally not from our user facing features covered by automation). We know from this with a great degree of confidence that the user facing features are well addressed. Our continued ability to maintain this level of coverage and confidence as we launched 2 additional websites in the passed year was largely in part to the whole team approach:)