This is a really great question! I think itās essential that every member of the team is equally valued. On my last team, the developers referred to themselves as āengineersā. They did not consider testers, designers, POs or customer support people to be āengineersā. But they would use the term āengineersā like āLetās have a meeting with all the engineersā when they meant everyone on the team. Or they would talk about something the āengineersā needed when the testers needed that thing too.
One way this team tried to be inclusive was with a āthing of the weekā. Any non-inclusive behavior or language we wanted to make ourselves more aware of and try to change, weād make a Thing of the Week. So we made one like this: "When you are talking about the team in general, or people on the team who are in different roles, just use the word āteamā, not āengineersā. It had an effect.
Similarly, remembering to not say āyou guysā but instead something like āyāallā, āhumansā, āmortalsā, āyou folksā was an effective Thing of the Week.
Personally I donāt worry about semantics like ātestingā vs ācheckingā, everyone on my teams understands the purpose of regression testing versus exploratory testing and other testing activities. But Iāve run into the āThat story is doneā quite a bit. I bring this up with the team and ask what we can do to make sure that we donāt say ādoneā when we mean ādone with writing the code - we thinkā. One team put in big letters across our task board, āNO STORY IS DONE UNTIL IT IS TESTEDā. Big visual charts are a great way to help people change their language and thought patterns.
In my experience, itās the managers who are keen to have reports. Itās a good idea to find ways to make the teamās accomplishments and problems visible to management in some kind of concise report. Iāve used old-fashioned risk analysis for this in the past.
My teams have found it more effective to start by setting goals to address our biggest obstacles. For example, if we have too many bugs slipping out to production, we want to have a goal to reduce that. Maybe we want no more than 3 high bugs in production in the next 2 months. OK, what will be our experiment around that? Perhaps: āWe believe that having a tester pair with a developer on stories will result in no more than 3 high bugs in production in the next 2 monthsā. Now we have a metric we want to track, and we want to make that as visible as possible. Maybe a big sign on our physical or virtual story/kanban/task board.
Metrics are so often abused and used to punish teams. And, theyāre easy to game. The most useful āgenericā metrics Iāve found are from Lean development and from the State of DevOps survey. Cycle time, from when we start building a new feature or story to when we release it on production, is a good one. It shows the length of our feedback loop. Ideally we are slicing that feature into end-to-end thin ālearning releasesā so we can get feedback from production use and keep shaping it - or killing it because it isnāt what customers wanted. Mean time between failures on production and mean time to recovery of failures on production are a couple of metrics which the State of DevOps found correlates with high-performing teams.
Check out Nicole Forsgrenās book Accelerate based on the State of DevOps survey for 5 years. She shares a lot of the science behind it.
Weāve talked a lot about this on the broadcast and in here. My advice is, keep bringing this up in retrospectives, possibly in standups if you are experiencing a roadblock. Always when there is a problem affecting testing, make it a team problem. Ask the team to brainstorm experiments to make the problem smaller. Of course this depends on priorities. For example, if the team is pretty new and has committed to practice test-driven development, and they are really struggling to learn how to do that, I wouldnāt distract them with problems around UI test automation. I would see if I could help them get better at unit-level test cases, and I would look for some stopgap temporary solution for the UI testing until the TDD is going better and the UI testing is then a higher priority. I hope that makes sense.
Alan and Brent say that developers own ācode correctnessā. I agree with that. Kent Beck called this āinternal qualityā in his XP Explained book. Developers collaborate with business stakeholders and other delivery team members to gain a shared understanding of each feature and story. They are responsible for designing that code properly for ease of understanding and maintenance, as well as providing unit-level regression tests for it to ensure those small chunks of code continue to behave as expected. Ideally IMO they are using test-driven development. All of that is the internal code quality. It matters most to the delivery team, though it is a benefit to the business because it means we can change the code quickly and fearlessly. The developers who write the code must own this internal quality. Thatās why it makes NO sense to have a tester write unit-level tests. I do know testers who get involved with static code analysis, which is another means of evaluating code correctness, and in some cases, this can make sense. But in general what Iāve seen work best is the developers are also responsible for running the static analysis to ensure their code is up to the standards agreed upon by the team, including quality attributes such as accessibility.
External quality is defined by the business stakeholders. They may compromise on lower external quality - perhaps the usability isnāt great, perhaps they know there are bugs and choose not to fix them now - for business reasons. Or they may be quite exacting about it, also for business reasons. Itās dependent on domain, maturity of the product and many other factors.
Janet and I wrote a three-part blog series about how we wrote More Agile Testing: Learning Journeys for the Whole Team. We used much the same process for our first book, though back then, it wasnāt as easy to do video chats and such. We did a lot of google chatting!
We started when we were both at a conference by doing a big mind map for the content for the whole book. This of course evolved over time but it was a good starting place. We divided the book into sections and sections into chapters. We created a release plan so that we ādeliveredā two draft chapters every two weeks. Each of us would take a chapter, work on it for as long as we could stand it, then trade out. We did a lot of interviews of people on teams around the world and got friends who were expert in different types of testing to contribute sidebars.
We actually had āuser storiesā for the book from our ācustomersā perspective. After we had all the draft chapters, we got feedback from our ācustomersā, a group of friends willing to give us feedback. I remember that we did a big reorganization of chapters and sections after that.
All that time we were mainly working remotely. We worked on the book every day, pretty much. We gave ourselves time off during the holiday season.
With the 2nd book, we took an idea from Ellen Gottesdiener and Mary Gorman that they used when writing their book Discover to Deliver. We got together at my house printed out the whole draft manuscript and put them on the walls. We actually cut. out sections and moved them around and taped them to different chapters or pages. It really helped us visualize the book and see how things would work better. We also printed out all 70 sidebars that people contributed and put them on the ping pong table, then decided where they fit in the rest of the book and taped them to appropriate locations. We also highlighted areas we wanted to update.
It was a fun process, I encourage you to read our blog series on it!
Interesting thoughts, Iāve never thought about the idea of there being internal quality within the codebase and external quality, what the business values. I would assume that both combined is true excellence, while if there is too much focus on internal quality you risk overcomplicated solutions, that may be over engineered, while if you focus too much on external quality you risk code that isnāt easy to maintain or build upon.
Thanks, TJ, for asking this! I have had horses all my life, I was quite the Dressage Queen for decades. Back in the early 00s I was leasing an upper level dressage horse from a friend, and her neighbor rescued some miniature donkeys from an auction. One had been abused and was terrified of people, and my friend agreed to adopt him. I rolled my eyes when I heard of her plan and said, āWhat will you do with a donkey???ā Of course when I first saw Ernest, I fell in love. I spent hours earning his trust. We found out that donkeys need other donkeys, so some friends gave me baby Chester as a birthday gift. I adopted Ernest too because my friend wanted to rescue another donkey.
When my husband and I got our own horse property, and moved Chester and Ernest home with us, we realized that they were too small to defend themselves against coyotes - and we had a lot of big, bold coyotes. I contacted Longhopes Donkey Shelter in Bennett CO near where we lived then to ask their advice. āCome adopt a standard size jenny (female donkey) and she will protect the little boysā. We adopted Marsela and she is the best guard donkey.
All three of them are trained to drive, they pull carts and wagons individually, or in the case of the boys, sometimes as a team. They also do work around the farm like haul logs and brush. They love to work, they love people, and now they are happily settled in here in Vermont!
Great question! This has been my experience over the years, too. Often on my teams, we testers ended up doing most of the UI test automation.
As with everything, I made this a team problem. On one team, we started having to add Ajaxy stuff to our UI, and our existing UI test tool couldnāt handle that. I got the whole team together to discuss choosing a new UI test driver library and framework. Our sys admin volunteered to do a proof of concept on Webdriver and found it worked well. Then, again as a team, we looked at various frameworks that we could ALL live with - devs and testers and sys admins. We had a couple rounds of ābake offsā where we automated UI tests for the stories in that iteration in both frameworks to see what worked better for us. We knew weād be using the tool set for years and we wanted to make a good decision. This investment paid off. We chose a framework, brought in an expert to give us a week of training on it with our own code, and were up to speed right away writing valuable, maintainable tests. The devs were happy to pitch in anytime testers needed help.
On another team, the developers were simply required to automate tests at all levels including the UI for every story they worked on. They all paired full time. I found that developers working alone tend not to provide very good UI test coverage. Having testers pair with developers had better results. Also, we introduced example mapping . The business rules and examples helped the devs think beyond the happy path test cases. We also made changes like moving to using page object.
I love the sounds of this as itād hopefully engage the whole team ā¦ and is light hearted that theyāll actually join in willingly.
Iāve been involved in too many projects where āthe automation guyā chooses. The downside is possible blindness to new options and a smaller supporting network.
Getting more people involved seems very healthy and should mean anyone working on automation can then draw on that wider network for support down the line.
I donāt think they are mutually exclusive. XP in particular promoted doing the simplest thing that could possibly work. The internal quality is about doing practices like TDD and refactoring that help ensure well-designed, maintainable code. The external quality does present more trade-offs, but itās up to the business to make those decisions. Working for a startup, Iāve had to adjust to the idea that we have to get at least a thin slice of a new feature out to production for feedback before it is perfect, which means bugs get into prod. But, we have the means to respond to them within a matter of seconds or minutes, that makes all the difference.
I hope thereās more shift to developer(programmer/coder)-tester(we are also developers!) collaboration. In my experience, having testers embedded in a cross-functional team helps product quality the most. I donāt have any science on what the actual trends are.
Iām one of those people who follows Grace Hopperās advice to ask forgiveness rather than permission. If you feel you need the blessing of the business stakeholders, Iād try to put it in terms they understand - money, including opportunity cost. If the team is spending time fixing bugs that werenāt found until the code got to production, they are not able to deliver as much new value. That is a real cost to the business. Not to mention the cost of task-switching, where programmers who wrote the buggy code maybe weeks or months ago have to remember how the code works so they can fix it. Preventing bugs makes more business sense than fixing them. At the same time, you need to bring risk and value into it. There are lots of ābugsā that customers can work around and they donāt care about those bugs if the things they value are working. I feel like zero bug tolerance in production can work, when youāre talking about new code, but if you have a lot of legacy code, it may note be good for the business to spend time fixing bugs that arenāt critical in it.
In my experience, the best approach has been for the whole team to continually talk about and plan testing activities. Each new release, each new feature, each new story - how will we test it? How will we automate tests for it, if any? Do we have the test data we need? Do we have the subject matter expertise? A lot of this can come out at āthree amigosā type planning meetings as well as team iteration and release planning. At retrospectives, look at the teamās biggest problems, are they related to quality or testing? The team can work together to set goals around solving the biggest problem and design experiments to address them.
That said, I do think most teams benefit from having at least one dedicated, professional tester. These are specialized skills, we can transfer them to others, but other team members have other deep skills. We need different people with different deep skills to solve all our problems and keep doing a better job of delivering value frequently.
I have had horses all my life, I was quite the Dressage Queen for decades. Back in the early 00s I was leasing an upper level dressage horse from a friend, and her neighbor rescued some miniature donkeys from an auction. One had been abused and was terrified of people, and my friend agreed to adopt him. I rolled my eyes when I heard of her plan and said, āWhat will you do with a donkey???ā Of course when I first saw Ernest, I fell in love. I spent hours earning his trust. We found out that donkeys need other donkeys, so some friends gave me baby Chester as a birthday gift. I adopted Ernest too because my friend wanted to rescue another donkey.