For me I immediately think of cutting down a company’s regression suite, as it was starting to impede our flow a little too much.
The first thing I had to do was get some promise of backup from someone with authority to make those changes. I negotiated a level of safety in removing them with management, getting a feel for their emotions around the subject. Questions like “if I removed this whole section, how you feel about that?” helped me to understand their reservations, and I could work towards a plan that felt safe but progressive enough to be worth the time.
I held meetings with every tester in the company who cared, and individual ones with the testers of each team, to establish what was in the regression list and what they knew about its history and purpose. This helped me to understand who valued what in the suite. It was very illuminating, because I found reasons starting with “those were temporary for a few weeks, you didn’t need to do them for the last 8 months” all the way to “we promise to make these checks because if this fails again we lose a big customer”. Questions included a lot of “what’s this and what’s it for? Is it important? To whom?”. Often I’d have to skip through levels of contrivance through different departments to find out the need for testing was a rumour or folk tale.
Then it was just a case of cross-referencing the automation suite and the regression suite and seeing where we could lean on simple checks and where we needed to keep, or even enhance the suite. Where we needed strong definitions to establish coverage and where we could introduce variation to find new problems. The nexus of the whole enterprise was finding purpose. To be doing something for valid reasons, and comparing it to existing behaviours, and the cost of each.
The questions were many and varied, and changed based on how the project progressed. I could establish some ideas up front, such as the need to check in with management, produce reports on removals and our reasoning, and so on, but I couldn’t imagine what I’d find when I started. Obviously questions had to evolve with our findings. When we ask “what’s this for” and someone says “we put that in months ago but I don’t know why it’s still there” that generates a lot of unexpected new questions.
Every problem is a people problem, so other people were vital in understanding their relationship with the regression suite, as well as the content of the in-team testing and what checks were in the automation code. Management helped give us our boundaries and limitations and safe words while we hacked away at the coverage. Testers helped us understand how varied their interactions with the regression suite were and how deep they went, and where the suite’s charters were lacking. Developers helped us with their feelings around coverage and what they wanted from regression testing, and how that misaligned with the regression suite.
I don’t know if I’d ask different questions now. I think I’d have a more direct approach, now understanding just how much of a mess any simple-looking social system is and how off-the-rails documentation can get even in a short period. I understand more how costs and value can shift quite quickly under a project or company. With that in mind the questions become just as exploratory as the last time, because that exact situation will never occur again, and I’d have to be adaptable to each one.
I learned about the importance of people in a very real way, when it comes to even a simple document that affects people. Each person is a relationship with that document, with a set of opinions and needs and desires both honourable and selfish. Common themes arose concerning strategy, purpose, cost and fear. Those are the things, I feel, that would improve a future approach to the same idea. Having to defend a cut to the suite helped me understand that it was out of value and cost-cutting, not laziness, that those cuts were made, and made me think more carefully about the impact of removing or changing things.
Hopefully any of that is helpful