The problem I’m trying to solve at the moment is to create visibility of our testing and processes to other areas of the business support, customer success managers, technical managers). I feel they have a sense of frustration because customers have issues and then they feel they could have been prevented or that we’re not doing anything about it (which is not the case at all).
What I’m thinking is to make our release test plans more public. Share the lessons learned sessions on our escaped bugs with them?!
Have you got any other ideas? Got anything that seems to work well at your company?
Definitely a dashboard that summarizes your outcomes. And you need to create a bug tracking system that is accessible for everyone necessary, so it becomes easy to check if anyone is working on the reported errors (maybe you have that already - than promote it). A regular newsletter might be another valuable format, but that depends on the company size, I would think.
I love the newsletter idea - very innovative! I think the bug tracking system may need looking at - we use two different things - one for customer support and then it filters into jira for the engineering teams. I will find out what access they have.
test plans. It is interesting for the project managers and users because of planning and budget. Sometimes the test strategy was actually read.
weekly status reports. 4 pages are not read, but scanned. Even using a 2 page report using PRINCEII information is briefly scanned.
test strategy workshops. These had a lot of impact. All stakeholders were involved. In one case I converted it to a mindmap which I actively used.
daily standups. The best ones are focused on the target audience and takes a maximum of 10 minutes in total. In the business standup software development terms are avoided. Any discussions are postponed after the standup. The same for long answers on questions.
hallway encounters. They were handy, if I made some short remarks about testing.
ad hoc meetings. Really powerful, if I had a clear request.
prototyping. During a demo the business could not specify, what they needed. I planned a meeting and sketched some possible solutions with them. This was really appreciated.
user acceptance tests. The involved testers had contacts with the business. These tests increased the visibility of testing.
deployment meetings. During these meetings I talked with different stakeholders, how to deploy new software. Also, who would communicate which information to the users using which channels? In one performance review this was highly praised. I have some blog posts, if you are interested.
retrospectives on testing. I asked for things, which went well, and things, which could be improved. This was helpful to improve the relationship.
We’ve started circulating our test summaries - our findings from specific test projects - to stakeholders across the company. They set out what we tested, how we tested it, what bugs we found and what other issues we identified, and suggestions for addressing them. We also make a point of saying what we saw that we thought was good.
Perhaps there’s an opportunity to find just one person in each of those areas you describe. Find someone who is curious about what testing is and the processes that surround it. Maybe you have an opportunity to have 1-2-1 chats with those people, to just ask questions about what they are interested to know about. They might not know what they want to know yet I imagine you’ll get a sense of things through the chat – and your passion will shine through regardless. I have a hunch 1-2-1 chats would give those individuals more confidence in the process and understand the fallibility of any process. And at that point it might be worth considering going wider with your messaging and sharing. Ideally those 1-2-1 people become catalysts or beacons for your message.
Yea mainly that, and if you are into test-case oriented approach maybe some test summary reports - or similar, I use Testrail and it has a bunch of reports that can be generated and stakeholders sure love those pie charts
maybe it’s so obvious that it doesn’t need to be said but from what I gather, perhaps they are not aware that QA isn’t about the number of bugs raised. It’s about coverage.
EG:
If there’s a critical feature that must be tested fully and I write 100 automated tests that found no bugs - that doesn’t mean I failed.
If there’s a customer issue raised later, then, again, on 1 vs 100, that was really good coverage and it’s nearly impossible to even imagine a “fully tested” feature. We do our best as humans with limited time and some issues do get to customers.
If the issues are really low-level (like, the “new tab” button on your browser doesn’t work) or the volume of defects is similar to the volume of testing (so, we spent 100 days proving the release and then we get 20 customer issues after a week), then, yeah, the QA process is broken - what are we doing to fix it?
Otherwise, that’s the nature of business, not the fault of QA and your leaders need to make sure the discussion isn’t “blaming” QA but everyone accepts responsibility for what they sold, what they developed and how the entire development was managed.
It’s like blaming the goalie for losing a football game… Or asking what a goalie is for anyway…
Honestly sounds like a leadership or company-culture problem. No single department should be advertising themselves and demanding due attention and understanding!
Lots of good stuff, but I’d particularly like to agree with what Simon wrote. I suggest that you take the initiative to have 1-to-1 conversations (not yet another meeting with N people) with people from these groups. Find out what problems they already have, what’s important to them, and think about how testing could help.
One way to look at this is a bit cynical, which is it’s easier to get people to tell you their problems than it is to get them to think someone else’s activity is important. But I think it’s more than this. Being people who span the gaps in the org chart, by getting off their backside and talking to people who aren’t in their day-to-day, could be uniquely useful. A by-product of this will be to raise your team’s profile, and will earn you the chance to tell people about what you do.
On a side note, this expectation on testing may be a problem .
Testing doesn’t change the product, testing instigates its status and the tester
informs relevant people about this information.
Thanks Bob! I should have expanded a little in my original post…lessons learnt! We created a working group with reps from the different groups and started discussing. I wanted to get some different views/ideas about creating visibility. Good point though about the 1-1. Will start doing this next week. One of the concerns is that we may not be testing how the users might be using the product. As you might be able to understand, there’s complexity and multiple problems here. Visibility, collaboration and education is going to be the main attack. It’s a brilliant team and I’m looking forward to working on this over the next month’s. Still have plenty of insights and ideas from this thread. So thanks again. Enjoy your weekends!
Thanks @joaofarias yes it might be. Will see what I can do here.
my experience is that when the “visibility” term gets thrown in, it’s a lost cause.
Visibility is management speak for “we need you to prove to us very comprehensively that the problem is not because of you… Because we must blame someone but we are also clueless and rely on silly reports and presentations to know anything about the business”
