Experimenting Pair Testing with Business SME in a Waterfall project

🗄️ Archive
,
1

Why Pair Testing Experimentation

Last week, I had been assigned to a production change to retest after the change was deployed to the test environment.

I found myself had lost after reading the ticket and comments. It seems to me the summary describes one thing, the description says something else. Moreover, it seems there are a few options had been proposed through the comments and various attached email trails. I was really confused about what were the changes and what are expected behaviours etc. I found I had more questions than I can think of what to test. I was really worried that I’ll miss something important or set up the wrong test data or condition because of my lack of understanding of the changes or the business context. As usual, I asked lots of questions to the business SME but it didn’t help much either this time.

Pair Testing popped into my head just then. I thought it’s going to be fun to experiment in the current project, I asked experts (@lisihocke @lisacrispin) and community and studied (googled) a bit. The session was all set to go :slight_smile: .

Current Project - Waterfall

The current process:

  1. SME talked with the developer about the changes and defects raised from production first.
  2. The developer does his magic and then throw his masterpiece onto me, with his magical phrase “here you go, deployed and ready for testing!”
  3. I then study hard about the change by reading through the tickets and comments. Most likely I’ll also be reading all attached emails where important information may be captured.
  4. I may have lots more questions after studying about the ticket. I reach out to SME and developers to get further clarifications.

Normally, this works out “well” after some back and forth clarifications. Not this time though.

The Plan

After deciding on the experimentation, I thought to make it a fun and useful for both of us. Then, we can experiment this a bit further to include developer as well. You know the 3-amigos for future changes to save a bit of time for everyone.

The Session Planned As:

  • 60 minutes session
  • A meeting room with a whiteboard
  • Quick 5 minutes overview of the changes & business context
  • Brainstorming initial testing scenarios for 15 minutes
  • Driver and Navigator swap every 10 minutes
  • One screen, one keyboard
  • Allow new test scenarios emerging during the session

Reality

How the Session Went:

  • The business SME had to leave early for another meeting. She came back and we continued the second part.
  • Meeting room wasn’t freed up from the previous meeting in time. We found a table in another room but no whiteboard.
  • We sit down on the table and quickly discussed the changes and business context.
  • We went back to my desk and decided our initial set of test scenarios.
  • We shared driver and navigator role during the session but not as planned to swap every 10 minutes.
  • We had two laptops, she was setting up test data on a back-end system while I was putting together the payload for the SOAP API requests. The process was more like, she set up data on her machine while I was watching, then I ran through the SOAP requests while she was watching. We moved onto her screen to verify the data uploaded as to how the requests were originated.
  • There were some questions we need to talk to the developer for clarifications, as well as something unexpected and the SME had to take them away to double-check with other business SMEs. We had run a few ad-hoc scenarios during the process as well.
  • We used OneNote as our note-taking tool for the session, so we both can co-author the notes and it’s easy to share with developers and others.

Outcome and Afterthoughts

  • We decided to try this experiment for a few more times, I recommended 3-amigos before the development start to keep everyone on the same page.
  • Business SME had come to ask me to run some quick ad-hoc testing together with her for a different area. So she can understand better about the current system behaviour in that particular area. (yay! she liked pair testing session we had on the previous day.)
  • I’ve learned more about the business context and data in another back-end system.
  • SME had the chance to see the SOAP API part in actions.
  • We paired up to complement each other with business knowledge and testing skills.
  • It is an effective way to test a complex or unfamiliar area.
  • Two screens and two keyboards were fine, as long as we are working together and always look at the same screen. It works better as we were driving our own device with familiar settings.
  • It would better if the developer is in the session as well, so any clarification can be dealt immediately instead of a separate follow-up process.
  • co-authorizing notetaking tool helps a lot when two screens are used.
  • Session notes style may serve well for the pair testing session.

Pair Testing is a fun way to collaborate and learn from another team member. Pair with business SME helps generate more relevant and realistic business test scenarios.

Useful materials

Next

The first experiment of pair testing with SME worked well in the current project. I’m looking forward to doing it more often and also expanding it as a 3-amigos session runs before, during, after development. Therefore, there’s little surprise to everyone involved and also aim to speed up the feedback loop.

As you can see, it doesn’t take too much to start a pair testing session and there are more benefits to realize when you are doing it.

4 Likes
2

It always doesn’t go as you planned. But what was the outcome?

3

Business SME seems liked it a lot, she’s now keeps coming back to me and ask if we could sit together run some scenarios for learning some new areas we haven’t explored yet :smile: