Testing Against Implicit Requirements with Nicola Lindgren

[mwinteringham (Mark Winteringham)]

For our first talk of TestBash Home 2021, @deament takes to the stage to talk to us about testing against implicit requirements, how to spot them and what shapes them

We’ll use this Club thread to share resources mentioned during the talk and answer any questions we don’t get to during the live session.

[antonella (Antonella Scaravilli)]

Questions we didn’t get to

1. I find there are some people who are keen for QA to act as gatekeepers for a release. How do you find this works with reporting your findings without judgement?

2. I find there are some people who are keen for QA to act as gatekeepers for a release. How do you find this works with reporting your findings without judgement?

3. what would you suggest as a way to get testers or devs “to read between the lines” or think about the implicit requirements

4. What do you think are usually the telltale signs of implicit requirements still to be uncovered in a project?

5. Do you identify any metric / how we can measure testing effectiveness applying this to our development cycles?

6. Do you ever have disagreement between dev and product when you reveal an implicit requirement?

7. Do you think, the risks with implicit requirements is higher for “newly formed” teams vs teams, who work long time together?

8. Do you find implicit requirements are discovered/identified easier when pairing/mobbing on testing or getting your head down and focusing on the problem?

9. Where in the time line in a project lifecycle? do you take up the implicit requirements at project start up, SBE? sprint start? exploratory testing?

10. Is there a good way to handle clients (or management) when implicit/unexpected requirements appear?

11. Do you advise acquiring some domain knowledge or gain some understanding of similar projects, when the testers senses that the requirements are implicit?

12. Do you use example mapping and more team meeting to uncover more implicit requirements?
    How would you go about explaining implicit requirements to someone who is new to the project, who has not have been there from the beginning?

13. How early do you think we need to start implicit requirements testing in the testing life cycle? Do we convert every implicit requirement to explicit ?

14. How would you go about explaining implicit requirements to someone who is new to the project, who has not have been there from the beginning?

Resources mentioned

Implicit requirements http://airfocus.com/glossary/what-are-implicit-requirement

Risk storming online https://riskstormingonline.com/

What’s a heuristic? Heuristics and Cognitive Biases

Heuristics game from Ministry of Testing Would Heu-Risk It? – Awesome Merch by Ministry of Testing

Useful Ministry of Testing article about heuristics

Software Testing Heuristics: Mind The Gap!

TestSphere Arena Expansion

Arena

European Accessibility Act details, it is a requirement in Europe European accessibility act - Employment, Social Affairs & Inclusion - European Commission

Maik Nog’s book recommendation
on the topic of implit/explicit: Tacit and Explicit Knowledge: Collins, Harry: 9780226004211: Amazon.com: Books

[deament (Nicola Lindgren)]

I’ll go through roughly half of the questions tonight, then the second half tomorrow

• I find there are some people who are keen for QA to act as gatekeepers for a release. How do you find this works with reporting your findings without judgement?
  I find it can be tricky as you may be forced to give a judgement on behaviour being good or bad, even when you just want to present information on how it is. If backed into a corner, I will give my opinion on how risky something can be if we were to go live with something that could be very problematic for end users. But usually benchmarking my findings or just presenting what I find causes the product owner or the rest of the team to act as a gate keeper.

• what would you suggest as a way to get testers or devs “to read between the lines” or think about the implicit requirements
  *I find starting discussions with some of your assumptions really helps (with how you, yourself, have read between the lines). Then this helps trigger other people to discuss other aspects - you may find you first discuss browser or device support, but the conversation will probably take you to other aspects of non-functional requirements etc. *
  I think it’s also good to have a go-to of things to start up discussions (HICCUPPS and the non-functional requirements I mentioned in my talk, tend to help me get things going)

• What do you think are usually the telltale signs of implicit requirements still to be uncovered in a project?
  Lack of written requirements (i.e. either they don’t exist or they are very minimum/bare)
  In discussions about the features, finding people have different understandings of a few things, to me, then implies there could be more things people have different understandings of (so it opens a can of worms)

• Do you identify any metric / how we can measure testing effectiveness applying this to our development cycles?
  I can’t think of any good reliable, measurable metric (as people can act in such a way to make the metric show good results, but testing could still be done poorly).
  This isn’t the easiest thing to measure, but I do find that when developers actively seek out the input of testers early and others in the team try to get testers more involved - then that is usually evidence that the tester has been recognised to be doing a great job (that’s not to say that if these things aren’t happening, the tester isn’t good - just that the team has recognised the value the tester adds).

• Do you ever have disagreement between dev and product when you reveal an implicit requirement?
  Yes. And that’s where the fun starts - we explain our viewpoints and then come to an agreement on what we think the website/app should do. I’m just happy that the disagreement was brought to our attention so we can address it
  I’ve found it can be useful to have allies, in past cases the UX designer has been a good one for this, but ultimately I’m not trying to argue that I’m right - more than there could be an expectation for this implicit requirement we should investigate.

• Do you think, the risks with implicit requirements is higher for “newly formed” teams vs teams, who work long time together?
  I think they are a higher risk for newly formed teams because they haven’t yet gained a massive understanding of what the product owner or business analyst expects/means when they say/write certain things. Over time, you can learn from past mistakes, and have past features etc, built together as a point of reference.
  There is definitely still a risk of implicit requirements for established teams, but I think having that shared history helps minimise the risk.

• Do you find implicit requirements are discovered/identified easier when pairing/mobbing on testing or getting your head down and focusing on the problem?
  Pairing or mobbing.
  More specifically: having discussions with someone so you can trigger ideas off each other.

• Where in the time line in a project lifecycle? do you take up the implicit requirements at project start up, SBE? sprint start? exploratory testing?
  You can find implicit requirements all the way from project start-up right up until exploratory testing the website/app in front of you.
  [/quote]

[deament (Nicola Lindgren)]

Here is my second half of answers:

@antonella

• Is there a good way to handle clients (or management) when implicit/unexpected requirements appear?
  I think it can be rather tricky as that depends a lot on whether the client thinks it should have been included/done in the first place, or if they understand that the implicit requirement wasn’t immediately obvious. It also depends on the pre-existing relationship you have with that client - if you’ve already established trust with them, then it’s easier - if your relationship with the client is fairly new: then not so much.
  I’ve found explaining how we saw things at that point in time (try and explain our perspective), really helps - also to not get defensive: it’s hard to get angry at someone if they’re not trying to fight you back.
  When it comes to money, then implicit requirements can result in scope creep: therefore I’ve found it’s important to try and be more explicit with things in the start. It’s not like you can’t do ANYTHING that isn’t mentioned in the pre-written requirements, but for everyone’s sanity - agreeing that the written requirements is (as it is written) what will be built is sometimes the best you can do.

• Do you advise acquiring some domain knowledge or gain some understanding of similar projects, when the testers senses that the requirements are implicit?
  It can help but it’s not necessary. Past experience on projects and knowing how to ask good questions (cue @christian 's talk on do you know how to ask a question) are also useful here when you sense there are implicit requirements.

• Do you use example mapping and more team meeting to uncover more implicit requirements?
  How would you go about explaining implicit requirements to someone who is new to the project, who has not have been there from the beginning?
  *No I don’t use example mapping - I need to read more on that, thanks *
  But I do use examples extensively in discussions to make sure the team is on the same page
  I also book in a few extra team meetings to uncover implicit requirements.

• How early do you think we need to start implicit requirements testing in the testing life cycle? Do we convert every implicit requirement to explicit ?
  I think the earlier the better - even in feature discussion/kick-off.
  It’s good to make known implicit requirements explicit. But I don’t think it’s possible to make every implicit requirement explicit, as that requires an awareness of every implicit requirement that exists - which is very hard to have.

• How would you go about explaining implicit requirements to someone who is new to the project, who has not have been there from the beginning?
  WIth new people, I ask about their background (i.e. what they have worked on in the past) and the level of requirements they had worked with - I then use this to frame some examples of implicit requirements that were (probably) there on those projects, that they themselves probably tested against, but weren’t necessarily conscious they were testing against.
  From that point, I bring them to where we are on our project and the sort of implicit requirements we have now.