How do you know what to test?


(Rosie) #1

I’ve put this question out to Twitter too, as the responses can be ever so useful.

When given something to test, how do you know what to test?

What guides you along the way?

What resources do you refer to to help?

Who do you go to for support?

I’m asking this as I know it’s such a common question and it would be really useful to have an epic discussion on it.


(Sarah) #2

Every time I test something, my starting point for knowing what to test (first) is closely related to another question: what don’t I know about this? This second question has many parts:

  • What don’t I know yet?
  • Do I know who knows? Or where to find the answer?
  • Can I use the application already to find out what I don’t know?

Once I have tested some, and gathered some answers to the above questions, there are additional related questions to ask:

  • What assumptions have I made at the moment?
  • Are there any newly discovered gaps in knowledge?
  • Is it already useful to have a second opinion on some of these gaps, or on some of these assumptions?
  • Can I give feedback already about something: how certain am I about some specific piece of functionality?

These sets of questions usually take me a long way already.

Resources I use:

  • OneNote (for notetaking)
  • Documentation (user/technical/…)
  • TestSphere cards (for defocusing, and sometimes to organize my mind/strategy)

Support:

  • Everyone in my team that I think can provide insight, or a second opinion, or guidance in prioritisation, or…
  • Sometimes co-workers that are outside of the project (usually as a sounding board)

(David Luu) #3

On a related note, I was thinking of this when I was just casually pondering that question one day: https://autumnator.wordpress.com/2015/02/23/what-the-fk-to-test/


(Stefan) #4

Good starting point.

I’d add:
What is that ‘something’ given to test. Can I see it? Can I ‘access’ it? Does it have a name/description? Is it a thing, a person, a thought process or something else?
What’s the purpose for me having that ‘something’ to test? Is it to play with it, to give feedback, to be part of a lawsuit, for compliance, for my information, for auditing, for releasing a product, for firing someone of incompetence, for a product purchase…
What part of that ‘something’ is of interest to the ‘someone’ asking me? Is it user facing, technology, development architecture, business workflow, optimization of testing, costs, efficiency, performance…


(Miranda) #5

I try to identify things that are the same, or similar, and things that are different. Building off of what Sarah replied, looking at the existing application or a competitor application.

Are there different roles with different access? Do different users have different views? For example, an online menu could have a gluten-free version, which only shows the items marked “gf.” Or applying a senior citizen discount on the price.

Are there thresholds to consider? For example, first time user has to accept a EULA. Failing login attempt three times locks the access. Quotes over $100,000 require manager approval.

Are there any date implications? Does something expire in a year? If so, what happens then? Is the cost or refund prorated for a partial year? Is there a grace period? What is the date format? (International dates are stored DD/MM/YY, versus U.S. of MM/DD/YY.)

Covering CRUD - Create, Read, Update (Edit) and Delete. When uploading files, what if nothing changes? Does the app take the time to process all of the imported data if it doesn’t need to?

Are there any time-related processes? For example, report aggregates run overnight? Bills generated at the end of the month? Tax documents generated at the end of the year?

Boundaries. For numbers, consider null, 0, 1, 1+, fractions, negative numbers, whatever the limit is, and limit + 1.

Any special characters (like El Ñino) in the text?

State changes. Toggling something On and Off. Do states persist or reset? For example, how does an expired account become active again? What value is used if something is toggled back On? Does it revert to default or was a custom value saved and is now displayed?

Looking at values in the database. Are numbers stored as strings? If so, how is sorting handled? (Strings would sort like “Part 1”, “Part 10”, “Part 2”…) Are numbers stored at floating point? How much precision is retained?


(Christopher) #6

So, someone has asked you to test something. What do they want to know?

They might want to know if the SUT still works after some code changes - might suggest testing some functionality.

They might want to know how the SUT performs - might suggest some load and performance testing.

Or the might want to know how usable/logical the SUT is - might suggest some usability testing.

Understanding what the stakeholder, that asked you to test in the first place, wants to know/learn from your tests, may help you determine what to test.