Are you in favour of formal test cases?

[rosie (Rosie Sherry)]

Are you in favour of formal test cases?

Bonus points for added community commentary, experiences and stories.

• Yes
• No

[Polls also available on X and LinkedIn)

[xbartels (Xander Bartels)]

what’s your definition of formal test cases ?. I mainly should call it test ideas in preparation of exploratory testing. How many times you have created test cases based on a specification / requirements document and then they change the specification/requirements during the implementation or overtime. (before delivering the feature)

[rickhough (Richard Hough)]

Limited, high level formal test cases assist conversations between the stakeholders and the development team. In regulated or critical systems telling people what you plan to test, then providing evidence of that testing matters. The old days of spending 3 months writing test cases has gone (I did that 30 years ago) but there needs to be a balance.

[ipstefan (Stefan Papusoi)]

I would like to choose it depends.
They can be good for non-testers who already know the product and consider this as a checklist.
It can be awful to provide them to someone to learn the application by testing using formal test cases.
They can be good for large scenarios(long-sequence testing) where there’s a need to note many details about configurations to be done ahead, data, states, multiple products, features, functions., and calculations to be made…
They can be awful in quick feedback loop processes.
They can be good for repeating the same thing to check it works in the same way.
They are bad at finding problems.
And so on…

I used to favor no test-cases.
But the way I see many companies operating these days(no testers, dozens of documentation files everywhere, hundreds of left-overs test data, no structured testing at all, randomly doing shallow checks) maybe some checklists in combination with Session Based Testing can be useful.

[james.elliott]

I like the ‘It depends’ stance as I think in my younger days, have a formal structure of test notes really helped me get in the tester mindset:

• What is the acceptance criteria?
• What is in/out of scope for this test session?
  *What are the known risks?
  *ect
  No that I kind of have these points engrained in all my testing activities, I really enjoy finding different ways documenting my testing.

Something I have been toying with recently is using a Miro board. I find that the flexibility of the different ways you can display data helps me to put in notes that are short and to the point and I can easily add in any kind of supporting evidence into a growing board.

Another one is Azure Data Studio notebooks, when I’m running database checks, its awesome having the ability to add in queries and the results into my test notes. The only drawback is that the flow is very linear, so if your doing a long test, you end up with a large amount of scrolling to get from the start fo the notes to the end.

I rambled on this more than I thought I would

[seasidetesting (Stephan Kämper)]

The answer is, of course, ‘it depends’ – as (nearly) always.

First of all, what is meant by the term ‘formal test cases’? I’d argue that typical unit tests (i.e., those written using Minitest in Ruby, JUnit, and friends) are formal tests (the fact that they’re written in a programming language qualifies them as ‘formal’).

Then there may be test cases that are executed manually but still may be called formal…

Either way, it depends somewhat on what product is tested. For medical hardware or software, I’d definitely want some formal aspects to be respected (to say the least).

[olly_f]

I’ll use test cases to make sure acceptance criteria are ticked off but no one looks at them as far as I’m aware so I can be pretty fast and loose about them tbh. I’m not a fan of test cases that are too detailed. They can take far too much effort to follow because they’re often overwritten and if they’re wrong it’s annoying to get to the end of understanding what they’re about only to see what they should say instead of what they actually are. Or could say because I’m doubting everything at this point

[maaret (Maaret Pyhäjärvi)]

In my three decades of testing, I rarely write formal test cases. When I write them, I write them in automation. But in most recent project, we wrote something which is kind of like format test cases, we just called them feature demo scripts. With those we set up prerequisite configuration and data, and explained in steps what to do to see a representative demo. We did these to support the whole team seeing end to end functionality.

In project before that, someone had written formal test cases and I inherited them and threw them away. I calculated that there was 5000 of them, and just to read them through once I would need to invest 11 full working days. They were not helpful, but hindering any good work we could do.

[hananurrehman]

Its a context driven thing.
I like to write test activities in a free form style which allows me to jot down my thoughts faster.
The way I see formal test cases is if they’re required in any legal or professional context:

1. You’re testing a software that has to comply with certain standards like ISO
2. The organization you work for is very document oriented
3. You’re testing software who’s outputs may be used in research work, court proceedings or investigations
4. You’re working in a contract and the client wants the documentation so he can use it later to do some testing on his own

[mirza (Mirza Sisic)]

A lot of valuable insight on this thread, personally I prefer a less formal approach as it saves time, but sometimes there is no way to avoid writing test cases - like in certain regulated industries you need old-school “test deliverables” as legal proof that a piece of software has been tested.

On the plus side, good test cases can serve as onboarding documentation for new testers, but if you have to write them, try to do them as efficiently as possible. Use shared steps, agree upon naming conventions, parameterize similar tests to reduce repetition, and to reduce maintenance, those sorts of things.

[johan.hoberg (Johan)]

Like many others have said already, my answer would also be “it depends”

Sometimes you just need “formal test cases” due to regulations, contracts, or other formal constraints, like already mentioned.

One aspect I have previously explored has been test cases as containers of knowledge/information, and why this is probably not an optimal idea in most cases:

Codifying Knowledge in Tests

This was an argument I heard at the time for why we needed to have test cases - documenting the knowledge. And for that I think test cases are inefficient.

[oxygenaddict (Richard Adams)]

They definitely have their uses:

• Compliance: When you need to test something in a specific way for a specific outcome.
• Smoke: Sometimes it is handy to have a defined set of tests that you can refer to when spot checking a feature for flames. These can also be handy documentation!
• Something that was really complicated and you’ll have to do it again one day (or use a wiki…)

If you are running test cases over and over then they are useful, but I’d question why you’re doing that.

For 99.9% of manual testing, I think you want something bespoke to the needs and risk of the story/task/epic that you’re testing.

[saviski6745 (Saviski)]

Formal test cases provide a structured approach to validating that the software meets its requirements and performs as expected.

They help ensure consistency, repeatability, and thorough coverage of the application’s functionality, and these especially become when different team members are performing testing and knowledge needs to be transferred over to different people.

It is however important to consider what test coverage you are looking to gain with any manual cases, and maybe aiming for 100% coverage is not viable for most testing projects. (especially if used alongside automated tests)