30 Days of Agile testing, Day 12: Test documentation

(Heather) #1

For day 12 of 30 days of agile testing I’ve had quite a varied range of experiences across different companies related to documentation.

My previous company our test documentation was really detailed, too detailed in the beginning. Steps for a login screen test case, for example, would be:

  1. Navigate to page.com/login
  2. Enter username “User1”
  3. Enter password “Password1.”
  4. Click “Login” button

Then there would be a test case for a user not in the system, another for empty login credentials, one for special characters.

We moved away from that after a while to a move checklist style of guidance written as a comment on the user story. Going back to the login example,

  1. Check existing user
  2. Check non existing user
  3. Check empty fields

When I had checked these I would update the comment to add in what I had tried and that the expected results were achieved if that was the case. This method seemed to be easier for the whole team to follow and freed up more time to test than writing documentation.

In my current job, the only test documentation I have is my automation suite and my terribly written notes from exploratory testing. Our user stories have no acceptance criteria but occasionally I’ll have some attachments from our UX person of what the UI should look like. Back to the agile manifesto “Working software over comprehensive documentation” I think we’ve taken this to the extreme but I’m not sure how to best document in a continuous delivery environment. I don’t want to write documentation for the sake of it because I’ve learned already that it’s just a massive waste of time.

What test documentation do you have? Do you use tools like Zephyr or TestRail? Would improving it be easy or do you think you’d need more buy in?

Roundup of 30 days of Agile Testing
(Chris) #2


Coincidentally, I started a to write a post about test documentation earlier today, then ditched the draft. After seeing your post, I changed my mind…

We have a COTS system, which we’re about to upgrade. The previous documentation set for UAT has become tired and needs an overhaul. Whilst waiting for answer about the current orgs project and document framework, I thought I’d have a look around Ministry of Testing website.

I couldn’t find much in the way of document conversations or templates and I wondered whether other testers feel there would be a need for a ‘Ministry of Testing’ set of templates. Nothing complex, but something that would help small projects get up and running quickly and something that could be added to…

WRT to your post, I’ve done both. E.g. detailed test scripts and checklists. Detailed test scripts aren’t much fun to write and execute. This feels like the bad old days of 1000 step test scripts. They offer no opportunity to explore. They typically direct the tester through a rigid process. Plus, they are a pain in the butt to maintain.

I like the checklist approach. They are easier to create and the tester has more freedom to deviate and try ‘stuff’ the rigid approach wouldn’t accommodate. IMHO (and don’t shoot me down), using checklists for a given piece of functionality means the tester spends more time testing and less time stating the expected result for the tiniest of steps.

(Heather) #3

I guess documentation is a bit like Marmite :stuck_out_tongue:

(michael) #4


Test documenation is rather important but I fear its maintenance can get too heavy.
I use both approaches : check list oriented during user story estimation.
I briefly warn about the scope or the kind of testing technique to use (depending on the coverage level wanted).
In the other hand, we write more detailled steps and expected results when filling a bug reports.
As acceptance criteria are supposed to be met when the piece of code is delivered, bug reports may become a good indicator of software quality.
The key is the keep documentation consistent enough so that anyone can have the latest version implemented in mind.
This is maybe only true for Agile environments where a lack of documentation is less important than a lack of communication.