Tools & ideas to represent and document testing framework


I’ve been looking around to find ideas about how to document the automated tests from project. There are multiple layers:

  • both UI and API tests
  • test reports generated
  • test data setup for each environment
  • test credentials loaded for each environment
  • running tests in pipelines (Jenkins)

I want to represent a high level structure of things how they are linked to each other.
Does anyone have suggestions for what tools to use to do this? or maybe some example of such high level representation?

for example this is Test Automation Framework: Design - Webkul Blog an example strictly for Test design

An opinion from someone who love to write documentation will follow, this is an opinion only.

…the moment you document something, you start to nail it down - implicit knowledge backed by intuitive flows are your best documentation. What am I saying?

  • documenting multiple unrelated things like UI and API tests in one place implies they are connected to each other, they probably have no relationship, so don’t try to force one to exist, it will merely tire out maintainers of what should be 2 separate documents in a wiki
  • generated test reports, are well, just generated, nobody gives a rats about a report that is a week old unless you are in a regulated industry, so publish only the latest report n a prominent easy to find place, and be done with. People who need the report will find it, just make sure it is easy to find and move on (unless you are in a regulated industry, in which case hire someone to engrave them onto stone for you)
  • test data setup needs to be scripted and in version control, if it’s hard coded, or hard to modify then it presents a millstone to change in your product test strategy in which case binning it or forcing people to re-invent it might actually be better than having it fixed and completely immobile (unless you are in a regulated industry, in which case a single stone tablet copy will be good enough documentation)
  • test credentials like all credentials must be in a secure vault and need to know only - do not other changing them regularly, but do ensure that even if they did leak, that nobody could in fact use them maliciously from outside of your network (don’t use certain password vault vendors, who shall not be named.) Do not put them on the wiki, and try to keep them out of code -NEVER store them in jenkins, jenkins has a special key store for this reason
  • pipelines are terribly hard to understand, sorry, but documentation of these will always suffer from rot (see my first point). My only remedy is to never ever shame anyone who asks for help understanding the pipeline

That said, write documentation but only enough to guide people to the actual sources of truth, not the detail. Often documentation is like a history book. It’s OK to document something well, so long as you know that a year from now your documentation is only going to be used to describe the logic behind the old system that got replaced when the company got bought out and you got forced to swap Jenkins out for TeamCity or something similarly geological. When I document I try harder to document the why, and where, than the how.

1 Like