Do you add code documentation to your automated checks?

We have a new article out from @yaniqued in which she shares her thoughts on the value of code documentation:

I really liked the points @yaniqued made in this article and it made me think more about the value of code documentation when done properly.

Iā€™m curious to know what other peopleā€™s attitudes are to code documenation. Do you use it? Do you not? If not, why and how do you mitigate the risks of unclear code?

2 Likes

I use it for every project, in particular one area I find very helpful (although not specifically inline with code) is a good readme.

In my org where we have a microservice architecture and multiple small standalone applications I find a readme very useful for jumping between projects as a nice ramp up. If the person originally implementing had to do something that was a little different than standard to solve a problem, having a good readme makes this so much easier to communicate from the start. Also at a readme level having a list of tags and purpose for those tags also helps visibility.

Along with that docstrings on page object methods or helper classes are also a great help.

Comments I am less verbose with (but still use), as a solid framework (page object model or otherwise) with well written class and method names should read pretty clear, so they are only added when there is a feeling something could be miss-read or needs a little extra clarification.

I like to look at it in the good citizen approach, I try to leave things as good or better than I found them and ideally how I would want to find it in a years time when I could really use that extra context picking it back up after so long :grin:

2 Likes

This captures my thoughts exactly! I want to leave things a bit better than I found it, and comments are helpful for everyone. While writing comments, I often see ways to improve the code, or to better verbalise intent.

I also love a README, and have tried out a writing style that focusses on ā€œthe readers needsā€. So the sections focus on questions theyā€™d probably ask or the thing they need to do.

For example:

  1. Running tests
    1.1 Run them how theyā€™d run in CI
    1.2 Run them all locally
    1.3 Want to debug or run some tests?
    1.3.1 Run a file
    1.3.2 Run a single test
1 Like

I read Clean Code by Robert Martin recently and comments are not recommended.
Off the top of my head, the main concern is that the code get updated but not the comments and eventually there is a discrepancy between the code and the comments.

1 Like

For helper functions and shared libraries used as part of the framework, yes. In the tests themselves, as a rule, no.

If the test itself isnā€™t obvious, given you have some domain knowledge, without documentation, then it probably smells and needs refactoring.

Usually, any place that one wants to add a comment is a sort of code smell - in most cases it can be resolved by introducing a variable, extracting a chunk of logic into a method or refactoring the code so that it will be clear. There are some rare cases where comments are the best way to convey a message, usually around intention or limitation that forces a seemingly wrong behaviour.
I think it was Kevlin Henney who said that if a person canā€™t express themselves well in one language they are fluent in (code), what makes us think they can do better in another (English)?

Another problem is that comments tend to deteriorate over time - people move lines without the relevant comments (because programmers are conditioned not to read comments), business reasons that were extra important in the past change, and constrains that were once insurmountable are solved by installing a newer version of a library.
In that aspect, docstrings are even worse, when signatures can be changed by the IDE refactoring capability without even opening the file (and the code review wonā€™t necessarily have the affected lines displayed on screen).

Good naming, though, is crucial.

Although this is all my opinion on the topic only. Iā€™m with @mtest . No code comments is a code-smell, too many comments is a code-pong. We seem to be talking about documentation as well as code readability here, but Iā€™ll moan about the latter first to inform on the former.

Iā€™ve been a coder for so long that I once had to double take when I spotted a duplicate semicolon in someoneā€™s code from 5 meters away from the screen; too many comments break the code flow you see. And for me, the layout and code style are more important than code comments. Code has shape. Because good code is beautiful code, indentation and all, after a while actually invoke the pattern of the code fragment in the reader. Code that flows is quicker to read and grasp.

I do have 2 magic code look bullets, which Iā€™ve not changed for a dozen years:

  • Naming of things
  • Strategic comments

Long code blocks are a code smell, they slow code comprehension and hide branch complexity, in much the same way that long comment blocks break your readerā€™s train of thought. Sometimes that reader will be you 10 years later. Try not to mix code and comment blocks, every function should fit into one screen, 30 lines max. (Remember data is not code.) Long functions can even tend to have more comments in them, ask yourself why.

Code never lies. Cleanly written code is very easy to read; code with big comment blocks is often there to explain code that the coder was hesitant about and that will invariably reflect in the code smell. This is where strategic comments come in. Strategic comments are about commenting to explain why you adopted a specific strategy .Your code strategy should be obvious to a reader anyway if you are following indentation and style rules, patterns like loops for searches, filters, builders and other patterns should all start to look like their kin if everyone codes neatly. You will be able to read clean code almost like spotting the girl in the red dress in the Matrix. The strategic comment is how you tell the reader who you chose a specific pattern - often itā€™s because of an externality like a API you consume that is either undocumented or behaves in an unexpected way. Never ever document ā€œthe codeā€, a comment on ā€œthe codeā€ has that danger of miss-leading someone later when the code fragment getā€™s copy-pasted.

Naming of things is super important, if you name a function according to some set rulesā€¦ just look at the language library that you use, you know what those functions do without reading their documentation donā€™t you? Thatā€™s because the functions are well chosen names, and since you have to be sure to break long functions into shorter ones where possible to satisfy DRY Don't repeat yourself - Wikipedia you will end up with more smaller functions, so come up with naming conventions that work. Pro Hint, functions with 2 verbs or two objects in them are often evil, because they break LISKOV Liskov substitution principle - Wikipedia which can even be applied not just to objects but also to methods or other encapsulations of a behaviour.

Basically if my code follows patterns and rules, itā€™s like a highway where all the road signs have an easy to grasp layout and I can move quickly from place to place. In the end I want comments in 2 places:

  1. Helping people use the code indexing tool by including a tooltip sized doc-comment to go with each function when I hover over it.
  2. When Iā€™m telling the reader about an unexpected behaviour like a zero/nonzero based index for example.

I agree with what @conrad.connected says. Some kinds of comments are a good idea; many kinds of comments are not. A blanket all/none approach I think is a bad idea in reality.

A rule of thumb for me for comments is that they contain a word such as because. For me, the purpose of a good comment is to say why (or why not) something happens. Good structure and good names get you a long way there (and remove the need for lots of low value comments). But itā€™s inefficient to force later programmers (including yourself) to reverse engineer the reasons for things by reading lots of code.

For instance, we do thing X here because it needs to be done before thing Y can happen.

I donā€™t think that comments are necessarily a code smell, but like everything we do they have a cost as well as a benefit. (More to read, can get out of date, etc.) We should limit ourselves to the cases where comments come out best from a cost/benefit analysis, and rely on things like good structure and good names where they are better at e.g. helping a reader to understand the code.

1 Like

The essence of the article and comments would seem to be ease of maintenance and reuse. For someone else, or for yourself a few months down the line. To make code easy to maintain (which is a separate topic from low maintenance sensitivity!), it should:

  • Be clear:
    • Names describe purpose clearly for all (so usually no TLAs / FLAs / abbrevs)
    • Consistent (coding standard, including naming standard)
    • Short (I used to say 30 lines, now it is usually no more than 10 to 15; abstraction is a great thing)
    • Moderate complexity at most
    • Commented only where it adds something (often about the why, sometimes what / how, esp. method + params + return description)
  • Be documented at the higher level also:
    • File / class purpose documented at the top if needed
    • Functional and/or technical design
    • Readme

Once I was trying to refactor the code of a tool that I did not write (FitNesse!). It was hard. At first I blamed the very short methods and lack of comments. Later I realized that it was only the overall design, so across the files, that was unclear. The code in each file was actually very easy to understand and maintain (good naming, short methods). A lesson that I have benefited from ever since.

PS: Just in case: TLA = Three Letter Acronym, FLA = Four Letter Acronym. :grin:

1 Like

I work in C#, so I use the XML documentation and tie in the documentation with Sandcastle, so my comments in the headers become the documentation in a website.

Itā€™s far from perfect and I know I could clean the whole thing up a lot, but since the application I test is probably about a year from being closed down, thereā€™s no need to keep working on it (and my CI/CD pipeline for said automation and associated documentation has been switched off, alas - weā€™re deprecating and shutting off anything thatā€™s not necessary. Not my decision, sadly).

Maybe Iā€™ll get to work on automated tests for something modern one day and wonā€™t have to deal with the end-to-end UI tests on antiquated code any more.

1 Like

I use Selenium IDE (browser plugin) for automation. Every step has a comment that, if present, is displayed in the script instead of the command. That documents each script it pretty well, and each test has its Jira ticket number at the start, so people can see why itā€™s included. Iā€™ve also written up my notes on the techniques used to get it to get it to run in a data-driven stylee, and acres about how to make it run reliably with Angular.

All code should be written clearly so it doesnā€™t require documentation. We have naming conventions and do code reviews etc ā€¦ the only comment I could just find now is ā€œ// code needs to be comment out after Fix of ticket:13600ā€

To be honest, I donā€™t mind a few comments, but if I can write code that is readable for everyone I donā€™t need too.

I agree on the second part XD
I do live in an environment where we spend time in refactoring and refactor the refactored code even. So for us, no code comments is fine for us if itā€™s clear.

1 Like

Something just reminded me now of a pain for all test code - test code often has to document incorrect product behaviours. I think there is a special burden on test code documentation. Because unlike production code, it will often by triaged by someone who never wrote nor owns that test code. This often means that test code get littered with environmental maintainer comments to direct the user to find where the environment is incorrect, not where the product is faulty.

We have not talked about test metadata in comments either. Code decorator functions which @skip or ignore a test or simply control how a test will run when against older versions where a behaviour is not present yet, or when a behaviour is removed. So, lots of excuses for more documentation in test code, which means being more systematic about use of comments, sure.

This is true though! Iā€™ve seen that. Same as our ā€˜wait until ticket X has been fixedā€™.
So sad ;(

1 Like

Well-documented code helps developers, including the original author and others who might work on the project in the future, understand the purpose, functionality, and structure of the codebase. This can be especially important when dealing with complex or unfamiliar code. Embarking on a journey to your dream residency? :rocket: Letā€™s talk about the true trailblazers ā€“ the ā€˜Top 2 Residency Personal Statement Writing Servicesā€™! https://easyreadernews.com/top-2-residency-personal-statement-writing-services-that-are-worth-try/ Crafting more than just words, they sculpt your aspirations into compelling narratives, ensuring your journey stands out in the competitive landscape. Like architects of ambition, theyā€™re here to help your dreams take shape.

1 Like

Welcome @bruniomiles to the sweetest Software Test community that has brought the best ideas and content sources in Software quality into just one place.

Your assertion just got me thinking more deeply. Do you see code and comments as having different purposes I wonder? Iā€™ve never thought of them as being the same purpose since one is essentially opaque to the interpreter, along with all of our fancy structural and naming conventions too.

Once again welcome, really good value point.

1 Like

I try to only add comments to explain why Iā€™m doing something, not how. For example when integrating with a third party the API didnā€™t work how we expected and the code I wrote seemed, odd, so I commented to explain my design choice (plus of course explained in the description).

The same applies to tests. My test should be readable. If the behaviour Iā€™m testing is odd, Iā€™ll explain that. Also sometimes a test case is a proper niche edge case, I might like a more verbose explanation.

When onboarding interns I did have them writing Arrange/Act/Assert. I thought the merit there is more to the writer than reader. What is your test? What is the output you expect? Wait you have half a dozen asserts, maybe not ā€¦

(Note this is mainly written from experience of unit tests and some component tests).

1 Like

Inspired to keep trying I am now @oxygenaddict . Iā€™m also a fan of things like Arrange/Act/Assert as a flow for code, and other ways of making your code actually state itā€™s intent like a story that clearly separates the setup, main-body and finally the ā€œchecksā€.

Itā€™s just hard to stay disciplined, and Iā€™ve often found that forcing people to write a small list of test steps at the top of a test helps loads. Almost like a template. Once the code gets reviewed, you can delete the comment-block with the steps, because the code should do what you said you were going to cover.

2 Likes

I really like the idea of removing comments after a code review! Not something that Iā€™ve thought about before.

2 Likes

Iā€™m a firm believer in writing code using naming conventions, patterns and style that ensures that the code is human readable, to the point of getting pedantic about function naming even in ways that eliminate a need for comments. Code never lies, comments are thus very much disposable.

Thatā€™s why I prefer more verbose code that anyone can read over terse and optimised code that not everyone understands. Advanced code that uses things like maps and reduce and LINQ are chances for people to learn, but should not be over-used if native code will do the same job, because all code maintainers need to be able to be comfortable with your test code. That said, code documentation metadata , and in comment links to help people find supporting spec docs is super useful too.

2 Likes