Whenever I was writing automation, I was always walking a fine line of having my test functions detailed enough that they let others know what the function did but not so long as to confuse.
I don’t feel like I ever struck a good balance between the two!
What kind of naming scheme do you use for your test functions? Is this something you decided yourself or was it set in stone when you joined the company?
Here’s a real life example.
I have a program with a UI, where you can enter a digital value or an analog value. If you enter the digital value, it should update the analog value, and vice versa. This means that two fields can easily infinitely loop to each other (digital and analog).
The field name is simply “C” (The meaning of “C” is clear to someone with domain knowledge)
The name of my test is:
TestConvertDigitalToAnalogCDoesNotRewriteItself();
Breaking it down:
Test: It’s a test. I always call test methods “Test”
“ConvertDigitalToAnalog”: The method I’m testing converts a digital value to an analog value.
“C”: c.
“DoesNotRewriteItself”: My assertion is that the value I enter will be the value at the end of the test.
So (and I’ve never actually written this down) my tests are written as:
“Test” “What feature/function/method/class/input I’m testing” “What action I will perform” and only sometimes, “What I want to see from the test”
Other tests from the same test suite:
TestInputToDigitalFieldOutOfRangeHigh() - out of range, when I write it, implies an error state
TestInputToDigitalFieldOutOfRangeLow()
TestInputToDigitalFieldWithNaN() - NaN is, of course, not a number. It also implies an error state.
Now here’s the thing. I never actually looked up or asked how other people do this. I just use what makes sense for me, so that the tester should know what is going on.
I think the longer names are important, I have seen people start to use conjunctions, which Brian shows us clearly you should avoid.
A naming convention that creates grouping is also useful, but I tend to want to use suite names to control groupings as well. So try to be consistent with the position you put the feature (usually first), and the critera usually last to get that to work. I find myself doing a lot of system testing, and I’ll sometimes go back and update the test spec title to better capture the “name-parts” I use in the code, since you spend more time looking at code than in the test management tool.
I mainly write unit tests right now so I do not know if this is applicable to integration tests / functional tests but here goes.
There are two main principles that I try to keep in mind. The first is to test the component as a whole instead of the methods, the other is to test (and document) the intent not the action.
This would for instance mean that instead of have a few tests like “TestCreateUserIndexOutOfBoundsException” there would be “TestUserRegistrationErrorRecovery” and in that test there will be plenty of actions that produces errors and asserts that checks the correct responses. There might be another which is “TestInputValidation”. This allows for a few things. If I have a component that is “UserRegistration” there will be a test which is the “UserRegistrationTest”, thus I do not need to add UserRegistration in the individual test because that is already documented in the name of the test file. The other thing is does is that when I change something I need to think if I have changed the intent of the component and update the tests. If I have not I do not need to add, change or remove tests.
From what I remember from Functional tests is that test execution is a thing and that you might have finer control over what tests that actually gets executed which would argue for finer control over asserts and not grouping them. But in that case I still would suggest a naming scheme in terms of “Will Always Run”, “Will Only Run if the this function have changed”, “Only for troubleshooting purposes if the previous tests failed in order to get a better understanding on what exactly went wrong”. Again the intent of the test here is documented with regards to execution.
If I were to continue to translate this to functional testing I would probably try to come up with a scheme like this in the case of a web application.
Group test according to User Missions (let’s use twitter as an example). Register User, Tweet (the verb not the noun), Retweet, Read Flow, Update Profile etc.
Each area will typically contain a few standard parts like “Basic”, “Input validation”, “ErrorHandling”, “Localisation” and maybe a few specific parts for that area alone.
So to get to the naming it would be Tweet.Basic, Tweet.Localisation. And then I would probably update test suites so I have one which is “All Areas Basic Tests” which I will always run for instance.
Lot’s of thoughts in one post. Good topic! 