Do you use parameters in automated tests?

(Gabriel) #1

Hi folks,

Quite often I hear at work that parameters in tests are a good thing - however, I don’t quite agree with that.

For me, a test case is a set of test inputs, execution conditions (though I would rather call it actions), and expected results developed for a particular objective. Certainly, an automated test case should have those elements (inputs, actions, outputs, and objective) clearly stated.

Due to that fact, I find myself willing to sacrifice writing flexibility in order to be able to understand each test case separately and visualize all their elements together – inputs, outputs, action, and objective. Parameters in test cases tend to separate the data from the test behavior itself, and that’s what makes me reluctant to use them.

What about you? Do you prefer your test cases with or without parameters?

P.S: I blogged a longer version of my opinion - feel free to comment here or/and there :smile:

(Teemu) #2

Let’s start with the different kind of views. First - some kind of parametrization is almost always needed. Why? Well - you have to develop your tests somewhere where you don’t mess up the real test environment. It is usually your local machine or virtual machine or you have some development server. In those cases the URL with web applications has to be parametrized.

Also at perfect world there might be multiple different kind of test environments. Again - URLs, maybe passwords etc must be parametrized. Of course you have to be able to version control those. So instead of cryptic command line options the parameters can be at the configuration file. And it can be version controlled.

Parameters are not evil. They are more or less mandatory in many more complex cases. But they must be under clear control.

(Gabriel) #3

@teemuvesala I completely agree with you that some kind of parameters is always needed - like environment IPs or browser to be used or login credentials. However, should those parameters be explicitly described along the automated test method body or should they be implicitly defined by some test framework abstraction?

The idea of letting environment/context parameters transparent to the automated test case would make me not list them as an input, as it’s something the test should not be aware of - it won`t appear on the test description or the test implementation. This approach would allow us to create a test to validate a behavior - no matter in what environment. I prefer that approach as it let me focus on the behaviors being tested.

However, we can also define the environment/context within the test case. This approach would force us to write different test cases for different environments - even if the behavior is the same - and that’s what makes me avoid this idea.

As I was looking for feedback on the idea of using (or not) parameters to simulate different input/output combination using the same test method (exemplified in the blog post), another question that needs to be done is: should the environment/context be parameters on test cases or not?