Software Engineering in Automation

(Jack) #1

Speaking to interview candidates on the phone, or reviewing coding test solutions, I am struck by the disparity between practices we expect of our fellow product engineers compared to what we follow for test framework code.

I had a look over the courses and material here on MoT and there isn’t much guidance or talk on the subject.
https://www.ministryoftesting.com/dojo/lessons/extra-extra-automation-declared-software-paul-grizzaffi?s_id=40208 – seems, from the description, to touch upon the subject.

Quote from recent call - “unit tests are for developers. It is not industry standard to test our [the test framework] code”

What do others think?

Do you unit test your core framework code? Is it version control? Do you use feature branches, code reviews, etc… to make changes?

If no to any of these, why not?

0 Likes
(Ola) #2

My general advice for test code is that it should be part of the production code and if that is not possible at least treated equally. As an example:

Lets say you have versioning and releases for your production code 1.0, 1.1 and 2.0. You need to have test code versioning and releases that either are 1.0, 1.1, and 2.0 or something of the likes TC 1 -> 1.0, 1.1 and TC2 -> 2.0. The simple reason for this is that if you change the code under test between 1 and 2 your test need to change to. If you have a critical issue with version 1.1 and you want to fix and test those you have a problem if your tests do not map to your code.

Then the testing the test code. I would say that your production code is in most cases a good enough test for your code. I.e. if your code misses bugs or reports issues that is not issues. Those are the same as a test is failing. That being said there is a test coverage technique called Mutation Testing that could be added to your tests that can help uncover when your test code do not cover production code. https://en.wikipedia.org/wiki/Mutation_testing

Also if you are working more in the end to end space with your tests you probably should have some unit tests of your code since those typically tend to have more logic in them that you could test. Same goes for all helper functions basically. Just to save time when people are doing changes in shared code that might alter the quality of your tests.

Finally how to hire these people. You basically want a person that is 30% tester and 70% developer or something like that. And in my experience it’s easier to get someone who is 30% tester and 0% developer, which I think is the reason for this topic.

Great topic!

0 Likes
(Jack) #3

Some interesting points and nice to see hear from another in favour of more SE in automation. I am not sure I quite agree with the production code being good enough to test the test code, but I guess it depends on what code you mean.

As for hiring, sadly it is difficult. It was the spark for the topic, but not the reason. I just feel SE practices for automation are not really given much space. There is a lot on how to get started in coding for testers, maybe not so much for those who already have experience and promoting good SE in the domain.

Interesting point on percentages. I would count myself as 100% tester and when I write code for testing (tools, frameworks, etc…), I do so as a 100% software engineer. But maybe you meant something different.

Thanks for sharing thoughts. Sometimes feels lonely as a Software Engineer in Test. Stuck between worlds :slight_smile:

1 Like
(Ola) #4

What I mean with the percentages is how much of your time you know of that domain. I.e. Test Strategies, Test Techniques, Test Methods, Test Tools vs. Design Patterns, Software Architecture, Version Management, Build Tools, Development Tools and so on. And of course ideally you should know it all. But given limited time and energy I would try to look for persons that have a little more than basic understanding of test and a very sound understanding of software engineering practices.

You may feel lonely now, but you are part of a growing field.

1 Like
(Swetha Reddy) #5

After completion of manual testing, everyone would like to learn automation testing, but we need to have an interest in automation and basic knowledge over it. As automation is the trending future technology which is to be learned by everyone but need to select the right way or path of learning it. I suggest W3Softech is a leading software testing company which helps in the learning of automation testing and outsourcing services also.

0 Likes
(Alexander Dunn) #6

There is no clear separation of test and production code at my company.

Where possible, end to end tests, both UI (Selenium) and API are written in the same language (C#) as the code they are testing. They are checked in to source control via pull review by the developers writing the production code.

Where frameworks have been written to support testing e.g. for case generation, unit tests are written to check functionality. Given how rarely this code is changed, these unit tests are generally not run as part of builds although I’m starting to think they should as my team grows.

The key point for me is that developers are responsible for general maintenance of tests - if a functional change breaks tests, modifying the tests is the responsibility of the developer concerned unless the change is significant enough to specify a system test change backlog item separately.

We only have a handful of additional non C# (postman) tests running as smoke tests.

For me, minimising the friction for running and fixing tests during development is important.

I have always set myself the goal of having my test code at least as well engineered as the code it is testing.

1 Like