How to best automate regression tests?

Hi everyone,

I am a lone tester within the software team of a geochemical consultancy. We build a commercial release of our software roughly once every 2 months but before it gets the OK I have to manually regression test it for the maximum of a week. When I started in this role there were a bunch of manual but outdated test scripts that went down the ‘happy path’ of parts of the software and I was told that I would need to update them. I was also informed that at some point having to spend 1 week going through these scripts over and over again doing the same thing would get quite monotonous and that I may want to look into automating them. Not knowing any different (this is my first testing role and I came into it with no experience or knowledge) I have spent the last 6 months or so using CodedUI click and record to automate a fair amount of these test scripts. However it has gotten to the point now where the maintainability of these automated UI tests is probably more of a cost than a benefit as they break more often than work and I can spend a lot of my time fixing them when I could be just testing the software. I now know that automating UI tests is not recommended due to this issue.

My question is: is there a better way for the ‘happy path’ regression tests to be automated?

Thanks for any help.

Adrian

The answer will depend on a few things:

  1. How good are you in programming?
  2. Do you have access to the code you are testing, or the programmers (The programmers are in the best place to help you create tests, as they know what they have done)?
  3. Do you really need to run the same “happy path” tests for every release?
  4. What is the software made from? (i.e. Does it have an API? is it a windows application? How does the UI (front end) interact with the nuts and bolts (backend)?)

To clarify on point 3 (Do you really need to…?), a quick discussion with your team could prevent the need from doing days of repetitive and boring click-work with no visible benifit except for keeping you busy. Say they are working on feature “B”, which has no impact on feature “A”. Why would you spend your valuable time running extensive tests on feature “A”? So instead of doing all the tests, you could only do the useful tests, or just focus on the new functionality.

1 Like

Hi @brian_seg, thanks for the quick reply. To answers your questions point by point:

  1. I don’t know know any programming/coding language or how to read one
  2. I could access the code most likely but due to the above limitation it wouldn’t be much help to me
  3. For the 2 monthly release regression testing there has been so many changes merged in that the expectation is that I am regression testing as much of the system as possible as the developers aren’t always aware a change in one area may impact another
  4. The software is made from C# and is a Windows application.

Thanks

Adrian

Hi @adrianjr …I am also from your category . but we are two people in Qa team . My manager said to start automation so we have started using selenium on c# . we don’t have any experience in coding but still we are able to excute the scripts …I think when you will follow the tutorials then it is not difficult for you to run the test cases …you can start from beginning like login page then proceed with the more details …hope this would be useful for you

If you have any budget to spare, I’d recommend looking into a tool Ranorex which should allow you to automate easy things relatively easy, yet for the complex things it is written in c# which would leverage your team expertise.

Ig you don’t have budget to spare, I’d look into robot framework and either autoit library or sikulix integration.

Since you don’t have experience programming, I’d look into becoming an “automation godfather” from among other programmers in team - people who’d accompany you in finding solutions to problems that you will encounter.

Hi @adrianjr

There are a number of thoughts in my head now, so I will outline them here:

I would look at this from a team perspective. As a ‘lone’ tester working with a number of developers, your focus has been on manually running tests, and it would seem that the testing lies just with you.

  • Should testing be something that the team tackle together? (Who tests when you are not there?)
  • How much visibility do the team have of your tests?
  • How much visibility do you have of the unit tests?
  • How thorough are the unit tests?
  • How much is being tested within the UI that could be better tested in the back-end to reduce reliance on front end tests?
  • Are all your tests based on ‘happy-path’?
  • Do you perform negative tests in your Regression pack?
  • With the pressure to look into automation, is this something that you want to do? Not every tester wants to write code, but there is more expectation nowadays that coding is part of the job.
  • If you want to get into writing test code, then C# makes sense as you can ask to pair with developers to help you with learning the language. As well as helping you learn, it will help developers to appreciate the testing that you do.
  • What tool should you use? You mentioned selecting Selenium Webdriver (for your UI level tests) and there are a lot of useful training courses to help you get started with a framework. But if you have tests that are at an API level, then you’ll need to look for another tool e.g. SoapUI.

I would look at how you can work with your developers to guide you, but try not to create a framework that is too cumbersome. I had an issue a few years ago when developers created a framework for us, but tackled it as though it was going to be live code, so it was too complex. All you need is something that you can work with - good enough to do the job.
Tests will fail, and you dont want to be spending hours trying to debug them, whilst the team are waiting to find out if the failure is a genuine bug or just failing test code.

Hope this helps,
Steve

1 Like

Possibilities of Failure with Record and Playback is more. Suggest to go with building own framework and start scripting from scratch.
Also, running of all tests as part of regression may not be the right approach for any projects. request to discuss with Developers to understand the changes and perform test only on those areas.