I am a lone tester within the software team of a geochemical consultancy. We build a commercial release of our software roughly once every 2 months but before it gets the OK I have to manually regression test it for the maximum of a week. When I started in this role there were a bunch of manual but outdated test scripts that went down the ‘happy path’ of parts of the software and I was told that I would need to update them. I was also informed that at some point having to spend 1 week going through these scripts over and over again doing the same thing would get quite monotonous and that I may want to look into automating them. Not knowing any different (this is my first testing role and I came into it with no experience or knowledge) I have spent the last 6 months or so using CodedUI click and record to automate a fair amount of these test scripts. However it has gotten to the point now where the maintainability of these automated UI tests is probably more of a cost than a benefit as they break more often than work and I can spend a lot of my time fixing them when I could be just testing the software. I now know that automating UI tests is not recommended due to this issue.
My question is: is there a better way for the ‘happy path’ regression tests to be automated?
Do you have access to the code you are testing, or the programmers (The programmers are in the best place to help you create tests, as they know what they have done)?
Do you really need to run the same “happy path” tests for every release?
What is the software made from? (i.e. Does it have an API? is it a windows application? How does the UI (front end) interact with the nuts and bolts (backend)?)
To clarify on point 3 (Do you really need to…?), a quick discussion with your team could prevent the need from doing days of repetitive and boring click-work with no visible benifit except for keeping you busy. Say they are working on feature “B”, which has no impact on feature “A”. Why would you spend your valuable time running extensive tests on feature “A”? So instead of doing all the tests, you could only do the useful tests, or just focus on the new functionality.
Hi @brian_seg, thanks for the quick reply. To answers your questions point by point:
I don’t know know any programming/coding language or how to read one
I could access the code most likely but due to the above limitation it wouldn’t be much help to me
For the 2 monthly release regression testing there has been so many changes merged in that the expectation is that I am regression testing as much of the system as possible as the developers aren’t always aware a change in one area may impact another
The software is made from C# and is a Windows application.
Hi @adrianjr …I am also from your category . but we are two people in Qa team . My manager said to start automation so we have started using selenium on c# . we don’t have any experience in coding but still we are able to excute the scripts …I think when you will follow the tutorials then it is not difficult for you to run the test cases …you can start from beginning like login page then proceed with the more details …hope this would be useful for you
If you have any budget to spare, I’d recommend looking into a tool Ranorex which should allow you to automate easy things relatively easy, yet for the complex things it is written in c# which would leverage your team expertise.
Ig you don’t have budget to spare, I’d look into robot framework and either autoit library or sikulix integration.
Since you don’t have experience programming, I’d look into becoming an “automation godfather” from among other programmers in team - people who’d accompany you in finding solutions to problems that you will encounter.
There are a number of thoughts in my head now, so I will outline them here:
I would look at this from a team perspective. As a ‘lone’ tester working with a number of developers, your focus has been on manually running tests, and it would seem that the testing lies just with you.
Should testing be something that the team tackle together? (Who tests when you are not there?)
How much visibility do the team have of your tests?
How much visibility do you have of the unit tests?
How thorough are the unit tests?
How much is being tested within the UI that could be better tested in the back-end to reduce reliance on front end tests?
Are all your tests based on ‘happy-path’?
Do you perform negative tests in your Regression pack?
With the pressure to look into automation, is this something that you want to do? Not every tester wants to write code, but there is more expectation nowadays that coding is part of the job.
If you want to get into writing test code, then C# makes sense as you can ask to pair with developers to help you with learning the language. As well as helping you learn, it will help developers to appreciate the testing that you do.
What tool should you use? You mentioned selecting Selenium Webdriver (for your UI level tests) and there are a lot of useful training courses to help you get started with a framework. But if you have tests that are at an API level, then you’ll need to look for another tool e.g. SoapUI.
I would look at how you can work with your developers to guide you, but try not to create a framework that is too cumbersome. I had an issue a few years ago when developers created a framework for us, but tackled it as though it was going to be live code, so it was too complex. All you need is something that you can work with - good enough to do the job.
Tests will fail, and you dont want to be spending hours trying to debug them, whilst the team are waiting to find out if the failure is a genuine bug or just failing test code.
Possibilities of Failure with Record and Playback is more. Suggest to go with building own framework and start scripting from scratch.
Also, running of all tests as part of regression may not be the right approach for any projects. request to discuss with Developers to understand the changes and perform test only on those areas.
Based on your information I can see that you are in crunch for time, being a lone tester and also trying to prioritize your testing. Your situation seems like a similar problem testers have when working in start up companies; where there are a lone tester as well in many cases. If I were you this is what I would do-
Firstly, I am a big proponent of Session Based Exploratory Testing (SBET). These are time boxed uninterrupted testing sessions focused on a particular goal (module, feature, scenario). You can find more details about it here - How I do Session Based Exploratory Testing (SBET). This is especially good for your situation where you can get quick feedback on the application instead of writing detailed test cases. You get more flexibility in exploring the product
Secondly, in terms of automation, once I get
A better idea about the product features
Types of bugs uncovered after doing SBTM
Identifying high risk areas
Identifying mundane tasks in manual testing which are time consuming
Having set of smoke test cases based on experience testing the application
Then, I can invest in automation.
You mentioned you are coming from a non-technical background. This is fine as there are so many codeless UI automation tools available to you to help in doing stable automation. Take some time to evaluate some popular tools people are using. I myself work on a codeless UI automation platform called Testim and you can read more about it here. Once you pick a tool, just have some high level stable tests testing the critical functionality of the system/happy path running consistently. Then slowly you can expand on this.
It is good to have SBET and high level automated tests running in parallel, this will give you good coverage of the application.
I would say after a month or two of getting to know the product, then you can do some time boxed experimentation with different tools that are available for automation to see what fits your needs.
Once you do the above, you have set the stage for more possibilities and more testing.