Help with my new team

šŸ—„ļø Archive
,
1

Recently I left a big company and I joined a small one (not a startup but almost) where Iā€™m supposed to start QA almost from scratch.

When I arrived they had some test cases written on Excel, some automated tests with Selenium/Java/Cucumber written by outsourced ressources.

The software is coded in php, only web browser version for the moment and no mobile version.

There are many bugs in production so the company expects big change with my arrival in the company.

My plan for the future :

  • hire more QA (about 1 qa and 4 devs per team)

  • decide if the team should go on with the framework Selenium/Java/Cucumber. They have some skills but I want to see if they are to produce new scripts correctly and with a good pace.

About automation :
I have some doubts about the automation tool. I have the impression that the current framework is fine but not so consistent with the need of scale, especially with few QA on it. Sometimes I think about codeless tools because it could help to produce new automated tests quickly but there are also some downsides.

The constraints are : the tool must work with Github actions/Gitlab for the CI and it can do database validation. So most of codeless tools are eliminated especially for the second reason.

I had a look at the following tools :

  • Endtest :

Good : fair price, SQL ok, CI ok

Not good : I think that we canā€™t create ā€œshared stepsā€ with this tool, so if an object changes in a page, you have to update all the test cases using this object :confused:

  • Testproject :

Good : free, SQL ok (with an Add-on, to be tested), CI ok, SDK available in order to use the Cucumber existing tests

Not good : recorded steps are not so clear in terms of labels. Anyway when you run the test you have screenshots which is nice.

Good : very good capability to locate elements, good UI/UX, free version available.

Not good : I have heard that the licence is expensive, to be confirmed. For the SQL part they have a documentation (CLI chapter) but itā€™s not so clear for me.

To be reviewed : Accelq, Functionize.

If you have some suggestion do not hesitate ! :+1:

3 Likes
2

Have that nice little talk to the development team about placing test code alongside product code, make sure you understand the branching and release processes before going into that. This will help you to get more developer eyeballs and hours on helping with automated testing. choosing a tool that shares a easy ā€œlanguageā€ will also help if you do really want devs to be writing tests, so you may want to look at Java based toolstacks. not a java fan myself, but thatā€™s what I would lean towards.

Unless you donā€™t want help from the devs. Although sounds like tight CI integration is a requirement, another reason to throw out any ā€œcodelessā€ tools, early. They often create unpredictable workflows around versioning of test-source-code, which creates unpredictable time taken to test and help do releases. I would ask for help to do a ā€œspikeā€ to work out if it is possible to scale up your existing tooling before giving up on it entirely. You can use the spike to learn what your other constraints are too.

1 Like
3

@sylvainpi

I had a look 3-4 years ago at this kind of tool, make sure that they can identify correctly all the elements in your software. Of course labels and button are generally ok with any tool, but sometimes you can have some issues with sophisticated objects.

For instance, letā€™s suppose that you automate a scenario where you answer to a survey : the textbox is easy to identify and to write inside of it, but what about questions having stars from 1 to 5 for example ? Or a sliding bar. Not sure that all the tools can behave correctly when interacting with this tricky elements.

3 Likes
4

Iā€™m in a similar position and this is a real can of worms with plenty of different approaches and options. Iā€™ve found the ā€˜Setting a Foundation for Successful Test Automationā€™ module on https://testautomationu.applitools.com/ useful in thinking this one through. Thereā€™s even a learning path for codeless automation if you decide to go that way. Hope this helps :slight_smile:

1 Like
5

Like @conrad.braam, Iā€™d like to have a closer look at code-based automation, too. There are some great low-code tools out there (I donā€™t believe in ā€˜codelessā€™), but if you do want more help from your developers, I think code-based is the way to go.

I donā€™t really understand the suggestion of Java, though, since your application is written in PHP. There are some good PHP test libraries out there, most notably Codeception, so I would try that out first. Its syntax is relatively straightforward (although you have to keep in mind that you are still writing code!) and because it is code, and runs on top of PHPUnit, a unit testing framework, it will play nicely with CI and version control.

2 Likes
6

Interesting Bas.
What low-code tools have you seen who actually work properly ?

7

The first one that comes to mind is Robot Framework. Offers an abstraction layer on top of the code (through keywords), so that makes it ā€˜low codeā€™ for me.

Another Iā€™ve used successfully in the past is SOAtest from Parasoft (and all Parasoft products really). Quite powerful tools.

1 Like
8

Thanks for this information!

1 Like
10

Are you sure you need the direct SQL checks? Meaning, are there GET/read endpoints in the API that can be used to verify the POSTs?

11

Can you expand on this?

  • What is a bug and to whom? is it users complaining about the product on public forums, is it logged errors in your backend system, is it a high number of calls to support, is it many product returns, is it a manager that opens the app, sees some thinks he doesnā€™t like, is it the dev team, is it the tester that has actually found some bugs;
  • What gives away the fact that there are bugs in production? Is it the bug backlog which is clogged by a thousand tickets that have never been solved? Is it the developers that say they donā€™t have time to fix the many bugs they have in their product?
  • How do they define ā€˜manyā€™ - are all bugs of equal impact? From past experience: a few C* level people were looking to a product and didnā€™t like the prices, image quality, layout, availability of offers or sorting - they categorized the product as crap and very buggy.
  • How many of the bugs have been seen but not solved by the developers? Can there be other problems with the organization of product building?
  • Are they expecting you to find more bugs than they already know about, to fix them, to manage the development team to fix them, to fix the organization, to triage the bugs, to look a little less for bugs so that there ā€˜arenā€™t manyā€™;
  • You talk about the ā€˜companyā€™ expecting you to do things. Does this mean that they have a problem at the company level? Usually a tester reports to a CTO/Dev-Lead/Product-Manager. You should focus on what your manager feels that is needed to have better visibility on the quality of the product.
1 Like