Hiring a first automated tester for our company

Hi All,
First time poster here to the community, hoping for some assistance from some friendly testers. Ive been a QA tester and now manager in the same software company for 12 years now. we are late coming to the automation game but i have been given budget to hire our first automated test engineer(this would be a senior role). The current team is just 2 manual testers with no coding experience.

The new hire would be responsible for working with the current team to identify area’s that can be automated, identify the correct tools/frameworks and eventually write the tests. initially they would work outside of current sprints and possibly focus on our large regression suite which is currently all manual

i would like some tips on how would we integrate this person into the development team since nobody in our company currently has any test automation experience and are there any pitfalls to avoid when beginning our journey into the automation world

4 Likes

There are many automation engineers out there, but for your situation I’d look for someone that not only has done automation, but has some experience in leading teams and (if possible) has experience in teaching others how to automate tests. Also, it would be ideal if that person has previous experience in transitioning from manual to automated testing.

Have your developers at the interview to determine the engineering knowledge and of the applicant, which should be a relatively straightforward process - test automation (with all it’s specifics) is software development, after all.

I would look for is passion for testing and eagerness to teach others how to automate, I think that would be the most important trait in the new candidate, that and how well the persons can interact with others.

4 Likes
  1. HOW – Prior experience of test automation framework development from scratch (How to develop)
  2. TEAMWORK – Ability and willingness to coach team members on automation
  3. WHAT and WHY – Knowledge and understanding of software testing overall, as the tester /candidate should understand WHAT/ WHY to automate certain tests/ user journey and why not other.
  4. DevOps – Having hands-on knowledge on CI/CD pipeline building is required to have all automated checks running continuously helping get ROI from the automated tests.
3 Likes

Welcome to the community David. :slight_smile: We could all roll off a list of many things to look out for, but without the “context”, and a grasp of the domain problems you face, it would just be a list of the potholes between New York and Los Angeles, but only if you took route 81, and switched right away to route 70 for example.

Basically, be prepared to buy or preferably “rent” for them a lot of updated “hardware” after just a few months once they get started. Automation eats hardware even if it is in a cloud. Be prepared to spend a month trialing a few tools. Also be prepared for your Build system to take much more extra load (mainly because now your software development lifecycle is going to accelerate). I mean that’s why you are hiring them. You are right about working “outside” the dev team initially - but know this, your dev team can help them a lot.

And on that final “help” point, the successful candidate needs to be someone who can talk on the topic of “automatability” from more than just 5 minutes. Because they will be asking the devs to make changes to the product often at it’s core, in order to make automation easier. Starting out with improving product logging, because I’ll bet your product logs are not easy to parse. So expect bug “tickets” for things that are not functional defects. Good luck, it’s a long journey, long.

5 Likes

thanks all for the responses so far, great ideas about involving the development team both in the interview stage and ongoing, our dev team are onboard with this idea and keen to help. Teaching others is another great point as our current manual testers certainly dont want to be left behind and are keen to upskill both for the benefit of the company and their own career development

For a bit of context, our application is a web based photo gift platform. We perform black box testing on the front end photo application mostly, with it being a web and mobile web based application, we do A LOT of browser testing to cover the possible browser combinations our consumers may use on desktop, tablets or mobile devices. We do some API testing but primarily the devs help a lot with this and we simply check the output on the front end. We also have a shopping cart, desktop application editor and production back end as part of our suite but the majority of our testing is focussed on the online editor. So another obvious automation point i guess is the browser testing as this can take a lot of time up, in particular on new features.

4 Likes

Hi, David.

As I understood correctly from your context, you have a huge app with different parts: Web, some API’s, mobile devices, desktop app as well.

Before starting to look for an engineer and make interviews, you need to have a clear picture of the automation strategy for the whole product and how it will fit the current manual testing activities as well as delivery processes.

  1. Identify high, medium, and low-risk features of the applications. It will be the first point for automation.
  2. Gather information about the most used parts of an application and its platform (e.g. is it web or desktop or mobile tablets).
  3. Identify the goals of automation testing: whether it is a help for testers to speed up or simplify the process a bit or a global management goal to transform the whole release cycle.
  4. Discuss and approve this automation vision with upper managers as well as the development team and architects.

Only after that, it is time to look for an engineer. At that time you will have a more or less defined job profile.

As for hiring an automation engineer.

  • Good automation engineers should have a slid core programming skills and have testing knowledge.
  • You can ask a developer to attend an interview and ask programming questions (but do not set a high limit - the candidate’s skill should be at the middle level).
  • If you see that engineer is also responsible for setting up CICD pipeline with tests - add such skills into the job profile
  • If a candidate tells you, that they first need to create a big and unbreakable automation framework and only after that (after 6-12 months) you can start to contribute tests - do not believe it. Test automation should bring value to the testing and development teams almost straight away.
  • Start by automating just a few most critical user journeys in one critical browser, make it stable, and integrate it into the pipeline. Automated tests should be executed either on each merge commit or at least for each nightly release candidate.
  • The simpler automation tests are - the better. But do not try to introduce radical changes like BDD(Behavior-Driven Development) from day one. It will appear very attractive to make manual testers write automation tests, but in fact, it requires a lot of contribution from business people and developers.

There are a lot of other things to consider when starting automation for the product. But in order to be more specific - it requires getting more context about technologies, delivery process, and product.

8 Likes

100% this, in my journey I had to fail a few times before I adopted the “small-wins early” route.

Not all manual testers want to become script jockeys, I can see why your app needed manual testers, and still will do. Make sure they are still valued. Force anyone writing an automated test to manually test for a while before embarking on scripting. It’s super easy to automate the wrong thing. Every automation-tester has scripted a test that later on fails to find the classes of bugs they were keen to detect. I have done that far too many times, that might be a good interview question along the lines of “How do you deal with making mistakes in your work?”.

5 Likes

Excellent feedback again guys, getting some great ideas. Your right in that the manual testing of our product will always be required, verifying UX and usability is key for our online editor. Very interesting points about starting small, this makes me question if automating our large regression pack we run during a cycle is the right approach, it could be a large task

Our delivery process is we currently only release 3/4 versions a year and these are usually ‘large’ drops (on occasion point releases are required for bug fixes but these arent the norm :slight_smile: We are a white label software, Our customers are typically printers who are capable of producing photo products (we are b2B not B2C), they host our application themselves either in the cloud or internally and re-skin it to their consumer branding. When we release our support team remote into their server and upgrade them ourselves. So our current CI/CD pipeline, whilst being worked on by development, is at a basic level, it runs a few unit tests against our pricing engine for our cart but thats about it.

Initially id like to focus the automated testing solely on our online editor. our shopping cart and backend production systems dont tend to change to much. Our desktop application is something thats almost a separate product these days and ive not thought about how easy or useful it is to automate for desktop applications. i may need to include this in my thoughts now

1 Like

What might we consider “almost” straight away? Obviously we don’t need unbreakable framework, but unless you go the hacky route of embedding test workflow logic implementation details (typically abstracted into test framework) into the test case(s) itself, you don’t get automated tests right away.

A proper strategy to consider may be to build a barebones minimal framework and then to write test cases that include pseudo code (or the expected framework function calls) for the missing pieces and then fill in the missing pieces that are needed in the framework to support the test case. This way you just build what you need as you go for tests being automated.

Would be interesting to see how candidates elaborate on what they’d do for this role/job for the automation strategy to bring value fast.

4 Likes

Automation has a lot of value - you just need to stick to your goals. Any maybe that goal is to only test one component if you can isolate it well. To be honest all the big projects I have done started in one corner of the product-suite. If your goal is faster SDLC, build some minimal automation rig that only smoke-tests the system, and then move from there. Part of building a automation system involves a lot of decisions, like

  • do you deploy onto real devices or virtual devices,
  • what does the cloud hosting of web services cost you in terms of money and time,
  • what test environments are realistic, and should you test in production-like or in a miniature environment
    Make the trade-offs that make sense to you. For example you will have different security domains for production and test, and may in fact entirely disable some security features in test environments. That’s still a very valid thing to do if you can mitigate the security testing risk. Testing as we all know is all about communicating risk and how you plan to offset that risk. In my experience, security was best not automating fully, but keeping security in every conversation, with emphasis on secure-by-design.

But, this does sound like a very cool project you are embarking on @davidcarterqa . I’ll add a case-study book recommendation : Experiences of Test Automation, by Dorothy Graham and Mark Fewster. It’s the book with a snow capped Alaskan mountain on the cover.

1 Like

These answers have greatly helped shape my interview questions guys so thank you, ill sit down with our lead developer today and iron them out.
We have a large migration project coming up where a lot of the code behind the scenes is being rewritten in a new language, so there will be a lot of regression testing of our online editor to make sure it functions exactly as it does today. This again is a project that id like the new automated tester to assist with. Its an exciting time for our company that’s for sure! and long overdue…

thanks for the reading recommendation @conrad.braam, copy ordered :slight_smile:

1 Like

Don’t, last thing you need is one tester trying to automate and competing with developers and manual testers.

Develop a strategy first, depending on your industry and architecture. Developers should likely be writing unit tests, api tests (with mocks), possible contract tests and UI tests that are layered not e2e. I notice you didn’t mention the Dev team or how strong they are. You all need to be pulling together, if you try and automate after the fact and don’t have a testability strategy then you are doomed to fail.

Once you have that all set up, then you might hire a SDET to provide domain expertise and compliment the existing developers.

I’ll give you an example, you have a god foresaken date picker which is nearly impossible to automate. Devs are busy writing feature so can’t replace it. The test automation guy writes an elaborate work around which takes him a week and is forever flaky. The better solution is to replace the datepicker with a better one and plan for how it will be testable. The SDET pitches in to free up resources to let the Devs work on it together and think about how to make it testable from the get go. Both the reimplantation and tests are done in half a much time and are reliable got forward.

1 Like

thanks for the insight Matthew. the date picker example is an interesting one. what would you consider the main difference between an SDET and an automated tester? would an SDET simply be a stronger coder?
a little more background, Our dev team do currently write Unit tests for any new features and some front end UI tests with cypress for specific features. but their main responsibility is getting new features developed and implemented, the job of testing them from almost every aspect (regression, integration, UI) is left almost entirely with the QA team. Whilst ive got the developers on board with implementing better unit tests, utilising more of their time for extra testing isnt possible. they are o hand to assist wit any coding questions since they are super strong on that side and the manual testers would assist on any test strategy questions and where to begin automating

One other reason to hire this role into the QA team is knowledge transfer and training, the current team does wish to upskill, both for the benefit of the company and their own career progressions. I feel the best way to do this is to learn on the job, using real world examples, from somebody who has experience of this kind of role.

2 Likes

Yeah I’m kinda splitting hairs with the terminology but typically SDETs have more of a coding background or broader skill set and are comfortable changing product code. Often when people say automated tester they mean someone who can write API/UI tests. It’s difficult for me to say because its basically my background. But the trend from the high performing companies as defined by Dora is away from that model. You need Devs to own quality, and be motivated to maintain tests. If you have different people maintaining testa than the application you will do neither well. All the evidence coming out is telling us that. So Dora would say the right solution there would be to hire more devs or push fewer features. There is a caveat that Devs need to know what the expectations are. Also they need to be enabled by have frameworks and pipelines in place. So there are two models I’ve heard work for SDETs, one is to embed in teams to add tests but also test the application to enable testing when appropriate. The other is the build frameworks but leave the individual testa tests for teams.

We kinda did that for production testing. SRE built the framework, DevOps manage pipelines and Devs write the tests for their features.

Currently my business unit doesn’t have any actual SDETs with have testers embed in teams who we want to act more like SDETS. But else where in the company they do have skilled SDETS embeded in teams and changing app code as needed. They also build frameworks for API/UI/performance testing.

I know this is easy to say from a big company but honestly it works at any size org. I’ve worked companies of all sizes and stages.

2 Likes