Hi All,
First time poster here to the community, hoping for some assistance from some friendly testers. Ive been a QA tester and now manager in the same software company for 12 years now. we are late coming to the automation game but i have been given budget to hire our first automated test engineer(this would be a senior role). The current team is just 2 manual testers with no coding experience.
The new hire would be responsible for working with the current team to identify areaās that can be automated, identify the correct tools/frameworks and eventually write the tests. initially they would work outside of current sprints and possibly focus on our large regression suite which is currently all manual
i would like some tips on how would we integrate this person into the development team since nobody in our company currently has any test automation experience and are there any pitfalls to avoid when beginning our journey into the automation world
There are many automation engineers out there, but for your situation Iād look for someone that not only has done automation, but has some experience in leading teams and (if possible) has experience in teaching others how to automate tests. Also, it would be ideal if that person has previous experience in transitioning from manual to automated testing.
Have your developers at the interview to determine the engineering knowledge and of the applicant, which should be a relatively straightforward process - test automation (with all itās specifics) is software development, after all.
I would look for is passion for testing and eagerness to teach others how to automate, I think that would be the most important trait in the new candidate, that and how well the persons can interact with others.
HOW ā Prior experience of test automation framework development from scratch (How to develop)
TEAMWORK ā Ability and willingness to coach team members on automation
WHAT and WHY ā Knowledge and understanding of software testing overall, as the tester /candidate should understand WHAT/ WHY to automate certain tests/ user journey and why not other.
DevOps ā Having hands-on knowledge on CI/CD pipeline building is required to have all automated checks running continuously helping get ROI from the automated tests.
Welcome to the community David. We could all roll off a list of many things to look out for, but without the ācontextā, and a grasp of the domain problems you face, it would just be a list of the potholes between New York and Los Angeles, but only if you took route 81, and switched right away to route 70 for example.
Basically, be prepared to buy or preferably ārentā for them a lot of updated āhardwareā after just a few months once they get started. Automation eats hardware even if it is in a cloud. Be prepared to spend a month trialing a few tools. Also be prepared for your Build system to take much more extra load (mainly because now your software development lifecycle is going to accelerate). I mean thatās why you are hiring them. You are right about working āoutsideā the dev team initially - but know this, your dev team can help them a lot.
And on that final āhelpā point, the successful candidate needs to be someone who can talk on the topic of āautomatabilityā from more than just 5 minutes. Because they will be asking the devs to make changes to the product often at itās core, in order to make automation easier. Starting out with improving product logging, because Iāll bet your product logs are not easy to parse. So expect bug āticketsā for things that are not functional defects. Good luck, itās a long journey, long.
thanks all for the responses so far, great ideas about involving the development team both in the interview stage and ongoing, our dev team are onboard with this idea and keen to help. Teaching others is another great point as our current manual testers certainly dont want to be left behind and are keen to upskill both for the benefit of the company and their own career development
For a bit of context, our application is a web based photo gift platform. We perform black box testing on the front end photo application mostly, with it being a web and mobile web based application, we do A LOT of browser testing to cover the possible browser combinations our consumers may use on desktop, tablets or mobile devices. We do some API testing but primarily the devs help a lot with this and we simply check the output on the front end. We also have a shopping cart, desktop application editor and production back end as part of our suite but the majority of our testing is focussed on the online editor. So another obvious automation point i guess is the browser testing as this can take a lot of time up, in particular on new features.
As I understood correctly from your context, you have a huge app with different parts: Web, some APIās, mobile devices, desktop app as well.
Before starting to look for an engineer and make interviews, you need to have a clear picture of the automation strategy for the whole product and how it will fit the current manual testing activities as well as delivery processes.
Identify high, medium, and low-risk features of the applications. It will be the first point for automation.
Gather information about the most used parts of an application and its platform (e.g. is it web or desktop or mobile tablets).
Identify the goals of automation testing: whether it is a help for testers to speed up or simplify the process a bit or a global management goal to transform the whole release cycle.
Discuss and approve this automation vision with upper managers as well as the development team and architects.
Only after that, it is time to look for an engineer. At that time you will have a more or less defined job profile.
As for hiring an automation engineer.
Good automation engineers should have a slid core programming skills and have testing knowledge.
You can ask a developer to attend an interview and ask programming questions (but do not set a high limit - the candidateās skill should be at the middle level).
If you see that engineer is also responsible for setting up CICD pipeline with tests - add such skills into the job profile
If a candidate tells you, that they first need to create a big and unbreakable automation framework and only after that (after 6-12 months) you can start to contribute tests - do not believe it. Test automation should bring value to the testing and development teams almost straight away.
Start by automating just a few most critical user journeys in one critical browser, make it stable, and integrate it into the pipeline. Automated tests should be executed either on each merge commit or at least for each nightly release candidate.
The simpler automation tests are - the better. But do not try to introduce radical changes like BDD(Behavior-Driven Development) from day one. It will appear very attractive to make manual testers write automation tests, but in fact, it requires a lot of contribution from business people and developers.
There are a lot of other things to consider when starting automation for the product. But in order to be more specific - it requires getting more context about technologies, delivery process, and product.
100% this, in my journey I had to fail a few times before I adopted the āsmall-wins earlyā route.
Not all manual testers want to become script jockeys, I can see why your app needed manual testers, and still will do. Make sure they are still valued. Force anyone writing an automated test to manually test for a while before embarking on scripting. Itās super easy to automate the wrong thing. Every automation-tester has scripted a test that later on fails to find the classes of bugs they were keen to detect. I have done that far too many times, that might be a good interview question along the lines of āHow do you deal with making mistakes in your work?ā.
Excellent feedback again guys, getting some great ideas. Your right in that the manual testing of our product will always be required, verifying UX and usability is key for our online editor. Very interesting points about starting small, this makes me question if automating our large regression pack we run during a cycle is the right approach, it could be a large task
Our delivery process is we currently only release 3/4 versions a year and these are usually ālargeā drops (on occasion point releases are required for bug fixes but these arent the norm We are a white label software, Our customers are typically printers who are capable of producing photo products (we are b2B not B2C), they host our application themselves either in the cloud or internally and re-skin it to their consumer branding. When we release our support team remote into their server and upgrade them ourselves. So our current CI/CD pipeline, whilst being worked on by development, is at a basic level, it runs a few unit tests against our pricing engine for our cart but thats about it.
Initially id like to focus the automated testing solely on our online editor. our shopping cart and backend production systems dont tend to change to much. Our desktop application is something thats almost a separate product these days and ive not thought about how easy or useful it is to automate for desktop applications. i may need to include this in my thoughts now
What might we consider āalmostā straight away? Obviously we donāt need unbreakable framework, but unless you go the hacky route of embedding test workflow logic implementation details (typically abstracted into test framework) into the test case(s) itself, you donāt get automated tests right away.
A proper strategy to consider may be to build a barebones minimal framework and then to write test cases that include pseudo code (or the expected framework function calls) for the missing pieces and then fill in the missing pieces that are needed in the framework to support the test case. This way you just build what you need as you go for tests being automated.
Would be interesting to see how candidates elaborate on what theyād do for this role/job for the automation strategy to bring value fast.
Automation has a lot of value - you just need to stick to your goals. Any maybe that goal is to only test one component if you can isolate it well. To be honest all the big projects I have done started in one corner of the product-suite. If your goal is faster SDLC, build some minimal automation rig that only smoke-tests the system, and then move from there. Part of building a automation system involves a lot of decisions, like
do you deploy onto real devices or virtual devices,
what does the cloud hosting of web services cost you in terms of money and time,
what test environments are realistic, and should you test in production-like or in a miniature environment
Make the trade-offs that make sense to you. For example you will have different security domains for production and test, and may in fact entirely disable some security features in test environments. Thatās still a very valid thing to do if you can mitigate the security testing risk. Testing as we all know is all about communicating risk and how you plan to offset that risk. In my experience, security was best not automating fully, but keeping security in every conversation, with emphasis on secure-by-design.
But, this does sound like a very cool project you are embarking on @davidcarterqa . Iāll add a case-study book recommendation : Experiences of Test Automation, by Dorothy Graham and Mark Fewster. Itās the book with a snow capped Alaskan mountain on the cover.
These answers have greatly helped shape my interview questions guys so thank you, ill sit down with our lead developer today and iron them out.
We have a large migration project coming up where a lot of the code behind the scenes is being rewritten in a new language, so there will be a lot of regression testing of our online editor to make sure it functions exactly as it does today. This again is a project that id like the new automated tester to assist with. Its an exciting time for our company thatās for sure! and long overdueā¦
thanks for the reading recommendation @conrad.braam, copy ordered
Donāt, last thing you need is one tester trying to automate and competing with developers and manual testers.
Develop a strategy first, depending on your industry and architecture. Developers should likely be writing unit tests, api tests (with mocks), possible contract tests and UI tests that are layered not e2e. I notice you didnāt mention the Dev team or how strong they are. You all need to be pulling together, if you try and automate after the fact and donāt have a testability strategy then you are doomed to fail.
Once you have that all set up, then you might hire a SDET to provide domain expertise and compliment the existing developers.
Iāll give you an example, you have a god foresaken date picker which is nearly impossible to automate. Devs are busy writing feature so canāt replace it. The test automation guy writes an elaborate work around which takes him a week and is forever flaky. The better solution is to replace the datepicker with a better one and plan for how it will be testable. The SDET pitches in to free up resources to let the Devs work on it together and think about how to make it testable from the get go. Both the reimplantation and tests are done in half a much time and are reliable got forward.
thanks for the insight Matthew. the date picker example is an interesting one. what would you consider the main difference between an SDET and an automated tester? would an SDET simply be a stronger coder?
a little more background, Our dev team do currently write Unit tests for any new features and some front end UI tests with cypress for specific features. but their main responsibility is getting new features developed and implemented, the job of testing them from almost every aspect (regression, integration, UI) is left almost entirely with the QA team. Whilst ive got the developers on board with implementing better unit tests, utilising more of their time for extra testing isnt possible. they are o hand to assist wit any coding questions since they are super strong on that side and the manual testers would assist on any test strategy questions and where to begin automating
One other reason to hire this role into the QA team is knowledge transfer and training, the current team does wish to upskill, both for the benefit of the company and their own career progressions. I feel the best way to do this is to learn on the job, using real world examples, from somebody who has experience of this kind of role.
Yeah Iām kinda splitting hairs with the terminology but typically SDETs have more of a coding background or broader skill set and are comfortable changing product code. Often when people say automated tester they mean someone who can write API/UI tests. Itās difficult for me to say because its basically my background. But the trend from the high performing companies as defined by Dora is away from that model. You need Devs to own quality, and be motivated to maintain tests. If you have different people maintaining testa than the application you will do neither well. All the evidence coming out is telling us that. So Dora would say the right solution there would be to hire more devs or push fewer features. There is a caveat that Devs need to know what the expectations are. Also they need to be enabled by have frameworks and pipelines in place. So there are two models Iāve heard work for SDETs, one is to embed in teams to add tests but also test the application to enable testing when appropriate. The other is the build frameworks but leave the individual testa tests for teams.
We kinda did that for production testing. SRE built the framework, DevOps manage pipelines and Devs write the tests for their features.
Currently my business unit doesnāt have any actual SDETs with have testers embed in teams who we want to act more like SDETS. But else where in the company they do have skilled SDETS embeded in teams and changing app code as needed. They also build frameworks for API/UI/performance testing.
I know this is easy to say from a big company but honestly it works at any size org. Iāve worked companies of all sizes and stages.
I would personally recommend you to collaborate with an automation tester that has experience working on manual testing activities. It would not only aid manual testers to align with automation testing practices but would even simplify manual tasks done by your team.
For small or big companies it is alwyes better to build a QA automation team. The hiring process depends on what kind of requirement you have or need for your current project. First, you have to figure out which tool is right for you so it would be perfect for you to hire test engineers that exactly match the demands. If you are working on test automation it is always better to hire an experienced person.
As you may know, these days many testing companies are becoming Automation Testing Companies and to implement automation properly they have to hire Automation Engineers with good knowledge of automation tools. Automation engineers should specialize in using technology to control the production and delivery of products as they work with automated machinery designing, programming, and testing processes. One has to be familiar with computer programming, and need to be able to think on his feet and troubleshoot emerging issues related to the equipment in order to become an Automation Engineer.
As per our understanding, first company has to shortlist the technologies in which hiring needs to be done. In Automation Testing there are various tools which are used by Automation Testing Companies to automate testcases. These tools help businesses automate alerts, integrate QA processes and test automation processes in the organization. They can also help solve automation problems, deliver reports and automate the QA testing process. Following are few tools which can be used for Automation:
Selenium
TestComplete
Cypress
Usetrace
PractiTest
To hire an Automation Engineer, organization should follow below points so that they can end up with good resources for Automation Testing:
Good knowledge on Manual Testing as well: In order to become a good Automation Engineer, one should have a good hold on manual testing as well. This is because one is going to automate the manual test scenarios in automation and therefore they should know about the checkpoints that needs to be implemented on various steps of automation test scripts.
Good hold on atleast one language : Mostly all automation tools require one or more languages for automation and logic needs to implemented in those languages only. Also, framework construction requires a lot of effort and in depth knowledge of language in order to be successfully implemented. So, one has to have a good hold on language so that development and troubleshooting can be done easily.
Automation Certifications : Automation is a relatively new field, but certifications are emerging to help people validate their automation skills. Below are some of these certifications:
ISA Certified Automation Professional (CAP)
ISA Certified Control Systems Technician (CCST)
ASTQB Test Automation Engineer Certification (ITSQB)
IIST Certified Software Test Automation Specialist (CSTAS)
IIST Certified Software Test Automation Architect (CSTAA)
We could all roll off a list of many things to look out for, but without the ācontextā, and a grasp of the domain problems you face, it would just be a list of the potholes between New York and Los Angeles, but only if you took route 81, and switched right away to route 70 for example.