Hi all
I’ve been told that “best practice” is to keep the Test automation against the product code. However, in my team we have been discussing having a separate test automation repo.
I was wondering what do you do and why?
Is one approach better than the other? Or simply depends on context?
My team member sent me this Where do you store your test automation code? - 
Content Discussions - The Club: Software Testing & Quality Engineering Community Forum | Ministry of Testing
Will have a read. Yet again, need to use the search first before asking 
I have used both and having separate is better in my opinion.
The only major advantage that same repo which i have used in case of API is you get the Classes for Seralisation and Deseralisation of API request and response.
For complex projects IMHO it’s better to keep it separate but collaborate better so still you can take all those advantages that single /same repo gives.
Hi, Thanks for sharing that!
I’ll post an opposing view. My team is currently migrating from separated to being colocated because being separated also discouraged developers from contributing (and also increased the gap between QAs and devs, decreased the code quality of the automation code, decreased visibility of QA work, etc).
I will say our teams are no more than 10 people in a cross functional team and the repos are very focused (eg, just the mobile client or just the front end client).
We also find it easier to adjust the automation in response to changes because it can be adjusted in the same branch and breaking automation truly is everyone’s responsibility to address.
We often choose not to take the optimal path for good reasons sacrificing something of value in exchange for a different value, sometimes this happens due to constraints and others due to opportunities.
As an example in my experience scripted automation coverage is most efficient when it is owned by developers and kept as close to the product code as possible. This has added benefits of a higher likelihood that it will be stack optimised.
Now given we are on testing forum and there are definitely a lot of testers working and perhaps leading and owning some level of automation, there are likely good reasons for that but in my experience they will likely have had to sacrifice a level of efficiency and potentially stack optimisation in doing so.
As an example I’ve worked with products where the good developer practices with automation just did not happen, the product got itself into a state where regression risk was overwhelming, developers were fire fighting and product development suffered as a result.
In that case handing over the automation to a dedicated group made sense, not so tied to the code base, allowed different languages and perhaps less coding skills to work on it so a separate repo worked well with this model.
It is a less efficient model, that is known and accepted but was regarded as the “best” model given the situation that allowed the regression risk problem to be addressed over time whilst freeing up developers to focus on development.
In an ideal scenario it would be a temp solution with automation transitioning to the efficiency model later but often that does not happen.
If you move automation to tester ownership or decide for a separate repo understand what you may be sacrificing, understand the value you believe you will gain. Weigh them up and find a balance that works for the team.
Context might obviously influence what choices you make and what changes you are willing to make.
If memory serves me right, I believe we have always had the automation in the same repo anywhere I’ve worked the last decade, except perhaps when we inherited a project already doing it differently.
There are a few things to point out that might be different to your case though:
Having the whole org pushing to the same repo might mean you need to pull/rebase a bit more frequent, but if the code base is somewhat well-organised and architected, and teams or pairs aren’t working on the same feature at the same time, merge conflicts shouldn’t (?) be a big problem? If it is, I think that is a smell of something else needing fixing.
If you already have a big org-spanning-spaghetti-hairball-mess of a project where people are pushing all kind of crap without running tests and validations on PRs and devs refuse to participate in making sure what they create is actually working… I guess it might save a lot of frustration to just do the automation in a nice quiet silo without all the interference 
We have it in a separate repo. Makes it easier to work on.
We’ve got separate repos for end-to-end and API tests. I don’t know why it was done but I assume because it would make it easier for the QAs (who write the majority of these tests) to work on them without getting in the way of devs deploying and vice versa.
A challenge that we’re overcoming is running the tests in pipelines has to be specifically set up. We also use feature branches so need to get tests set up to run against PRs.
I also have a bit of a concern that it puts a bit of a wedge between the code I’m writing and actual good practices for typescript. I’m pretty sure what I write is some mongrel version of JavaScript and typescript and being isolated probably helps to exacerbate this as I’m probably more cobbling tests together than writing good ones that adhere to the standards of a language (a separate thing I’m working on resolving)
I prefer having automation code in the the same repo as implementation code. A few reasons why:
I prefer them being in the same repo but depending on your deployment strategy it might not be the easiest or most feasible. I have about 35 microservices so we couldn’t put all the tests in one repo. So we had to pick and choose what goes where and how they’re structured.
Things that absolutely should be in the same repo are Unit Tests and Integration Tests. We want to trigger every-time a developer alters, updates or does a refactor. Developers should also want this so they can change code with more confidence.
For things like End-To-End tests or performance tests, since those take a good while to run. We generally keep those in a separate repo and have them run on a schedule. Here we do our Smoke/E2E tests on a 2 hour schedule and our performance tests we run ad-hoc. But to keep them up to date and aligned, if they fail, they will post an alert to our teams chat to get fixed.
I usually keep tests with the product code so changes stay in sync, but I’ve used a separate repo when teams or tech stacks were very different.
I tend to separate unit tests from this question.
Unit tests clearly belong with the source code. They are part of the implementation feedback loop.
But when people say “test automation”, they often mean E2E/API/UI checks. Those usually have their own setup, dependencies, environments, test data, and sometimes a different release cadence. In that case, a separate repo can be cleaner.
The tradeoff is ownership. A separate repo is fine if failures still feed back quickly to the product team, and if the tests are versioned against a clear target environment or release. Otherwise it can become a detached QA silo.
Separate is better.
Otherwise you run into merge conflicts, versioning and release blockade issues.
In practice, I go with the old-school belief that “Quality Assurance” or anything related to testing code/apps outside the development team should be separate. This ensures unbiased evaluation by the testers, no resource overlap in terms of repository access, and no temptation on the part of development to take a peek and “get ahead” of the testing.
Obviously there are considerations when budget and the overall team is small, but whenever possible that is my approach. Some software development shops view “unit testing” as “the” testing and all the application code and unit tests are not only in the same repository, but rely on each other in various ways. DevOps and Agile development often have this, however there are often good reasons for this, related more to up-time, time-to-market and security than to budget or organization.
Perhaps the first question you should be asking is what is your need when it comes to customer-facing functionality? Are there financial considerations where having a more DevOps style infrastructure helps the organization avoid risk and losing considerable money when up-time is not consistent, or when new features are not made available rapidly (think financial systems and federal regulatory changes, or hospitals and compliance adherence)? Would testing efficiency be improved with blended repository resources or hindered?
This is all about organizational and operational design, of course, and comes down to what is best for the overall quality and perception of the brand over trying to fit into a specific standards-based infrastructure just because “that’s how it’s done”. Do the deep analysis, assemble the right reasons for either choice and then implement it with intent and consistency.
Would love to hear what direction you wound up going and why!
Most of the places I have worked have gone for separate repos for E2E testing. 2 reasons surface, and they both point to scale of product and life cycle. If you have many teams in many countries, who do you trust to do integration testing? Who do you trust to do the component/unit testing? How is the product sold, is it SaaS or is it in boxes? So I’d like to draw a graph as to show where the friction gets highest.
(uni): If you are a SaaS business, you are not maintaining multiple product versions, so you never need to run the same test suite against an older version.
(split): If you sell boxes and support more than one version, you don’t want to have a forked testing tool-stack and forked suite. Keeping 2 versions of test code in working order costs, so you will tend to either push test code to one or the other product version branch and never to both.
(uni): All code goes into one product and projects deliver into the main branch often.
(split): Prototype or long running projects mean you do custom work to the framework which may break tests that are not part of the prototype/project when you later merge it. Avoids that entirely and just don’t fork the tests, so tests and test-tools. You thus skip the stressful overhead of big mergefests when a feature merges at the same time as the test tooling.
(uni): Does one team own the E2E product and the E2E quality? Sure go with one repo.
(split): If getting the teams to talk is hard because they are in different time zones. Go with a separate repo.
(uni): The testing tools use roughly the same languages as the product, so any major testing stack changes are easy for the devs to own.
(split:) Test Technology stack frictions if you ship actual products or boxes. Separate repos then make it easier to re-invent your QA tooling and stack, like if you decide to switch vendors, you do tool migration work on a single branch, and can immediately test more than one product version after the new tool works.
I should really unpack this with more concrete descriptions and better wording.
All places I’ve worked we have always kept things in monorepos and then folders like src/ test/ e2e and so on