Hi there @teamofone and @qofd . Welcome to the most awesome Software quality community in the world. No, in the universe. Most of us have not been in your position, but it’s so good to see that bigger teams starting out have the same issues or worries. Hope we can encourage you this week.
OK I’m trying to find out what it is you are testing, and far too often I wonder why it is that online forums never list peoples place of employ so we can approximately work out if they are testing a cloud-service, web, embedded-app, aerospace/automotive mission-critical or entertainment/gambling app. Because with that little bit of shape or context a person can point to more specific resources. I’m looking at your questions Jenny, and thinking broadly.
- the tests that the developers have coded up, where, how and when do those tests execute? If the answer is manually, then I have an idea for you to build a sandbox that will help.
- it’s tempting to hire a contractor, I used to contract, they work hard and help, but once they go, the problems will most likely return, because it sounds like the issue today is endemic. Contractors are a tool, not a solution, they cannot “fix” anything, they can only help build things.
- it’s normal for bugs to escape, and you are finding yourself in a situation where the team is trying to prevent them, trust me the devs don’t want escapees either. Bug-fixing is not fun, slows them down, and is tiring and demoralizing. Automating, however wont plug that gap that quickly.
Sherilyn is right, you want to get a Continuous Integration (CI) system going. I would start with a wiki page to plan your moves. I start with a page of what I have in the way of tools I could use and do use already. You might be using a script language, a cloud platform of some tool like fiddler and a variety of other tools. capture these on your wiki (we have a tools wiki here actually Ninja training for software testers | Ministry of Testing). Write down all of it.
Next, you want to build a test environment, a sandbox if you like with instructions how to set up a computer to run a test, this is where knowing what your product is would help, but for now gather the steps or guides to run those tests onto the same wiki page below your tools list. Work out how much it costs to set this up in terms of time and work out if you need to buy a new computer or a bit more VM runtime in a cloud somewhere. Work out if networks and security make running your existing automated and manual tests hard.
Next, before you stat to brag about your new way to prevent bugs escaping, try set up a script which you want to be triggerable from your automated build system, and work out how to grab all the test run artefacts, every last log, especially logs from external systems, into one place, and how to push those logs back into the build log/artefacts in Teamcity/Jenkins/whatever. Keep it broad though. Once you can manually trigger your Frankenstein test suite, little questions like, “Can I get it to test a branch”, “Can I get it to test an different configuration of the product”, or a different environment, and so on will emerge. Give yourself plenty of time to work out which problems you do and don’t want to solve. At this point I like to get the team lead and management on board to work out what priorities they have. They may want you to automatically trigger this to run every night now, take your time, look at security-workarounds and hacks you had to make, look at your test fixtures, and environment cleanness. These will take a while to bed down. Be prepared to fix your Frankenstein every morning, be prepared even to start over from scratch, and defend against that if needed.
Oh, and get the devs to help by getting them to do peripheral things like reporting and email integrations, they love that kind of thing. I hope my ramble, and it is a ramble; is useful to you Jenny. I hope the encouragement from Sherilyn has buoyed you up today too.