# Analysis Tools to avoid the "Test Everything" Test Strategy **URL:** \ **Category:** Archive\ **Tags:** strategy, test-strategies\ **Created:** [2 August 2021 10:40 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362 "2021-08-02T10:40:44Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![damien.gault](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/damien.gault/32/9317_2.png) [@damien.gault](https://club.ministryoftesting.com/u/damien.gault)\ **Post date:** [2 August 2021 10:40 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/1 "2021-08-02T10:40:44Z")
Hello, My name is Damien and I am Lead QA working in Financial Services. \*\*I would like to know what analysis Tools or collaboration techniques teams are currently doing to help come up with a more targeted Test Strategy ? With the goal to reduce the number of tests written for a new feature. \*\* I am trying to change the mindset of our team/department, to move away from the “Test Everything” Test Strategy mindset. As a result our test suite is at 16000 automated Cucumber scenarios. Which is now difficult to maintain with a lot of duplication. When working on a new feature, we end up writing tests just to demonstrate that it works with an existing feature. I would love to find a tool or come up with a strategy, that can scientifically inform us that, we do not have to write these tests because etc. I am keen to find out what teams are doing to help with the Analysis phase when working on a new feature. Looking forward to hearing your awesome ideas. Much appreciated. ---
**Author:** ![samuelm](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/samuelm/32/10017_2.png) [@samuelm](https://club.ministryoftesting.com/u/samuelm)\ **Post date:** [2 August 2021 15:37 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/2 "2021-08-02T15:37:32Z")
Hello Damien, I stumbled just last week across this paper about “Change Driven Testing” which might help you: > **[(PDF) Change-Driven Testing](https://www.researchgate.net/publication/337400883_Change-Driven_Testing)** > > PDF | Today, testers have to test ever larger amounts of software in ever smaller periods of time. This makes it infeasible to simply execute entire... | Find, read and cite all the research you need on ResearchGate Also I am devoloping as a side project a tool myself which tackles exactly the problem you are describing. The tool uses Git and static source code anylisis to determine which tests of the defined test suite(manual or automatic) should run in order to test the newly introduced changes. I am still not done with the development, but I can provide a prototype if you are intrested in trying it out. The tool works however so far just with Java applications. ---
**Author:** ![pwong](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/pwong/32/13337_2.png) [@pwong](https://club.ministryoftesting.com/u/pwong)\ **Post date:** [2 August 2021 20:06 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/3 "2021-08-02T20:06:42Z")
That tool sounds interesting Samuel, I don’t have a java app but I would be interested to follow your project. What is the best way to follow updates on it? ---
**Author:** ![samuelm](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/samuelm/32/10017_2.png) [@samuelm](https://club.ministryoftesting.com/u/samuelm)\ **Post date:** [3 August 2021 05:45 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/4 "2021-08-03T05:45:26Z")
Hmm to be honest there is so far no way you could follow my project😅 However anyway I thought about putting a beta version online to get feedback from the community and inform about updates for people like you who are interested 🙂🙂 Sounds like a little project for the weekend😉 will keep you informed ---
**Author:** ![damien.gault](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/damien.gault/32/9317_2.png) [@damien.gault](https://club.ministryoftesting.com/u/damien.gault)\ **Post date:** [3 August 2021 10:28 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/5 "2021-08-03T10:28:35Z")
> [@samuelm](#): > > “Change Driven Testing” Hi Samuel - “Change-Driven Testing” sounds really interesting! I have read about Test Impact Analysis previously. > **[The Rise of Test Impact Analysis](https://martinfowler.com/articles/rise-test-impact-analysis.html)** > > Test Impact Analysis speeds testing by predicting which tests need to be run for a change in the source code. The ability to only run those tests which test the code which has changed sounds like a great addition to any teams CI pipeline. Test Impact Analysis complemented by Test Gap Analysis sounds extremely efficient. I will have to read up on this more! Thank you for sharing this. Damien ---
**Author:** ![theology](https://avatars.discourse-cdn.com/v4/letter/t/9fc29f/32.png) [@theology](https://club.ministryoftesting.com/u/theology)\ **Post date:** [3 August 2021 16:20 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/6 "2021-08-03T16:20:47Z")
@damien.gault these “16000 automated Cucumber scenarios.” are small test cases where you are validating 1 step as 1 Test case. I will suggest to have End to End test case that will reduce your count and you can easily analyze report and conclude the release health. ---
**Author:** ![damien.gault](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/damien.gault/32/9317_2.png) [@damien.gault](https://club.ministryoftesting.com/u/damien.gault)\ **Post date:** [4 August 2021 12:43 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/7 "2021-08-04T12:43:14Z")
@samuelm do you know of any tools which can help with… 1. Creating Treemap visuals of your code 2. Test Impact Analysis 3. Test Gap Analysis I’d love to know if there are existing off the shelf tools that work with Java. Thanks ---
**Author:** ![samuelm](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/samuelm/32/10017_2.png) [@samuelm](https://club.ministryoftesting.com/u/samuelm)\ **Post date:** [4 August 2021 18:32 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/8 "2021-08-04T18:32:49Z")
Hi Damien, I think Teamscale does it. I did not try it out myself and I don’t know if they support Java, but I think they do. Here is the link: > **[Teamscale - A Fast Code Quality & Test Analysis Tool for Development Teams](https://www.cqse.eu/en/teamscale/overview/)** > > Teamscale helps teams to analyze and improve the quality of their software. Developers, testers & managers get fast, personal feedback on their work. If I got it right teamscale is developed from the same company which put out that paper which I sent you some days ago. Hope this helps​:wink:😉 ---
**Author:** ![samuelm](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/samuelm/32/10017_2.png) [@samuelm](https://club.ministryoftesting.com/u/samuelm)\ **Post date:** [22 February 2022 07:57 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/9 "2022-02-22T07:57:11Z")
Hi - sorry for the delayed answer. I finally found some time to set up a website - so you can follow the progress of this project. Here I am demonstrating the usage on a demo project: > **[Demo: How to avoid the test everything strategy - PushImpact](https://pushimpact.io/demo-how-to-avoid-the-test-everything-strategy)** > > Today I am very, very, very excited to give you a first impression of what a PushImpact report looks like. Without furder ado - here it is: PushImpact Demo Report of Hackers Cheat Sheet Before you check out the demo, you might however need some... One hint: In general the demo works on a mobile device, but it is much more fun on a normal computer screen. ---
**Author:** ![conrad.connected](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/conrad.connected/32/10123_2.png) [@conrad.connected](https://club.ministryoftesting.com/u/conrad.connected)\ **Post date:** [22 February 2022 08:15 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/10 "2022-02-22T08:15:35Z")
What exactly am I seeing here @samuelm . Without having to read the code, are we talking about instrumentation or adding decorators to product code or just using a domain or language specific tool to provide code churn insights but in a way that a tester can really use? ---
**Author:** ![samuelm](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/samuelm/32/10017_2.png) [@samuelm](https://club.ministryoftesting.com/u/samuelm)\ **Post date:** [22 February 2022 09:57 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/11 "2022-02-22T09:57:56Z")
Thank you for the interest @conrad.connected . It works by adding Java Annotations(I think that is compareable to decorators in python?!) to product code. For instance let’s assume you have a class Login that implements the authentication flow for a user. After you annotate this class with @Spec(id=“login”) all the relevant code for this class is calculated(for instance which methods calls this class and what methods do these methods call and so on). Also it is compared with the code of the last commit(or any other commit). By doing so relevant changes in the source are identified. At the end a report in html form is generated which includes the following information: 1. The feature login did change/or not change 2. How many lines / how much percentage of the feature did change 3. Which methods/classes/etc. did change or have an impact on this feature 4. The exact diff of the code that did change ---
**Author:** ![mikeharris](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/mikeharris/32/12295_2.png) [@mikeharris](https://club.ministryoftesting.com/u/mikeharris)\ **Post date:** [22 February 2022 11:55 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/12 "2022-02-22T11:55:32Z")
I worked with my team to analyse risk and wrote this bog post about it: [How can you work with your team to identify risks? – TestAndAnalysis](https://testandanalysis.home.blog/2021/11/16/how-can-you-work-with-your-team-to-identify-risks/) ---
**Author:** ![melissafisher](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/melissafisher/32/8020_2.png) [@melissafisher](https://club.ministryoftesting.com/u/melissafisher)\ **Post date:** [23 February 2022 16:15 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/13 "2022-02-23T16:15:01Z")
Risk based testing technique is an example of how you focus in on what’s important. Exploratory testing is another way to help you find problems that matter. ---
**Author:** ![conrad.connected](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/conrad.connected/32/10123_2.png) [@conrad.connected](https://club.ministryoftesting.com/u/conrad.connected)\ **Post date:** [24 February 2022 08:33 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/14 "2022-02-24T08:33:19Z")
I have never really tried to use FMEA but have read about it a fair deal @mikeharris .The FMEA model is quite exhaustive, but in software systems we have more data to drive even better decisions available to us. It is easy to drop into “a tool or an AI” will do this for us if we can collect enough clean data? For example why score risk from 1 to 10, when a scale of 1-5 is around 200% faster to apply and just as accurate on average. I love scoring risks, all together like this, numerically; because it lets us communicate those risks clearly. I feel we that other good pointers to test focus areas do exist; like understanding the role of product architecture, environment and dependencies. Having good language skills and grasp of the environment code lives in, is crucial to applying models like the FMEA system. I find it too easy to rely on experience, and that is unhealthy, and not robust. ---
**Author:** ![mikeharris](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/mikeharris/32/12295_2.png) [@mikeharris](https://club.ministryoftesting.com/u/mikeharris)\ **Post date:** [24 February 2022 09:02 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/15 "2022-02-24T09:02:10Z")
@conrad.connected Using the full FMEA process may not be what is best for your context. In the example I give in my post the team only used part of the FMEA process as that it what worked for them: [How can you work with your team to identify risks? – TestAndAnalysis](https://testandanalysis.home.blog/2021/11/16/how-can-you-work-with-your-team-to-identify-risks/) What tool or AI would you use instead? ---
**Author:** ![conrad.connected](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/conrad.connected/32/10123_2.png) [@conrad.connected](https://club.ministryoftesting.com/u/conrad.connected)\ **Post date:** [24 February 2022 09:11 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/16 "2022-02-24T09:11:46Z")
Mike, when I worked at a certain huge company, we had a lot of test case failure data as well as code static analysis tooling available to us. A rather clunky web service was cobbled together to show which test cases could be grouped together as likely to fail based on which “component” you thought the changes you made were in. Simply by looking at a history of how often a test would fail based on code change in an area. some of that mapping was semi-manual mind you. That’s because feature and component mappings are often, but not always directly connected and simple. It was effective at “autonomously” eliminating huge batches of test cases, (that were obviously irrelevant to a human observer,) but when you have a CI/CD system that wants to run every single test for every single code change merged, it starts to make huge sense. ---
**Author:** ![mikeharris](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/mikeharris/32/12295_2.png) [@mikeharris](https://club.ministryoftesting.com/u/mikeharris)\ **Post date:** [24 February 2022 16:12 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/17 "2022-02-24T16:12:11Z")
@conrad.connected That sounds really interesting, and a good solution for understanding existing features. The team I wrote about was trying to understand the risks on a new feature, for which there was no data. ---
**Author:** ![darren.keig](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/darren.keig/32/10211_2.png) [@darren.keig](https://club.ministryoftesting.com/u/darren.keig)\ **Post date:** [1 March 2022 14:39 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/18 "2022-03-01T14:39:17Z")
We’ve lately been trying to solve the problem of what to automate and we started using a simple spreadsheet that Angie Jones talks about in her talk on the topic ([Which Tests Should We Automate - Angie Jones – Sr. Automation Engineer, Twitter - YouTube](https://www.youtube.com/watch?v=VL-_pnICmGY)). This really helped us take a more risk based approach and identify the things that would have significant impact on the customer and that we would look to immediately fix. We used it as a whole team activity as well, not just QAs, so that developers started to gain an understanding into the idea that not all things are equally important! ---
**Author:** ![simon\_tomes](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/simon_tomes/32/14009_2.png) [@simon\_tomes](https://club.ministryoftesting.com/u/simon_tomes)\ **Post date:** [1 March 2022 16:24 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/19 "2022-03-01T16:24:48Z")
Oh yeah, that “What to Automate Sheet” is excellent. A team I worked with happened to use that and it seemed to start good conversations. Kudos to Dorothy Graham and Angie Jones. > **[What to Automate Worksheet.xlsx](https://drive.google.com/file/d/11JVP8z7KHYEsxEqkO1wul5tPfLAk1TsG/view)** > > Google Drive file. ---
**Author:** ![sam\_qa\_wise1](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@sam\_qa\_wise1](https://club.ministryoftesting.com/u/sam_qa_wise1)\ **Post date:** [16 March 2022 16:27 UTC](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362/20 "2022-03-16T16:27:53Z")
I am definitely late to respond to this thread but aqua cloud ALM has protection from duplicates. And when the number of scenarios grows, it doesn’t slow its performance. So if you will decide to switch from Cucumber try to find something with duplicate protection. [Next page](https://club.ministryoftesting.com/t/analysis-tools-to-avoid-the-test-everything-test-strategy/52362.md?page=2)