I had a bad experience in the last agil projekt with the team and their expectations while getting emails from night run results.
The approximately 200 regression tests E2E run on all stages about 6 hours. Next morning after the changes on dev, thus stage fail several tests. After maintaining first on dev it took about two weeks till the other stages are fixed .
They gave me the guilt for the time consuming and not have all green on all stages all ways
My suggest was not sending the email after night run automatically. Instead give only information about real problems. But that was not an option for the PM and TM . So how you would react?
I can relate. I am the sole person that maintains multiple automated test suites and when they break badly it can take weeks to fix them. Although I can understand your feelings of guilt for it taking lots of time to fix them I would say that if anyone should feel the guilt it should be the person who merged in the breaking changes in the first place, I don’t think there should be any guilt though really. This ultimately comes down to team buy in, if they want the benefit of the confidence that comes from the tests passing then they need to pay the cost of maintaining them.
In regards to informing the team about failures, I am the filter for the test results. So if some fail I will investigate and then inform the team whether they are failing due to genuine bugs or if they are just tests that need updating.
Personally I wouldn’t accept any pressure from my team to fix failures under a particular time constraint, they take as long to fix as it takes.
Hello Arian!
Thank you for this response. I can say that this project was to blame me and they (the management) wanted to change from UFT to Selenium and they where looking to find a way to kick me out. The never used ALM and loojed after my effort to realize the automation. And as i see how the Selenium guy work i was surprised how low the real object recognition in web object was. Using the dom. He never knew the real position and ask me hiw i did. 2 weeks later i was out from that project.
Sorry to hear about what happened to you. In our automation projects that make use of CI/CD pipelines, we have a primary pipeline and a secondary pipeline. The primary pipeline includes only the stable tests. Any test cases that are new, edited, or need updating are routed to the secondary pipeline. The product team receives only the results from the primary pipeline. Automation engineers review the results from the secondary pipeline and then send out only valid app failures to the team. Once a test is updated and stabilized, it is moved from the secondary pipeline to the primary one.
“LogiGear - here to help shape the future of test design"