I’m thinking about how do we know if a release is successful or not - what are your thoughts on this?
This is a question that I think spans far beyond the efficacy of development and testing. A release may be implemented to spec and functioning properly, but the release may not be successful if, for instance: it isn’t marketed to customers well, or perhaps the initial requirements don’t actually meet their needs. There’s a lot of non-technical work that goes into making a release successful. For this reason, I always enjoy hearing from our business team as to what customers think of our last release, as it’s really the only way to know for sure if it was successful. I think frequent communication between business and technical teams is crucial for this reason.
Now, from a purely technical perspective, I would consider a release successful if we were able to deliver a high-quality product that met the requirements given in the time-frame we committed to.
I think I need to learn more about product success in it’s entirety - not sure if there’s any resources out there to look into this. I’ll have a google to see what i can find.
Some of these might help to create a bigger picture
- automated sanity tests in production
- Monitoring in production
If the automated tests in prod fail (which are inside of the CI/CD pipeline) it will automatically be rolled back.