How do you know if your release is successful?

🗄️ Archive
1

I’m thinking about how do we know if a release is successful or not - what are your thoughts on this?

2 Likes
2

This is a question that I think spans far beyond the efficacy of development and testing. A release may be implemented to spec and functioning properly, but the release may not be successful if, for instance: it isn’t marketed to customers well, or perhaps the initial requirements don’t actually meet their needs. There’s a lot of non-technical work that goes into making a release successful. For this reason, I always enjoy hearing from our business team as to what customers think of our last release, as it’s really the only way to know for sure if it was successful. I think frequent communication between business and technical teams is crucial for this reason.

Now, from a purely technical perspective, I would consider a release successful if we were able to deliver a high-quality product that met the requirements given in the time-frame we committed to.

4 Likes
3

I think I need to learn more about product success in it’s entirety - not sure if there’s any resources out there to look into this. I’ll have a google to see what i can find.

4

Some of these might help to create a bigger picture

  • automated sanity tests in production
  • Monitoring in production

If the automated tests in prod fail (which are inside of the CI/CD pipeline) it will automatically be rolled back.

2 Likes
5

I think it can be subjective to the team/product what counts as “successful”. So this sounds like a team activity to me. Find out what this means for you and judge your releases accordingly.
For our team, working with a highly configurable product, we are happy with a couple of critical bugs, while for others this might be bad. I think @alanr also brought up a good point. A release might be technically good but if customer reception is bad is it still a successful release?

Some ideas on what release success could be based on:

  • bugs found after release (number, severity, …)
  • usability (features, documentation, …)
  • release process (downtime, problems during release, …)
  • team feedback (speed, delay, process, …)
  • customer feedback (hotline calls, acceptance of new release, …)
  • …
1 Like
6

Releasing late with no bugs is also a failure, for the business. And probably means your team needs to make a load of process improvements. It’s fair enough to think only about what you do control as a team, but as pointed out by @sles12 and @alanr , getting to production early with a healthy strategy to deal with the issues after release can also count as success.

1 Like