Are these acceptance criterias correct?

Hi, in a meeting with the product owners we defined a list of performance requierements. I got feedback from the solutions architect regarding the acceptance criterias one of them, because as his says is very “ambiguous”. What do you think? Can it be improved?

User story: As a user I must be able to send 3000 messages every day and get confirmation of those messages in a reasonable* time.
*We wrote reasonable time because the system is new and there is not information about how long it takes for a message to go through all the systems involved, but we wrote a note that says that around 2 min per message sounds acceptable)

Acceptance criteria:
1: It takes 2 minutes for a message to be processed through the whole system.
2: The system must be able to process 500 messages in one minute. *
3: The system must be able to process 6000 messages in one hour . *
4: The system must be able to process 17000 messages in one hour . *

We have statistics regarding the volume from the previous system that will be replaced. So those criteria are the worst case scenario that I have seen analyzing the data. Is it possible to reduce the 2-4 into one? I guess that we are not going to run a test for a day :smiley:

My immediate reaction is that, if the system takes 2 minutes to process 1 message, then 500 messages in 1 minute seems unlikely.

But I know what you mean there. So why not replace 1 minute with “per minute.”
Then afterwords, the word “sustainably” could help with establishing scale.
And the consequences of overloading the system might be addressed, such as “without slowing down” or “without rejecting messages” or even “without errors”.

Then I would be sure to run the test for more, until one of the afore mentioned faults happen, just to see which dies first, my test program or the system (and more importantly, when does it die).

And the current worst case scenario might not be enough. What happens when you grow? The worst case scenario will just get worse.

1 Like

Suggestion: use “Criteria for non-acceptance”, as they are (in my experience) more easy to communicate about.

Criteria for non-acceptance

  1. We will send 20000 messages spreaded over 24 hours. If the processing of any of these messages takes longer than 2 minutes, the product’s performance will be considered ‘not acceptable’.
  2. We will send 500 messages in a burst over 1 minute (60 seconds). If after 1 minute (60 seconds) those 500 messages are not processed, the product’s performance will be considered ‘not acceptable’.