Hey guys,
there is something I need some input for…
we have to specify non-functional requirements for every quality characteristic, e.g.:
Main Characteristic: Efficiency
Sub Characteristic: Runtimes
REQ_1: every response to a user-request occurs in less than 250 ms.
REQ_2: the server-startup needs less than 15 seconds.
So, for Efficiency there are some ideas and things you can meassure. But what about the other characteristics like Usebility or Scalability and so on. I have no big ideas, and the requirements should mostly be measurable, right?
Do you know a list of examples or know how I could define specific requirements for all the sub characteristics? Thank you.
I must admit, I had to look up “non-functional requirements”, as I have never used, or even heard the term before. After looking it up, I have ideas.
First, the external reference.
This document is a list of things you might want to consider for your testing (either requirements or otherwise). A lot (maybe most) of the list is consistent with what I just found on non-functional requirements.
Second: While some quality characteristics may need to be measureable, not all of them are.
For example, if you or your customers hate working with your UI. How do you measure that? If the UI makes your eyes bleed, would you measure the quantity of blood? How do you measure Intuitiveness? (This is especially interesting as a tester, since a lot of what is intuitive to me, is not to a non-tester, since I have been doing this for years, and most of my users have not).
The reason they’re called “non-functional” requirements is because they aren’t, strictly speaking, about what the application does so much as about how it does what it does. Naturally, a fair number of non-functional requirements are rather hard to measure, and need functional proxies.
For instance:
Usability - Since there’s no direct measure of usability, you can use a proxy something like “All features used daily will be linked on the main page.” i.e.: once user’s have entered your application, they will only need one more click to do the activities they do most often. Another helpful proxy is “when viewed through a color-blindness filter, all parts of the screen are clearly visible” (I use RGBlind with Firefox to check the sites I work with)
Intuitiveness - Again, there’s no way to measure intuitiveness, so you need to be imaginative with your proxies. Naming conventions help - they give guidance such as labels that say what you should be doing. It’s the difference between naming the form submit button “Finish”, “Save”, “Save and Close”, “Close”, or “Submit”. Or “Done”. The closer your field labels match what the form is doing, the more intuitive it will be. Another possibility if you can do it is to have a set of test users drawn from your company and watch them interact with the application. The more they can figure it out for themselves, the more intuitive the app is - which is something you can measure (help requests per hour)
Scalability - Scalability is something that can be measured with load tests and a lot of infrastructure. You start with a production-equivalent environment and a load test that mimics normal user patterns. Then you run the load test repeatedly, starting with something more or less around the same as the current user load, and increasing the load by a set amount each time until you either break the system, or meet whatever the scalability goal happens to be (it might be “can handle up to 50,000 logged in users without performance degrading” or “can handle 50,000 simultaneous logins without performance degrading” - the exact goal will depend on the application).
The short-short version is you look at each non-functional requirement and work out how it ties to your quality characteristic (and if it does), then if the requirement isn’t something you can directly measure, you work out what you can use as a proxy for it and go from there.