How do you design performance test cases?

🗄️ Archive
1

Hi,

I found this blog about test case for performance testing How to Design a Good Test Case for Performance Testing - Software Testing Genius and also this one How to create highly effective performance tests | TechBeacon

I’m wondering what are your tips to design a good performance test case from functional point of view ? and what tools do you use for implementing such kind of test ?

Best Regards,
Emna

3 Likes
2

In the past I asked customers to make click paths for the most important tasks. It is basically a set of screenshots and actions of the users. Nowadays I would use a screen recorder to capture the actions. E.g. this is the click path to buy a ticket and a snack for the cinema.

Five years ago I described a new way to find real life situations. I used SFDIPOT for performance testing:

Here is the corresponding Q&A:

Tooling is a challenge. In the past my team members used LoadRunner. It is very powerful, very expensive, and challenging to learn. In another performance test, JMeter was used. This is a cheaper alternative for load testing.

The costs of the test environment can become quite high, if a production load will be used for performance testing. Also take permissions and test data into account.

5 Likes
3

K6 is a breath of fresh air in the load testing tool space. Could be a good tool to identify if there is a bottleneck through your application stack. I would only use it for backends or running against APIs. If you can afford it running a load test for longer periods of time can expose issues. Think hours. This exposed issues like a lack of a database context manager for code resulting in degradation over time. Wouldn’t have found it until we ran into weird production scenarios like a failover to another datacenter/region.

2 Likes
4

Performance test cases are, for me, based on the customers needs.
A lot of customers tend to say 'we need a load test" but don’t really know the definition of a load test or about what they need.

Sometimes they mean that they need a peak test or endurance or stress. Designing the test cases for a performance test starts here.

  • Test data is so important also, how is it created? Is it available? Is it realistic?
  • Performance test environment. Is it shared?
  • When can I run the performance test?
2 Likes
5

I tend to push back heavily on people that request load testing without specifying any context. Unless you have some performance or service level objectives, you’re investing a lot of effort to measure something with no real purpose.

If you do have objectives, then it becomes an engineering problem to figure out what you really want to measure and how to do it. jMeter, gatling, etc are rarely the answer for me since just soaking the system with REST requests is usually insufficient to measure things meaningfully - i.e. it might give me a high level number, but doesn’t help engineering teams identify the bottlenecks and improve.

Sometimes the answer is to add instrumentation to the database, code, etc, before running the test, other times it’s to write smarter mocks/stubs that gather timing data, etc. It all depends on the architecture of the system, and what we’re trying to measure/accomplish.

5 Likes
6

I assume that the requirements are expressed in terms of percentages. E.g. 95% of requests for thing X must get a response in at most Y time, and all requests for thing X must get a response in at most Z time. This is to allow for real life weirdness.

I would look for cases where it’s most likely to be slow, and see if that’s still OK. Things like: the first time a new user logs in, the first session after new code has been deployed/the system has been rebooted, the first session after a long period of inactivity.

There are interesting architecture/cost/complexity questions that affect all these kinds of thing. For instance, serverless code will go cold after a while, unless something periodically pokes it. Is there something that warms up caches after deployment? Are there multiple instances of the code that share a cache? If so, are they deployed all at once or staggered?

Another area to check is integration with third party systems. For instance single sign on or payment services.

2 Likes
7

There is only one rule of designing a “good” performance test case: the test case must represent real life usage of the system under test by the real users (or other application), otherwise the test case and the performance testing doesn’t make a lot of sense.

So you need to identify the valid business scenarios and workloads and carefully implement the test case to reflect them with 100% accuracy.

With regards to the “tools” - the only strict requirement is that the tool must support all the network protocols which are involved in the communication. If it does - it can be used. Other criteria could be:

  • Your company (or you personally) could possess a license for the paid proprietary solution, in this case it makes sense to utilize it
  • Some tools allow test creation using GUI like JMeter or Neoload, in some you need to define the test scenario using XML or YAML files, in some you need to write code, some have record-and-replay functionality, some don’t
  • Some tool can scale (run in clustered mode) some are standalone only

An example research and comparison can be found in i.e. Open Source Load Testing Tools: Which One Should You Use? article

2 Likes