At TestBash Brighton I facilitated a poster stand at the UnExpo. My stand was titled âPerformance Testing Strategyâ and I split my poster into 4 categories: Types of Tests, Metrics to Report, What to Test and Tools.
The eagle eyed amongst you may spot âExpertsâ in the section below, but my discussions didnât really head in that direction. I added Mark Tomlinson (@markontask) myself.
Better late than never, hereâs a list of ideas that people came up with during the lunch break:
Types of Tests
- Stable Load - a sustained load on the system under test. An âexpectedâ number of users.
- Small Load - does the system meet our NFRs under a small expected load?
- Stress - Putting above maximum expected load on the system to identify the point of failure. Helps you identify how much your system can handle before it fails.
- Soak - a test which runs for an extended period of time. Allows you to identify how your system handles a specified period of concurrent use.
- Spike Test/Realistic Varied Load - does the system handle going from a small number of users to a large number quite quickly?
Metrics To Report
- CPU Usage
- Memory Usage
- Response Time
- End to End scenarios - how long it takes to perform a specific user journey.
- Trends - isolated metrics are great - but is our performance improving (or not) over time?
- Pass or Fail - I.e. does it meet our NFR?
- Network Traffic
- Disk IO
- Number of passed/failed requests
What To Test
- Scalability - is our system scalable during a Spike test, for example?
- Databases - are our database queries quick enough? Can they handle the load from the API?
- Concurrency - if we have high back-end activity, is our UI still performant?
- Prioritised Areas - focus on the areas most important to the business.
- APIs
- SLAs - are we meeting the requirements that weâve sold to the customer and are contracted to?
- Monitoring - is our APM solution reporting what we need? How easy is it to identify issues in Production?
Tools
- JMeter - web app, databases
- Gatling - web app
- WebPageTest - plug in your websiteâs URL and have it tested from multiple browsers and locations
- Apache Bench - allows you to identify how many requests per second your application can handle
- HP LoadRunner
- Visual Studio Load and Performance Tests
- Artillery - highly recommended to anyone first getting started in performance testing!
- YSlow - analyses web pages and identifies why they are slow
- K6 - a JavaScript load testing tool.
- NeoLoad
- PageSpeed Insights - analyses your web page and identifies some potential performance improvements.
Other Takeaways
- Most people appear to be in the same boat when it comes to performance testing. Some companies have dedicated performance testing teams, but other, smaller, companies are having to do performance testing within scrum teams. This often leads to an âad-hocâ and inconsistent approach.
- Teams are trying to balance quicker delivery with having sufficient performance testing coverage.
- A lot of people I spoke to bemoaned the lack of sufficient application monitoring in their production software.
It was great to just chat to people about performance testing. It seems that performance testing in smaller companies is the responsibility of people who may not necessarily be experts in the area.
Itâs definitely an area to explore further if it interests you, as it can become extremely valuable to your teams to have performance testing in place if you donât already have a dedicated performance engineer.
