Which Load / Performance Testing Service do you prefer?

Hey everyone, my company is trying to explore a way to run load tests and we’ve tried setting up a cluster in GKE to run remote distributed testing. Before we dive head-first into this strategy though, we wanted to make sure we didn’t overlook any of the services that might be able to make the whole process easier.

Currently we’ll use JMeter scripts, so we are leaning towards services that might support that, but I’m open to exploring services that might require scripting or an API to run tests.

Here’s what I’ve looked at so far but don’t have much experience to compare between them.

  • GKE Cluster using our own configurations - higher side with costs but we can control more of the whole process and aren’t limited to anything for most part, as long as we are willing to pay.
  • Redline13 - seems to be one of the most affordable options and was pretty easy to jump into. Uses AWS so there aren’t really any limits to run time or # of users.
  • Blazemeter - has the most limitations, but I doubt I would hit the upper ceilings before needing to talk to a Sales team for increased limits.
  • Flood.io - looks like they use virtual hours to bill usage, which makes it a little hard at the moment to figure out how much it costs.

Thanks, any feedback or corrections on my current findings are greatly appreciated!

3 Likes

We just use JMeter hosted on-prem with TICK Stack

Fancy dashboarding :stuck_out_tongue: and customizable. Just perfect for us.

3 Likes

Well it kinds of depends on your use case… How many requests per second , http or other protocols required. How complex is the test scenario. Where are the servers hosted.

In most cases my advice right now is to avoid clustering:-

  1. Clustering is not needed for load scaling… WRK will do 50,000 requests per second. K6, 10,000 per second on moderate hardware.
  2. Clustering adds significant complexity even with tools that support it.

I work for a large SaaS company all our services together globally can’t handle than many requests. If you really did need more traffic than that, then network architecture and cost would be more important than toools.

So I’ve been moving teams from JMeter to K6 because it works better with version control and continuous integration. Feedback so far is good. We deploy them to out own docker clusters locally as well as AWS.
Advantage to self hosting is lower cost but also better control over CI integration.

The one caveat is k6 is open source with commercial backer. Currently clustering is not full supported in the opensource version, so if you really need big loads you either pay or it, or build your own.

2 Likes