Performance tools recommendations

šŸ—„ļø Archive
1

Hi folks
I currently use JMeter in the company as the performance tool of choice. With the increase in JavaScript-heavy sites, Iā€™m finding JMeter increasingly difficult to recommend for what are pretty limited apps with small user-bases.
I am currently doing some study on an app named Flood Element, which in the short time Iā€™ve used it appears to be more akin to an automated test framework than the JMeter experienceā€¦
My question is - have you used Flood Element or heard anything of it or can you recommend any other (reasonably priced/free) alternatives? Iā€™m specifically interested in browser-based tests that return feedback like ā€˜it takes a user on average 1.2 seconds to open the page, 10 seconds to complete a form and receive acknowledgementā€™ etc. With the ability to scale page interaction to multiple concurrent usersā€¦
Many thanks Dan

2 Likes
2

I have heard of them but didnā€™t know they were doing browser based stuff. Itā€™s not a new idea, Its popped up every few years since webdriver came out. Possibly before that .

Issue is browsers use about 100x more resource then API test tools. Flood.io have found a way to make that a little more manageable so is only 10-20x more expensive with their own numbers. Gap would be bigger against something fast like k6 or faster like wkrt.

Flood.io provided numbers, Vu/node:
20-50 element browser tests (playright).
5 selenium tests
500 API requests

Ironically for a JS heavy app, you might expect the lower end of that.

Whether that makes sense would depend on your use case:-

  • How many users you need
  • How often you need to test
  • Budget

No first hand experience, but love the concept. If the cost is manageable for you then Iā€™d say go for it.

On the flip side you are paying to burn CPU cycles on browsers which doesnā€™t test your app and may be harder to debug, Iā€™ve found often when presented with the costs that teams quickly work out a way to do the testing. Why would the relationship between the browser and API be that complex.? May be simplify the API instead, or extract the Js logic in a Library to drive load testing. Or duplicate the core logic in a load script like we usually do

I do see the attraction, I will likely have a trial myself. And would be Interested to know how you get on!

Ah of course smart bear has load ninja, and hp have browser mode for load runner now. You have also been able to run selenium tests from jmeter for a while now. Not that I recommend doing that.

2 Likes
3

bonus content

Another option is donā€™t performance test!! Turns out thatā€™s what the cool kids are doing. Performance testing is actually out of fashion. What about performance for my customers? I hear you ask!!

Well there a newer ways to manage those risks.

  1. RuM - real user metrics
  2. ApM - application performance monitoring
  3. Canaries A/B testing
  4. dark launches / traffic mirroring
  5. automated deployment/ roll backs.

It might be if you are doing regularly deployments with an automated pipeline. You might get more benefit for your users investing in canaries or dark launches than investing in traditional performance tests. If you want to discuss that further let me know.

2 Likes
4

Perhaps: https://k6.io/ ?

5

Hey @danuk ,

If youā€™d like to explore k6 I absolutely recommend chatting with performance testing legend Nicole van der Hoeven. She shares excellent content which might be of use to you. Let me know if you would like an introduction.

Good luck.

2 Likes
6

Thank you for for both your replies - I had to google some of those terms but seems sensible, given the expense of performance testing (or having a production-like environment to hand to test with).
I suspect that strategies like canaries and dark launches are more suitable for fast moving industries - not something Iā€™m familiar with :wink:
We already use performance monitoring and rollbacks but delivery here is more waterfall than agile, so there is typically a long development process and emphasis on customer confidence before final deployment. Let me do some more reading though as you raise some good points! Cheers

1 Like
7

Thanks, I will certainly look some more at K6. Am I right in thinking that Nicole moved from the Flood.io project to K6? Iā€™ve watched a few talks that sheā€™s done and they have been packed with useful stuff.
My only thought is that K6 is also based on protocol-testing rather than browser testing and I do like the idea of having browser tests that more accurately reflect complex Single Page Applications (eg MS Dynamics) - although Load Ninja seems eye-wateringly expensive to me! Thanks very much for input, this has given me some food for thought.

2 Likes