Hi all,
My company is introducing an API and moving away from our current process and we will also be increasing the volume of data we will be importing. in order to make sure we can handle this we need to test that our shiny new API is A) Capable of handling the increased volume and B) If not, it fails gracefully.
Example scenario:
3rd parties submit files in a txt or csv format which we import into our system and then process, resulting in an export to either the supplier of the files, or other end users.
We are now looking to increase this load by a factor of X, but the new volume we will be processing would be… problematic, for our existing software.
To test this we need to verify that our API can handle the import, the processing of the raw data, and the export of the transformed data.
The main priority at the moment is to test the import function with large volumes of data across 2 separate scenarios, 1) Using the UI (Customer facing), 2) Directly to the endpoint.
Currently we use the (free) version of Postman to test the end points and make sure that they are working correctly, but now I am searching for recommendations for a product where we can test the volume going to this endpoint.
We used MS Test for our API performance tests. You may want to isolate some parts of this operation. For example, if your objective is to evaluate the endpoint performance, you might mock out some of the processing. If your objective is to evaluate the processing, I recommend some consideration for dependent systems during the testing.
You can also look up https://github.com/yandex/yandex-tank
I didn’t use this tool extensively, but in some quick checks it proved to be powerful enough. It has a learning curve and you also need to decide which load generator to use (phantom, pandora, JMeter, or BFG).
I’m not sure, but I think there is also some kind of UI service developed on top of it.
SoapUI allows you to do multi-threaded test runs, and it does handle REST as well as SOAP, I’ve used to do quick load tests in the past but I have to say I prefer Postman these days as my primary API test tool for its greater ease of use. However, you’ll probably be better off using a dedicated performance testing tool like JMeter or Gatling if you’re wanting to build performance tests into a deployment pipeline.
I like Postman, though I dont have a lot of exp with other API testing tools tbh. Doubt that this will be a regular test run feature, though knowing my luck it might be!
