Generate Web API tests and find bugs quickly

Sharing Lounge
, , ,
#1

There are many tools for testing REST APIs that let you build your API and write some Javascript used when executing tests. I believe these tools act as an extension to coding the tests manually, and you are still left to think about edge cases.

I have built a tool that uses Property-based testing to generate tests for an API given, e.g., an OpenAPI specification. I published an article of how I was able to discover bugs in a released version of GitLab CE: How to create 1040 tests targeting Gitlab in 5 minutes with Humlix

Somehow GitLab let these bugs slip through to production, which shows how hard it is to find bugs that require out of the box thinking for the input data.

Implementing Property-based testing code can be challenging and leaves you with more testing code to maintain.
It would be interesting to hear your thoughts on using a tool like Humlix to generate tests for your REST APIs.

2 Likes
#2

Very interesting @humlix, thanks for sharing. I will add it to my long list of things to check out!

1 Like
#3

Happy to hear that. Feel free to reach out to me if you need any help or assistance if you choose to check it out later on.

1 Like
#4

Hi @humlix,

I pretty much fell at the first step, because I cannot POST/PUT XML data.

My only choices in the Humlix app are:

  • No body
  • Form data
  • URL Encoded
  • JSON

As most of our API’s handle XML input/output I would need to wait for XML body to be supported before I can really give the tool a propper test.

Is XML Body support on your roadmap?

2 Likes
#5

XML body support is in the roadmap and currently not available; sorry for that.
I can´t promise a date for XML support right now, but I can update this thread or send you a DM as soon as support for XML is added.

On the https://docs.humlix.com/ site, there is a Getting Started guide that will give you a feeling of Humlix using a built-in test API. There is also a video on the home-page (https://www.humlix.com) showing the example of when Humlix finds bugs in production code from GitLab.

Let me know if you have any other issues with Humlix, and I will be happy to assist.

1 Like