Hello,
I would like to hear from all of you about the API automation tool that you have expereinced and would love to hear why you have choosen it ?
Do you happen to expeirence automated API tests with in CI/CD pipeline of your software delivery process ?
What is your test workflow ? Please share all your API test autoamtion expereince
~ Cheers
Musaffir
For API automation, I use C# for the projects I’m working on. It’s the language of the application I’m testing, and my tests live with the rest of the code. I chose it for that reason, as well as the fact that the devs don’t have to get out of their usual work environment to run my tests. It also encourages them to write API automation, which they have 
For CI/CD we use TFS/VSTS for our .NET framework projects. The process is basically: build code; run unit tests; if all pass, push to TEST env; run automated tests. Because other teams use our non-prod environments, I also do manual testing after things are pushed, then manually approve it to start the process again in BETA.
there’s a lot of great info here if you’re looking for more : http://katrinatester.blogspot.com/2015/09/api-web-services-microservices-testing.html
I used REST Assured because it can be used with Java and run with Maven using unit tests, so it integrates nicely with any CI/CD pipeline.
I’ve been using it in many projects and it is so easy to setup and write tests on it. My reason to choose it is mainly because I’ve got previous experience with Java and Maven that for me that was the obvious tool to use when needed to introduce API testing. Another advantage of it is that it uses the structure given/when/then to write the test so it is very intuitive even for people with non advanced coding skills.
Thank you so much @g33klady for your valuable time and putting details here and I must say its a wonderful process 
Thank you @stardust for sharing your experience as well here 
I have some experience with rest-assured and I too found it easy with its fluent style apis and given/when/then structure.
But I guess you need to spend some time initially to set up the automation test framework using res-assured, and its full of coding (in java) … it could also be taken as an advantage for you as it has all the feasibility of taking ALL the benefits that Java gives you.
I suggested this at my company, but the management was not so impressed especially when they understood that it requires coding to develop tests ( though I personally likes it )
Currently I am looking at a tool which can speed up the process and we can develop tests more easily and start integrating it with our CI/CD pipeline
I have experience in SoapUI too which we didn’t take forward because it need to be installed/present in the machine where the test is running. So that would be an expensive one for us especially if we need to configure the all the machines where build is happening, if we go with pro verson… and the free version has its own issues
Recently I have been playing a lot with Postman.
I liked postman for its simplicity, ease of test development, ability to incorporate Javascripting etc and especially the power of automating the tests with Newman command line tool. So you can easily make the tests as part of your CI/CD pipeline.
One slight inconvenience I saw with Postman is , it is not so easy to integrate with version control system.
It has a real time sync option with Github, but not with Atlassians Bitbucket ( the one we use at my company 
). So you would need to manually export the test files from Postman GUI and version control it . That would slow down the process and may bring some issues at later point of time especially when Team with multiple people start developing the tests.
So my journey continues in fixing a proper tool for achieving api test automation 
Cheers
Musaffir
In terms of test applications, I use:-
Thanks @chris.adams for your response. Very kind 
Hay @musaffir, what is your future test strategy / process in your company? Do you want to outsource the tests to the developers, so that is a collaborative Tool needed?
cheers
sobsi
Hello @sobsi06
We have no intention to outsource the tests to developers.
(at least we want the test to be part of the application code base itself, so during development work if dev wanted to run tests (say regression tests) against their changes, that should be easy…)
and we want the all these things to happen in single pipeline as part of the build system ( we use Atlassians Bamboo)
So pipeline will be like
Dev committing code > Build system pull these changes > Building it > Running Unit tests > Deploying to test server > Running automated integration api tests …
So its important for my tests to be version controlled and maintained in a version control system ( we use Atlassians Bitbucket),
There I see inconvenience if I go with Postman for developing tests especially i our eco system.
I could have used Res-assured and resolved it, but for some other reason( lots of coding) we didnt take that as well.
Please let me know your thoughts
Cheers
Musaffir
Hi Musaffir,
I blogged about how we chose an API tool on my blog, here: http://testernaut.blogspot.com/2018/05/choosing-api-tool.html - this speaks to how we chose, languages etc.
Shortly after this, we found that frisby.js had some flaws and we switched to IcedFrisby.
There was a feature we needed for a particularly rough API that required cookie authentication (ew), which required some changes to IcedFrisby to support. We could have implemented the bits we needed via some hacks our side (probably), but contributing to the repo seemed like the best way, and one of the devs was encouraging me to get involved in Hacktoberfest. So I did, and I improved some docs and other things whilst I was there (and earned my Hacktoberfest t-shirt!)
The company now has a very decent collection of API tests, and I’m a maintainer on the repo.
On your question about CI: we used a CI system to monitor our live environment using a subset of the tests (because regularly testing login and other business logic operations is way more effective and informative than pinging). We wanted to get to what I think you’re talking about though - builds being automatically deployed and tested, but we got scuppered on the “deploy” part - it was non-trivial, and wasn’t worth the investment whilst the company was on a path towards .NET Core and Docker.
Hope this helps!
Dan
Hello @dancaseley
Thanks you so much for replying and sharing your blog post, It was a wonderful read to me , am sure I will re-read it again
I didn’t know about frisby.js, I will explore the same and try to learn it . I will get back here with my findings and comments
Warm Regards
Musaffir
Hello @dancaseley
I found frisby.js framework as really cool, but I guess it need coding efforts to develop the tests . And as this based on Node.js and java scripting, the team should have competencies on those technologies. I have been playing with Node.js, NPM etc earlier, so I feel a bit ok with this. But my team is not technically aware or used to javascripting etc. So I guess it will take a lot of time for Team to get used to it and start developing the tests .
This was the same reason we decided not to go ahead with JAVA based Rest-Assured framework.
we are looking for some tool which can help us in speeding up the test development activities. Like Postman but it has its own few inconveneince as mentioned in other replies.
I will see what tool best suites for us and find a balance
Thanks for your valuable time
Warm Regards
Musaffir
A type of software testing which includes application programming interfaces (APIs) testing as a part of direst as well as integration testing in order to determine if that meet expectations for reliability, performance, functionality and security is known as API testing. API testing enable communication and data exchange among two software systems those are separate from each other. A software system putting an API contains of several functions/sub-routines those can be executed by another software system. API testing is nowhere similar to GUI testing as it works on business logic layer of the software architecture of an application.
There are several tools for API Testing like SoapUI, Runscope, Curl, Cfix, Postman with jetpacks, dotTEST and Eclipse SDK tool - Automated API testing etc.
Out of all these tools SoapUI tool is the best preferred tool considering its following listed features:
SoapUI has been built specifically for API testing.
It supports all the standard technologies and protocols to test all kind of API’s.
Quick and Easy Test Creation - Drag and drop.
Helps in performing Data driven testing.
Provides Script Reusability.
Provides multi environment switching.
Integrates with several other tools like Maven, Hudson, JUnit etc.
It also provide Vulnerability Testing.
Can create a custom workflow /functionality using Groovy.
It can also estimate a web application’s load balancing capacity.
Security, Functional and load testing.
It is protocol dependent instead of being Browser dependent.
Note: Some of the above listed features are only available in the SoapUI NGPro.
Hope this information is helpful for you.
I use the Postman for API automation testing. And I found that we can run all Postman collections in CI/CD by using the lib Newman - the cli for running postman on NodeJS env.
Hope this help.
We have just started API-level testing using Curl and Postman/Postman Inspector/Newman, we decided initially on these tools as most of our developers have experience using Postman and it integrates quite easy into our CI process.
However, we are currently evaluating other tools to see if they’re a better fit.
We’re still currently finalising our workflow, but it goes along the lines of:
Build env -> run unit tests -> if passed, run test automated tests -> if passed, push to master branch -> release
I’ve been using Postman and was able to put together a useful regression suite for a specific project at my company after playing with it for a while. Danny Dainton’s Github series, including his template for newman reports are life-savers!
I’ve also been trying to explore more flexible solutions, particularly javascript frameworks since that’s what our devs use the most. I’ve tried supertest but never got around to deep-diving into it. @dancaseley, IcedFrisby looks very neat and I will look more into it. Thanks for bringing it up
Did you heard about Katalon Studio? If not, I can share somthing about this cool tool which I very love.
Katalon Studio listed as a Top Automation Tool by Microsoft https://www.katalon.com/resources-center/blog/top-automation-tool-list-microsoft/.
Katalon Studio is an automation solution for API, Web, and mobile testing. The application integrates all necessary components with built-in keywords and project templates into a complete automation framework. The tool was developed to support various testing technologies (Data-Driven Testing, TDD/BDD, Page Object Model, etc.) and integrate with the modern ALM systems (JIRA, qTest, Slack, CI, Katalon Analytics, etc.).
Expecially, it is a free tool. And the Katalon support team is very helpfull too. So many companies have started working on it. I used it to get started with API testing too.
Hope this helps.
I’m chiming in a bit late, but maybe this will be helpful to someone else who is doing research. It may be helpful to read user reviews of various API testing tools to get a feel for which one best serves your needs. You can check out user reviews for the top-ranked API testing tools on IT Central Station: https://www.itcentralstation.com/categories/api-testing-tools/tzd/c809-sf-50
If you read some individual reviews for some of the top-ranked tools, you will get more insight into specific use cases and pros/cons. In your case, for example, you can look at ease of use, andspeeding up the process. As an example, one user on our website, a test analyst, reviewed Parasoft SOAtest, and noted the time saving aspect: “Using this tool, we were able to set up a nightly, still growing, company-wide regression test – dramatically decreasing time being spent on (manual) regression testing.” (full review here: https://www.itcentralstation.com/product_reviews/parasoft-soatest-review-45035-by-agiletes080c/tzd/c809-sf-50)
I have used the following API testing tools/frameworks-
Each one have their own advantages and disadvantages. Personally, I love Postman as they have a solid UI interface and it is easy to use and perform API testing.
If you are looking into integrating API tests with UI tests; to avoid using one tool for UI testing and another tool for API testing, you can use a tool that I am currently working on called Testim. We have given the ability to integrate your UI and API tests into one single test and there is no context switching between different tools/frameworks.
So, I would suggest evaluating the list here and the recommendations given by other people in this thread and see what meets your needs.
Best of luck!!!