Katalon Studio for API testing

Evening All,

I am currently using Katalon Studio for an API project and I’m pretty impressed so far, especially with the simplicity of the scripting and creating tests (I was looking at Postman as well, which I find better for the “manual” or exploratory API work (Katalon only really good when you are at the stage where you want to automate).

Just wondering if there are any others users in “The Club”?

My main question really is on the team and collaboration side of things, i.e. how you are ‘supposed’ to all work on a common team set of tests (so i.e. I create a test or update a test, and then everyone gets this in their workspace), i.e. how do you many production/master test repository. Can anyone offer any thoughts?

Regards

4 Likes

You can use Git for collaboration.

In which format does the tool store tests ? For example, if you had a Python project, then all your code would be Python. If you have a tool like Katalon, then your code might be all json or xml & your scripts, test steps etc. would be embedded inside that code. The format matters because it can potentially add a lot of noise around your changes. E.g. If you change a variable name in some script, then the code for your test might show many more changes besides the variable naming, thanks to the way the tool is designed. This noise can make it harder for others to see what really got changed.

Btw, have you asked your team why they chose Katalon studio?

PS - I don’t use Katalon, but I have seen a few low code tools like it. I admit that I don’t like such tools for automation. They create more problems than they solve & suggest that the QA org is not technically strong.

@pmfrench
I actually prefer postman over katalon for automation, I think katalons UI is more ‘messy’ but that’s just my personal opinion :smiley:

As @raghu said some kind of source control is required to work as a team, looking at their site:
=> Katalon Studio | The #1 Codeless Automation Tool

A video example:

1 Like

Thanks for the replies. As a “non-coder” I found the Katalon Studio a breeze to use and with a few code snippets it satisfies most of my scenarios. There is one thing I haven’t figured out yet (if it is indeed possible) which is to earmark a full response as being expected result rather than having to code in JSON element checks for each and every element. I know Postman appears to be the crowd favourite, and I suspect I just haven’t yet worked out how to get the most out of it. I’ll put some time in on YouTube and have a look for a course of videos.

@kristof I definitely agree about the UI - I don’t like it for doing any manual/explloratory type stuff, only when it comes to defining automated tests.

1 Like

Could you elaborate a bit more on why you are trying to do this?

I’m looking at it like this:

You have a body response which can be pretty huge like the JSON example below, You assert the whole response body: it is the same and it returns 200 OK & your test is green.

2 weeks later your test turns red stating " body does not match body response".
At this point you have no clue what went wrong and you’re in for a lot of fun debugging.

What I want to see in my tests is:

  • templateOverridePath doesn’t match
  • removePageCache doesn’t match
  • taglib-uri path doesn’t match (due to json-schema changes)

That way you know exactly what failed. It tells you instantly more about the problem.


JSON taken from (fake data of course): https://json.org/example.html:
{
“web-app”: {
“servlet”: [
{
“servlet-name”: “cofaxCDS”,
“servlet-class”: “org.cofax.cds.CDSServlet”,
“init-param”: {
“configGlossary:installationAt”: “Philadelphia, PA”,
“configGlossary:adminEmail”: “ksm@pobox.com”,
“configGlossary:poweredBy”: “Cofax”,
“configGlossary:poweredByIcon”: “/images/cofax.gif”,
“configGlossary:staticPath”: “/content/static”,
“templateProcessorClass”: “org.cofax.WysiwygTemplate”,
“templateLoaderClass”: “org.cofax.FilesTemplateLoader”,
“templatePath”: “templates”,
“templateOverridePath”: “”,
“defaultListTemplate”: “listTemplate.htm”,
“defaultFileTemplate”: “articleTemplate.htm”,
“useJSP”: false,
“jspListTemplate”: “listTemplate.jsp”,
“jspFileTemplate”: “articleTemplate.jsp”,
“cachePackageTagsTrack”: 200,
“cachePackageTagsStore”: 200,
“cachePackageTagsRefresh”: 60,
“cacheTemplatesTrack”: 100,
“cacheTemplatesStore”: 50,
“cacheTemplatesRefresh”: 15,
“cachePagesTrack”: 200,
“cachePagesStore”: 100,
“cachePagesRefresh”: 10,
“cachePagesDirtyRead”: 10,
“searchEngineListTemplate”: “forSearchEnginesList.htm”,
“searchEngineFileTemplate”: “forSearchEngines.htm”,
“searchEngineRobotsDb”: “WEB-INF/robots.db”,
“useDataStore”: true,
“dataStoreClass”: “org.cofax.SqlDataStore”,
“redirectionClass”: “org.cofax.SqlRedirection”,
“dataStoreName”: “cofax”,
“dataStoreDriver”: “com.microsoft.jdbc.sqlserver.SQLServerDriver”,
“dataStoreUrl”: “jdbc:microsoft:sqlserver://LOCALHOST:1433;DatabaseName=goon”,
“dataStoreUser”: “sa”,
“dataStorePassword”: “dataStoreTestQuery”,
“dataStoreTestQuery”: “SET NOCOUNT ON;select test=‘test’;”,
“dataStoreLogFile”: “/usr/local/tomcat/logs/datastore.log”,
“dataStoreInitConns”: 10,
“dataStoreMaxConns”: 100,
“dataStoreConnUsageLimit”: 100,
“dataStoreLogLevel”: “debug”,
“maxUrlLength”: 500
}
},
{
“servlet-name”: “cofaxEmail”,
“servlet-class”: “org.cofax.cds.EmailServlet”,
“init-param”: {
“mailHost”: “mail1”,
“mailHostOverride”: “mail2”
}
},
{
“servlet-name”: “cofaxAdmin”,
“servlet-class”: “org.cofax.cds.AdminServlet”
},
{
“servlet-name”: “fileServlet”,
“servlet-class”: “org.cofax.cds.FileServlet”
},
{
“servlet-name”: “cofaxTools”,
“servlet-class”: “org.cofax.cms.CofaxToolsServlet”,
“init-param”: {
“templatePath”: “toolstemplates/”,
“log”: 1,
“logLocation”: “/usr/local/tomcat/logs/CofaxTools.log”,
“logMaxSize”: “”,
“dataLog”: 1,
“dataLogLocation”: “/usr/local/tomcat/logs/dataLog.log”,
“dataLogMaxSize”: “”,
“removePageCache”: “/content/admin/remove?cache=pages&id=”,
“removeTemplateCache”: “/content/admin/remove?cache=templates&id=”,
“fileTransferFolder”: “/usr/local/tomcat/webapps/content/fileTransferFolder”,
“lookInContext”: 1,
“adminGroupID”: 4,
“betaServer”: true
}
}
],
“servlet-mapping”: {
“cofaxCDS”: “/”,
“cofaxEmail”: “/cofaxutil/aemail/",
“cofaxAdmin”: "/admin/
”,
“fileServlet”: “/static/",
“cofaxTools”: "/tools/

},
“taglib”: {
“taglib-uri”: “cofax.tld”,
“taglib-location”: “/WEB-INF/tlds/cofax.tld”
}
}
}

Yes I get your point there, and agree with what you say (in terms of good practice), but the option of doing this as a quick way of doing a test would be good option if the context was right (for example, if the results aren’t likely to change). I agree though that coding in element checks is better, and we’re mainly managing this through data driven approaches (managing the results on a spreadsheet)

1 Like

In this example above would you coding checks line by line?
One of our current APIs has almost 400 lines in the response, I’d rather avoid having to code a check for each line, if there is a way I could basically check the whole thing at once.

1 Like

Even if it has 400 lines I would prefer to check line by line. It might take you a while to write the tests but you’ll be so much happier when changes happen or errors start appearing. That’s my personal opinion.

But yea, you can just compare the whole json, that works too.
if something goes wrong, you’ll just have to debug a lot more :stuck_out_tongue:

1 Like

Thanks :+1:

Don’t suppose you have any tips for how to check the whole response (or at a large portion of it) in one go do you :wink:
The response has a date and time and guid in the “header” but outside of that the response will always be the same based on the test input request.

1 Like

Depends :smiley: in Postman sure, that should be something like this:

pm.test('Check entire json response, function() {
    const response = pm.response.json();
    const isEqual = _.isEqual(response, dataDrivenResponse);
    pm.expect(isEqual).to.be.true;
}

dataDrivenResponse being your file which contains the expected outcome.
In katalon … I don’t know I haven’t tried that yet :smiley:

1 Like

I have not used it for a good two years but git functionality was built in to Katalon before - Git Integration (BitBucket Integration) | Katalon Docs

1 Like

Thanks, I’ll give that a go

2 Likes

Yeah, we seem to be facing some issues with the built in functionality, and using Gitbash at the moment as a work around. I’ll take a look at that document (thanks)