Ask Me Anything: API Testing

I believe SoapUI is still used as Postman doesn’t support SOAP services and those are still out there!

1 Like

Questions that could use Community input!
These are questions I don’t know the answer to - API testing is a huge area, and there’s lots I haven’t done yet :smiley: Hopefully the community can answer some of these questions!

Is this for mobile apps? I believe you can do that with Charles, as well as Fiddler - set up a proxy from your device so you can see the traffic on your computer that is going on the device. I’m not a mobile tester, so would love some other ideas here if anyone has them!

Great question for the community! I don’t write my API tests for performance testing (we have a separate team that uses their own tools for that), so I’m not sure which would be best. Folks, feel free to chime in on this one!

I haven’t used them, but in general I like to see the code I’m working with. “codeless” to me just means configurations that creates code that you can’t see/touch. I prefer to have full control over the automation I’m writing. Maybe other folks in the community have experience with these, though?

The same tools you’re using to write individual tests should work for end to end as well. I haven’t worked with Kafka so I’m not sure of the nuances there

This isn’t an area that I’m very experienced in - I’ve used mocks in unit tests, but not for developing against. I know there’s a feature in Postman but beyond a brief tutorial I haven’t used it.
I suggest googling or asking the community!

I have never tested gRPC but I’m sure the community has some great ideas!

Unfortunately I’ve never used jmeter so you’d need to ask the community this one!

I’m not familiar with proto messages, so I’m not sure how mocking them would differ from mocking otherwise. Maybe the community can help?

As I said in the AMA, I haven’t had the opportunity to work in the GraphQL world so I’m not sure. The community could help on this one!

1 Like

Swagger or OpenAPI is souped-up API documentation that is generated by the code itself. It’s a quick/starter tool for manually testing an API but it’s not customizable as a user. It’s more for a single session at a time, and not saved
Postman/Insomnia/etc are customizable and you can save those settings and share with your team etc, build automated tests within them, etc

1 Like

I have not used any tools that do that. One thing to remember is the Swagger/OpenAPI documentation is generated from the code, and more often comments in the code (not the API code itself). This means it’s not always accurate or up to date.
They could be a good starting point, but I don’t have any experience.

1 Like

I think of it more like a cloud within a slice of the pyramid, if that makes sense. They all kind of mingle together on that section. Or maybe all side by side? Hope that makes sense!

1 Like

This is certainly still integration testing, in my opinion.
For how to effectively test them, I would think about it like user stories or flows. That way you can prioritize the testing as well based on priority of those flows/processes.

2 Likes

Both of these types of testing have their place. Contract testing is done when you’re consuming an API from a 3rd party generally (whether it’s another team, or company). These are good to help you both (provider and consumer) ensure that changes being made by the provider of the API don’t break things the consumer(s) need/use. This is down to things like datatypes as well as the structure of the responses and response code/response body pairings.
Functional tests actually make sure things still function the way they used to! A provider of an API may or may not make any changes, but we need functional tests to make sure we didn’t screw things up either.

1 Like

@lgibbs captured sketchnotes from the session too!

1 Like

I see those as one and the same, so I’m not sure how you’re differentiating here.
I answered a question during the live AMA about separating tests between API and database, for instance, if that’s what you’re getting at here?
For that, essentially “it depends” of course, but you need to see what your database is doing vs what the API is doing, and verify they’re doing their things correctly.
For instance, if a DELETE from the API isn’t actually supposed to delete from the database (it sets a flag in the record, a type of “soft delete”), then you’d want to check both the API and database for that operation. But if it’s really supposed to remove from the database, then an API GET for that same item you deleted would suffice in my opinion.

I think they both have their merits. I think there are some details here I’m not getting that would better inform an answer. I hate to say “it depends” but it really does! Feel free to provide more detail and I’ll try to answer better :smiley:

This depends on the API you’re testing. It looks like postman does support SOAP (which I didn’t think it did), so that might be a good place to start. There are a lot of great tutorials out there, and it’s a pretty well used product so you can get support (on The Club, slack, or twitter) if you’re stuck.

I have no idea lol I wish I could time travel… not sure what you’re referring to here unfortunately

That’s something you’d need to Google, I really don’t know. I would assume not - you might need to use SQL Server Management Studio or similar to do that

First, I think that anyone that works in software is very technical, we just have different specializations!
I think if they’re not familiar with how APIs work, I’d start there. There are some great courses on The Dojo for that like https://www.ministryoftesting.com/dojo/courses/introduction-to-http and https://www.ministryoftesting.com/dojo/courses/the-building-blocks-of-the-internet-mark-winteringham

Once they have that familiarity, I think folks can do some tutorials with a tool like Postman to become familiar with that and how it interacts with the APIs.
There’s a lot of value in using the same tools as the dev team, so having one of the devs run folks through how things work with your APIs in Postman would help, too.

I’m not super clear on the question here, but if you’re just looking to make sure tables are populated and there’s no existing API endpoint, you can query the database (assuming you have access to do so)…

Mountebank is used for service virtualization, so not really testing all services completely - assuming you’re using something like supertest with mocha to connect to the APIs…
Pros are you’re able to test the various APIs in isolation
Cons are if you rely on Mountebank too much, you might miss critical issues. It’s great to test in isolation but you need to see how they’re working together to get the best picture of your APIs health from these tests.

For contract testing, PACT is an industry standard I think, and is supported in multiple languages.

I have never tested gRPC but I’m sure the community has some great ideas!

I’m not familiar with proto messages, so I’m not sure how mocking them would differ from mocking otherwise. Maybe the community can help?

I would approach this the same way I would testing microservices which act the same way in some cases.
Test the APIs in isolation, mocking what needs to be mocked.
Then testing user or process flows without mocking, to ensure the connections work still etc.
As for the tool to run the tests, that depends on what tool you’re using to write them. They can be written in any language, really, and lots of tools exist to make that easier for folks that don’t do much coding. For the non-coded tests, that depends on the tool (setting up “playlists” of tests etc).

If you have consumers of your api, this is very important to check (and hopefully have consumer-driven contract tests that check that).
Otherwise, the schema may not matter as much as the data. For instance - does it matter that what used to be a String value is now returned as a DateTime value? Maybe not, as long as the data is the same.

1 Like