30 Days of API Testing Day 2: API Exploratory Testing

šŸ“† 30 Days of Testing
,
1

30 Days of API Testing Day 2 challenge is:

Day 2: How would you approach API Exploratory Testing?

I was thinking about how you might exploratory test an API. I wasnā€™t sure where Iā€™d start so took to Google for some inspiration which returned an article by @maaret on the Dojo from a few years ago ā€œExploratory Testing an APIā€.

So I will be using this as inspiration for todays challenge.

11 Likes
2

I started reading that too, itā€™s a well written piece. I think my brain just refuses to not document the tests Iā€™m doing in a way that can be repeated, and that others could execute.

Iā€™ll do exploratory if I donā€™t have all the facts, like with a legacy application that barely anyone still at the company understands, but Iā€™ll still document it like I would if I was planning from a spec. As I learn, I fill out the blanks in my plan. I wonder if this still counts as some kind of exploratory testing, only with a retroactive test plan :slight_smile:

2 Likes
3

Very good article - explore with various of input data patterns. I.e. decimal vs whole number vs alphanumeric vs numbers with dashes etc. Or explore handling of different street address formats etc.

2 Likes
4

Not wanting to be too-tool-focused, but I always pull Postman when I want to explore an API.

Itā€™s very easy to create complex requests, add cookies and headers, use heavy data do boundary testing, etc.

Chrome Network tab is another great tool.

2 Likes
5

I want to give a shout out to insomnia.rest as an alternative to postman that has a better UX

1 Like
6

My approach is the following:

  1. What are the requirements and documentation on api? If any, I would begin with testing of them. Are they full, unambiguous, сonsistent etc. Is there something to improve on them?
  2. Try to understand how should api work in the whole according to the information gathered from requirements, documentation,talks with api developers (if possible).
  3. Range all api features by business priority (basic, additional, extended). It will affect the sequence of testing proccess.
  4. Define all the possible situations with exceptions and errors, and how api should resolve them.
  5. Perform api testing with simultaneous monitoring of possible api changes in order to adjust test ideas.
3 Likes
7

I based my answer on Elizabeth Hendrickā€™s ā€œExplore It!ā€, which includes passages regarding API Exploratory Testing. My current approach is to:

  • Determine how and by whom the API is intended to be used
  • Gather examples of intended requests and responses
  • Add variation to requests by changing type, range, and length of variables and/or omitting some variables
  • Analyzing changes in the responses and look for clues that can tell how the scenarios I tested are influencing the response
4 Likes
8

I donā€™t have a lot of experience with exploratory testing, so some of this may be cheating :slight_smile:

1 Like
9

I think I would approach API exploratory testing in a similar manner to exploratory testing other areas of a system:

  • get a basic understanding/overview of what the service does, and itā€™s context
  • try calling the service
  • observe the data required to make the call and observe the data returned
  • develop a picture of the contents of call and response and identify potential variations and resultant impact
  • try various calls and relate the changed behaviour to any functionality/business rules
  • continue this approach, applying techniques such as heuristics to exercise the API in positive and negative response scenarios
  • if necessary, target some areas with missions/charters/tours to focus on potential risks
4 Likes
10

Iā€™d start in an ad-hoc way seeing what it is the API is supposed to provide then build out charters based on specific functionalities. Then figure out any automation that can be made to make setup quicker, e.g. if tokens are needed use Postmanā€™s test function to auto pull and fill tokens. Setup the ENV variables along the way.

For the testing itself I use key charters to determine the risk factors and where testing is needed. Endpoints that have GET only access Iā€™d use to verify data and find ways to pull all, none, hidden data etc. For POST, PUT Iā€™d possibly look at basic security issues with SQL injection (still a top ten OWASP security vulnerability) but also incorrect formats etc.

Finally Iā€™d try and look for the upper bounds, if I can, of how much data I can pull/request DoS style.

2 Likes
11

Being new to API testing here are some of the steps I would take for testing the API.

  1. Explore the documentation for the API if any exists.
  2. Ask around for what the intended uses are and what applications we know of are currently using our API.
  3. Review the code for the API or go over the API with the developer if possible.
  4. Make a note of requests, expected responses/results and error messages.
  5. Test each request validating that the input is validated.
  6. Test upper/lower boundaries for input.
  7. Test to make sure I get the expected responses.
  8. Test error conditions in cases of invalid input.
  9. Test versioning and how package is updated.
  10. If integration/unit tests exist review those. If they do not exist suggest or write the tests.
4 Likes
13

This is my approach. I canā€™t think of anything to add to this. Thank you.

1 Like
14

Great article ! This is a must read .

15

I would explore the following items:

What is the API we use in our product
What is the devs specific purpose/use of the API
DX/UX - apply the DX to the US concepts
Repetition for discovery and learning

1 Like
16

Maaret covered it so well in her article that itā€™s hard to add anything :slight_smile:

Beyond varying payload data and headers, exploring around business scenarios and user privileges, I have a personal favorite of ā€œsometimes mandatoryā€ fields in complex data types (as in, a generic CV entry object with school/work/etc. as a type element and the profession element mandatory for work entries only).

I also use heuristics such as SFDIPOT or SLIME for inspiration.

Finally, overreliance on the GUI for enforcing business rules is a fruitful direction. Iā€™ll probably be repeating that one a lot during this month but somehow the idea that a Web GUI is just another API client and if you use a different one you still need to ensure your inputs are validated/cleansed etc. is foreign to a lot of people. Havenā€™t they read any castle siege stories or what? :wink:

1 Like
17

My Approach is below:

In order to explore any API, first of all we should have basic understanding of APIā€™s and its testing

Pre-requisites :

Know about the type of web services specifically, is it SOAP or REST

You should be familiar with XML, JSON formats and syntax.

What is the structure of the message like header, body, parameter etc.

Then should know some of the clients like PostMan, and Advanced REST client for Chrome and how it works

You should know what is a API Request and What should be there in the response?

What are the HTTP methods current service/API supports - like GET, PUT, POST, and DELETE.

Most important is knowing how to configure a request in PostMan/ Ad RestClient

You should know about list of HTTP status codes and what it mean.

After that Explore given service:

Read/ Explore the documentation if it is available and try to understand.

Know the endpoint, and what are the operations it supports

After getting basic idea

Try to call the service by carefully looking for what are inputs required to make a proper call, and check the returning results.

Understand how the invalid requests and exceptions has been handled.

6 Likes
18

I donā€™t know a whole lot about APIs or testing them, which is why I am doing this challenge!

However I have started to learn a little about POSTMAN. With my limited knowledge, this would be my first tool if I wanted to start poking around in an API. In an interview with Amber Race over at
Automation Awesomeness, Joe Colantonio writes:

When most folks think of automation they donā€™t necessarily think of using test automation tools, but thatā€™s a mistake.

In Amberā€™s experience, when she is performing back end testing she finds herself using Postmanmore and more to assist her in exploring her teamā€™s APIs. It really helps her to learn what an API does.

The main way she does this is by tweaking the API request to see what comes back. Itā€™s a really useful tool for both automated and manual testing.

She explained that she is often given a scenario ā€” load testing a web server, for instance. In situations like this you donā€™t always know a whole lot about what the web server is supposed to do; you just expect to load test it without any context.

To understand how to create a realistic test, you need to really look into the game to see how itā€™s communicating with the server, then try to replicate those calls using a tool like Postman. From there itā€™s just a matter of seeing whatā€™s there and determining what works and what doesnā€™t.

Hereā€™s a brief tutorial about sending API requests with Postman.

-Dave K

3 Likes
19

From our friends participating on Twitter:

20

Iā€™d start with reading Exploratory Testing an API post by Maaret PyhƤjƤrvi :slight_smile:

Iā€™d use exploratory testing techniques I already use for testing systems across the entire stack such as:

  1. Map out the endpoints & associated params of the API under test.
  2. Map out the endpoints & associated params of APIs that communicate with the API under test.
  3. Compare responses with their requests
  4. Experiment with different variations of params to see the API behaviour.
  5. Identify example messages from typical user journeys to derive variations of those journeys (e.g in/valid tokens).
  6. Replay troublesome user journeys from production through the API to reproduce undesired behaviour.
  7. Identify data being passed through the API & use the STRIDE threat model to discover vulnerabilities of that data.
  8. Compare the API with itā€™s documentation to identify discrepancies.
  9. Force the API to fallover - how is the error handled? What is the impact to the customer?
  10. Open the console in Postman! (Top tip in Alanā€™s video)

EDIT Iā€™ve just re-read Ash Winterā€™s ICEOVERMAD mnemonic
Daniel Donbavandā€™s SPEEDLOADS mnemonic & Linda Royā€™s VADER mnemonic - these would serve as great starting checklist from which I could launch deeper explorations.

3 Likes
21

I started to learn about API Exploratory Testing by reading post by @maaret on Exploratory Testing an API and Johnny Mnemonic ICEOVERMAD.

Following are the points that I would cover while doing Exploratory Testing of API-

  1. Explore the Scope of the APIs developed:
  • Parameters,Inputs,Constraints in API.
  • Requests/Responses
  • If the API/APIs are dependent on other APIs (Nested APIs.)
  • Server being used
  • Error Handling (Exception, Error messages being used)
  • Flowchart of the APIs.
  • Interact with developers who have developed the APIs to get the in depth knowledge.
  • Interact with the users( developers who will use the APIs in their code) to know the purpose of the developed APIs ( their expectation from the developed APIs.)

2.Understand the explored scope and come up with Testing Scenarios to cover the complete scope:

  • Passing Valid / Invalid inputs and parameters.
  • Nested API scenarios
  • Load testing scenarios to check the performance.
  • Scenarios to check the Authentication and Authorization.
  • Exploring the Scenarios in which the API will fail.
  • Checking if the API will justify the purpose for which it was developed after being integrated with the Application.

3.Documentation: Document the complete testing process so that it can be referred by the developers and can be used in future for testing. Documentation should be done from the start i.e from identifying the testing scope to compiling the test execution report.

3 Likes