How do you decide between API vs UI testing at different levels in testing pyramid?

šŸ™‹ Questions
,
1

Hello,

How do you decide when to test using API and when to test using UI?
When following the testing pyramid both the API and the UI can be tested at different levels. However, when reaching the top of the pyramid and going with online tests we shouldnā€™t be testing everything using UI.

So what standards or what use cases do you test using the API and what do u test using the UI?

Thanks

5 Likes
2

Hi @mzbib and welcome to Mot!

Great question!

So following the test pyramid, you want to test as close to the code as possible. So if you have Back-end validations and you have to chose between UI or API then you can validate those on the API. Most of the functional tests will be checked on the API ā€¦ UNLESSā€¦ you also have front end validations, so then youā€™ll want to do both.

Depending on what technology and how your application is build, youā€™ll want to decide if you want to test on the UI or API. Most of the time if itā€™s possible to test via API, you should pick API since itā€™s so much faster.

I personally tend to test most of it through API and only test ā€˜visualā€™ things automated via UI. For example: If an image is loaded or if a red error bar is showing.

2 Likes
3

If you want to check UI code, you render the UI and mock the use case code.
If you want to check the API code, you build API objects and make calls to them.

4
  1. Similar to what @kristof wrote: Test where the feature you want to check is implemented. If in a class use unit tests, if in a service use the API.
    BUT!
  2. If the functionality under consideration seems to be a good candidate for a check through the UI, do consider if you can push it down. If that means that some risk is not covered, then maybe that is okay or maybe that risk can be mitigated in another manner. If not, then go through the UI.

Example of 2: You want to check for a web shop the flow of logging in, creating an order, paying for it and getting a confirmation email. This is the kind of thing that users do, and they do it through the UI. But if the backend has services for this stuff then you can check everything but the UI itself through API calls, for example:

  1. Fake logging in by setting whatever you need in the database.
  2. Call the services to create an order and add products.
  3. Do payment.
  4. Check the email status and preferably also content through an API or in the database.

The big question here is: What am I trying to check? Is it that a user can actually use the web shop? Then you probably have to go through the UI. But if I want to check that the backend services play nice together, the above scenario can be perfect. Just make sure there is some exploratory testing also that will check that the UI also works.

3 Likes
5

And remember that the test automation pyramid is not law but heuristic! So applying it still involves the little grey cells.

6

Going to simplify it a little . An api is basically a developers UI. So automate once through the UI and varying combinations of the UI can be done through the api.

Depending on what you are testing on the UI, some of that can even be pushed to the front end unit test., ie. Validating the drop down menu has all the options.

7

Going to simplify it a little . An api is basically a developers UI. So automate once through the UI and varying combinations of the UI can be done through the api.

Do you mean that the API is the interface (rather than UI) for a developer? I would think that function calls are the interface for a developer, as APIs are meant for software. And then there are files, databases, etc. that both developers and testers can use (plus the UI).

But suppose that APIs are indeed a developerā€™s interface. How does it then follow that you should automate once through the UI? Why not 7 times or 0 times or all times? What does APIs being a developerā€™s interface have to do with automation strategy? Something seems to be missing. Are there some assumptions hidden in your statement perhaps? Please elaborate.

1 Like
8

Yes an API is an interface for a DEV. I like to coin as a ā€œdevelopers UIā€

An API is a way for a front development to communicate with back end services.

A simple example would be to pull data and display via API.

Create a UI automation using that api to pull a specific data.
You can then create edge and negative scenarios via api

This is a way for QA to push automation to the lower levels of the Automation Pyramid.

9

One point is if you want to test BY GUI or THE GUI.

  • BY: You test the whole system (especial including server code / business logic) and use the GUI as entry point. This is comparable slow, therefore APIs might be better here, but sometimes you have to.
  • THE : testing client specific things. Works the client, no matter if the server is correct (surely a bug in the server can cause problems for the client). This can include testing the integration between server and client.
    • We have dedicated GUI automation which goes through our client and just checks that basic elements are available. No check business logic, just that all views are available and windows can be opened. It happens from time to time that either client oder server changes have more impact than expected and breaks basic stuff in the client
      • Precondition at our system is that our API automation already run. Having already processed data in the system makes it way easier to automate checks at our client. Otherwise I would had to automate many business logic related actions.

You may have already guessed it: We use API automation to check our server and then use GUI automation to check the client.

And keep in mind: When automating checks for the API you typically not only check the API itself (arguments, response) but also the server behind that implicitly. API automation is/can be heavily the check of server code / business logic. Arguments and response are only the interface to that.
It is most times BY the API (while also including it). More seldom just THE API.

1 Like