Postman or Playwright for API tests?

[richardhinton]

I get a lot of push from developers to do API testing in Postman. Which usually leads to Newman being used for inclusion in pipeline.

Recently I’ve moved to doing API testing in Playwright as it offers more control and better reporting.

I’d love to hear the communities thoughts on this and which way you and your team have been going? Pros and cons, etc.

Thanks

[kristof (Kristof Van Kriekingen)]

@richardhinton

Playwright as it offers more control and better reporting.

Can you explain this, maybe there are some things in Postman that you don’t know yet?

@richardhinton

I’d love to hear the communities thoughts on this and which way you and your team have been going? Pros and cons, etc.

Not gonna lie, I’m a huge Postman fan.

• Ease of use
• Low learning curve or people already know it
• Non-technical people can easily pick up Postman and use it
• Easy exploratory testing with Postman compared to Playwright.
• I use Postman if I’m to lazy to open Burp, for some data driven fuzzing.
• Our Developers also use it

I use this for reporting with Newman: GitHub - DannyDainton/newman-reporter-htmlextra: A HTML reporter for Postman's Command Line Runner, Newman. Includes Non Aggregated Runs broken down by Iterations, Skipped Tests, Console Logs and the handlebars helpers module for better custom templates.

I’m not against Playwright, it’s just not as appealing to me compared to Postman.

[richardhinton]

Thanks for the feedback, it’s a big help

[c32hedge (Caleb Crandall)]

I haven’t looked at Postman for a while so it’s possible this has changed since I did (or maybe it’s possible and I just couldn’t figure it out), but the lack of ability to store the scripts in a source control system like git was a real dealbreaker for me, along with basically having to use the tool’s own IDE.

I haven’t used playwright, but I do use Cypress for API testing and really like it—especially because I can mix API calls with UI-based actions and assertions to get better coverage of full workflows.

[ckenst (Chris Kenst)]

The last company I worked at, we built a lot of API tests in Postman, and then used Newman to run them. In the end, I wouldn’t do this again.

Newman and Postman are different enough (as of 6 months ago) that failures and passes will appear differently between the two. It’s not a lot of differences but enough that you might get different passes and failures between the two.

Furthermore, in order to get Postman tests into Newman you have to export 2 files and commit them to code, which means a potential disconnect between creating the endpoints (or updating them), creating tests and then running tests. If I were doing it again, I’d just write the API tests next to the API code and run it all in the same CI.

I haven’t tried Playwright for API tests but with code you can use a true Dev Environment and have more control over all that happens. You just won’t get the quick to start interface Postman offers.

[kristof (Kristof Van Kriekingen)]

You’ve gotta use what you feel comfy with. We can only address our pro’s & cons and tell about our experience, but you gotta match it onto your project and see what fits best.

Going to defend Postman here abit! (Not saying you ‘need’ to use it)

@ckenst

Newman and Postman are different enough (as of 6 months ago) that failures and passes will appear differently between the two. It’s not a lot of differences but enough that you might get different passes and failures between the two.

That’s only because of not-properly using of SetNextRequest, an example people don’t often know is it might be because requests names must be Unique. (Which makes Newman skip some requests sometimes or be stuck in loops. all by all, it’s a good find that newman doesn’t work like that because it will make you find bugs in your code ) I’ve experienced this too and found out the hard way that my way of naming requests and coding wasn’t optimal

@ckenst

Furthermore, in order to get Postman tests into Newman you have to export 2 files and commit them to code, which means a potential disconnect between creating the endpoints (or updating them), creating tests and then running tests. If I were doing it again, I’d just write the API tests next to the API code and run it all in the same CI.

@c32hedge

but the lack of ability to store the scripts in a source control system like git was a real dealbreaker for me

This was indeed the issue at the start of Postman but has changed a while ago.

Here’s some guidance: Better Practices for Git Version Control in Postman | Postman Blog

[sharmon (Sean H)]

We’re actually including API tests directly in with our code now. So, we are utilizing WireMock and Contract Testing currently to achieve this. They’ve been really helpful as devs can now own those tests and our QA can focus on more on more Exploratory or Business logic testing. We still do use postman for quick hits and such, but we’ve been working to build out our own framework for wiremocking/contracts so we can utilize them in the pipeline without any other needed plugins.

[juanalvarezarquillos (Juan)]

I won’t use Postman to develop API testing. It is a nice tool to test and play with the APIs, but when the collections grow, it is a hell to maintain.
If you want to create an automation to test some REST API, go with pytest + resquests, or Java + RestAssured, etc but start developing.
I don’t know about Playwright, sorry.

[stwoo2019 (Shuk Tak Woo)]

I’m just about to get my hands dirty.
I was just dumped an early version of an API that will be called by a React application.

I am a newbie at this as have used Postman (bog standard HTTPClient) in the past without pipeline. At the initial stages it really helps to work with the developers to iron out issues found via exploratory testing.

Now, I want it automated and in the pipeline. I have a brief understanding of Newman that will definitely overcome the limitation of running collections in Postman without a paid license.

However, I am thinking Playwright API so that I have a constant (What’s the word?) tool as will be using Playwright with C# for UI testing.

Keeping an eye on this thread as Iearning. Thank you

[utchbe (Butch Mayhew)]

I use Postman for all my API Exploratory testing, though that may get moved to Bruno here soon once version 1 is released. Our team does manage a Postman collection that we try and keep up to date and semi organized with updates and new changes happen within our APIs.

For our API automation we use Playwright. We have over 1300 API checks this includes coverage for over 300 endpoints. The devs and testers have high confidence when making large risky changes to API input/output and for a great portion of the logic that we will see the impact in our automation runs.

I Playwright and would choose the same path again. I’m 13 articles in to sharing about my experience along with a full project of API test examples that I’ve created and share about here - The Definitive Guide to API Test Automation With Playwright: Introduction I’ve got 3 more articles I’m planning to write and am open to covering more if there are additional questions or areas that I haven’t covered.

It’s helpful to note I am using Playwright Test / Typescript so I get the Test runner out of the box.

Pros

• Only 1 test framework to learn for UI/API checks
• You have the power of any other library at your fingertips using Playwright test or Playwright as a Library
• Built in test runner has excellent reporting, specifically if you are capturing traces for your test runs, which allows you to view the requests, request headers, response headers, and response body in a similar way Chrome dev tools renders network requests.

Cons:

• I haven’t found a way to send a request without a user-agent
• I haven’t found a way to easily measure http request duration (there are workarounds but nothing built into Playwright request method)

[fullsnacktester (Ben Dowen)]

I would start by exploring an API using a tool like postman. But for automation, I’d use a tool like rest assured (java/kotlin) or just+super test (JavaScript/TypeSctipt).

Why? Because code based automation means easier source control and team collaboration, review of PRs etc

I didn’t know playwright had dessicated API testing features? I might check that out. I thought it was browser based.

[stwoo2019 (Shuk Tak Woo)]

Hey there,

That is a really useful Playwright guide.

I must admit that I defaulted to C# because that is out main programming language. However, a front end is being developed in React means there is a case to use TypeScript to perform the API tests.

Great practices is excellent section. It is not too difficult to write some (bad) automated test, but more effort is required to write good test cases that are easy to read and maintainable.

I’ve been culprit to poor naming practice. I was following the naming practice on a specflow, playwright with c# project, it was so hard to read the test method names in the test explorer in Visual Studio 2022.

After 0.5 day of hacking, I’m turning into a convert. Really love playwright.

Thanks.

[priti_testrig (Priti Gaikwad)]

Hello,

At Testrig Technologies, as testing professionals, we frequently utilize Playwright for API Testing. Here’s a breakdown of the reasons why?

1. UI and API Testing: Playwright is versatile, capable of both UI and API testing.

2. APIRequestContext Class: Playwright employs the APIRequestContext class for API testing, allowing you to send a variety of HTTP(S) requests over the network.

3. APIRequest Creation: APIRequest instances are created using the APIRequest class provided by Playwright, making it easy to interact with APIs.

[adamfahey23 (Adam Fahey)]

I also prefer playwright when testing api’s. Your articles are GOLD. I Just bookmarked them and I plan I review them and practice my api testing again with playwright. Thank you for making them.

[stwoo2019 (Shuk Tak Woo)]

With the 330 endpoint and 1300 api checks, did you consider layering that BDD layer of Gherkin (in my case Specflow)? Why?

[utchbe (Butch Mayhew)]

I didn’t consider this as I didn’t want to add any additional complexity to my tests. I used Playwright Test Runner that is available in Typescript/Javascript. With Describe, test, and test.step descriptions I didn’t feel I needed a Gherkin layer.

With the default Playwright Test Runner this is an example of the report Playwright Test Report

[ivoqa (Ivo T.)]

Thank you for the demo report. When things are passing, however, it’s all nice and dandy, what matters most is when things start failing, how does report look like then and are how fast are you able to reproduce / backtrack to origin of the failure?

Would you be able to add such a case to the demo?

[ivoqa (Ivo T.)]

OK so here’s my take. Postman in my opinion is not a “full fledged” automation tool. It is a fantastic tool, however, for quick API testing, setting up test scenarios in collections and doing limited amount of automation. When it comes to source control, updating tests and Newman integration, the process is quite tedious and cumbersome. It sure can work, however you need Postman app to develop tests, you cannot really do it in the code, editing .postman_collection JSON file unless you’re a total nerd

Not to mention the huge debate on Postman forums about security and privacy. Allegedly dozens (hundreds?) of companies around the world are rethinking whether to ditch Postman completely and choose some other tool due to those concerns. Problem is, there’s not decent alternative to Postman, it’s like switching from Windows OS, for Christ sake

What I missed in most other comments is the fact that automation software should be as close as possible to production code. For example: your backend is in Java and UI is in React? Well, choose one of the two, whichever will be closer to your tests and/or who’s developers will more help in the automation effort. That way you can share snippets of code with developers, or even full features, components or libraries. Unless, of course, you’re Google and have a sea of SDET’s at your disposal in dedicated teams, doing just automation

Another fact to consider is: if you’re testing only the APIs, do you absolutely need a framework that is best utilised for UI testing? Perfect example is Cypress. Again, it’s a fantastic tool but if you only need it for API tests and UI will never come, it might be too big of an effort for your project’s needs. That is of course just my personal opinion.

I am in a similar situation, I need to choose automation framework as best as I can. We have only APIs, zero UI. Our code is TypeScript/NestJS and devs use Jest for unit tests so naturally, I chose TypeScript with Jest and supertest. I might change my mind in near future, however, depending mostly on client’s needs. For example, I miss some out-of-the-box test runners and reporters like Selenium, Cypress or Playwright have, but I’m not there yet.

What I’m trying to say is (TLDR): choose the right tool for the job, not what you feel most comfortable with. I know it’s a tough one but that’s the only proper way.

Last but not least, and forgive me for being blunt, I am a bit taken aback by the title itself - “Postman or Playwright for API tests”. It’s like giving us the answer together with the questions and that’s certainly not a good QA best practice. There’s myriad of different tools. Yes, your options might be Postman or Playwright, but that’s a specific use case

[utchbe (Butch Mayhew)]

I purposely failed a few tests from a repo I use (branch is here - GitHub - playwrightsolutions/playwright-api-test-demo at bm/failing-on-purpose)

Below is the report so you can see how failures look. I do have traces enabled, so you can see all the network requests headers and responses from the trace, giving critical information when debugging.

Playwright Test Report - Live report with failures, and a flakey test is included (I have retries enabled so you can see each retry on the failure as well).

[ivoqa (Ivo T.)]

Thank you very much! I Must say I am VERY impressed with the whole report, including UI, the links, the data… close to perfection.