Iād love to hear Iām reading this wrong, but it sounds to me like theyāre going to limit the number of local runs available. Iām guessing this will be via local client updates.
To better align with the value Postmanās test automation offers for professional testers and quality engineers, we will make the following changes effective February 15, 2023, for free customers, and effective March 15, 2023, for new paid plan customers. Existing paid plan customers will see these limits effective upon plan renewal after March 15, 2023 (except Enterprise users, who have unlimited runs per month):
Local Collection Runner: Runs will be available with Free and Basic plans at 25 runs per month, Professional plan at 250 runs per month, and Enterprise plan at unlimited runs per month
Cloud-based Collection Runner: Runs will be billed on a consumption basis as part of the monitoring calls entitlement
Iām sad to see they make use of the vendor-lock-in.
I hope more automation people will make it into coding so they can use libraries like RESTAssured.
Also I fear the are not given the time and will pushed to similar tool when the company do not want to pay the fee.
I can not stand such UI tool for API automation as I see myself very limited.
I am hope I am also reading it wrong.
It would be bad if this happends.
Postman has always made it easy for new testers to learn about APIs testing.
We will just have to wait and seeā¦
I was wondering about that too, the only times I used collection runner was when I needed to create something multiple times, so Iād just run the collection multiple times. Other than that I use it for exploratory testing 99% of the time, I do hope that wonāt be affected as Iām very much used to Postman after using it for many years.
āLimits apply only to local runs via the Collection Runner, not via Newman or Postman CLI.ā
I guess it depends how heavily you use. the collection runner, as opposed to other options. Personally, when I occasionally use Postman instead of say Insomnia, or HTTPie, I use it for exploration and then if I move onto automation Iāll use a coded solution. So Iām not worried at all about limits to collection runners.
This is of grave concern for me. A recent Webinar with them on the source control aspects is for sure geared to the high tier of subscription.
We do a lot of API validation testing with Postman - and not necessarily running a whole collection - but folders within it. If the limitations apply there as opposed to full collection monitoring, then as a small business we would have to look elsewhere.
Will keep monitoring this chat for anyone who is able to settle my fears!
Another alternative for testing APIs exploratory, calling them however you currently need them (āchaoticā emergent) instead of a fix order for a specific test case:
VS Code with the plugin āREST Clientā
You have to create .http files in which you note the URL, authentication and arguments.
Then the plugin displays in the file a little āSend Requestā button above your request.
Once executed the response is show in another tab.
Also you can use variables for e.g. taking value form a response as input to another request.
What bothers me the most about this is that it is local, client side functionality that runs on user owned hardware. Yes, itās specifically the Collection Runner. For now. If theyāre willing to heavily restrict this functionality thereās no logical barrier, no obviously protected cut off point from locking down other parts of the application in local-only use.
PostmanCLI is unaffected. I would be surprised if it remains that way.
Iām sure they would appreciate specific and constructive feedback. They are not a totally unapproachable company. And folks who work there are outstanding members of the community in their own right.
I totally understand your concerns, itās worth making sure they hear them.
We use Postman at work. Weāve got a few collections we export and then use Newman (their CLI tool) to run as part of our CI job. Thanks to this we rarely use the āLocalā Collection runner.
Seems like now people will just find other ways to run their collections.
Itās also a good example of a freemium product changing what is free and what is premium. This is why we canāt have nice things.
I contacted Postman Support and they were able to provide information on our monthly usage and if this change would impact us. It has been determined that this will be an issue and cause us to change our subscription to the level. I encourage others to reach out to Postman Support.
We stopped using Postman cloud because it was limited to three users. Those collections are now out of date, and TBH I never felt comfortable storing credentials there anyway. It is my understanding that they are planning to limit local runs using the test runner, which we also donāt use. Postman is a quick way to share and understand API definitions (and maybe poke it a little bit) but we do most of our API testing using other tools.
To clarify. The limitation has been applied only to running the entire collection via the āCollection Runnerā within the app.
You can still run individual calls locally, and your entire collections via Newman, Postman Cli, or scheduling (monitors/webhooks) without being impacted by this imposed limit. (You are only limited when running the entire collection via ācollection runnerā).
The Postman staff have (from what I have seen) been forthcoming on the community forum etc. by providing information about the new limits but also providing details on how else you can run your collections to workaround the limitations.
@woolsm if you are referring to āscratch-padā then yes I believe this is also affected;
Your Postman plan gives you a limited number of collection runs you can use each month. This limit applies to collections that you run in a workspace or the Scratch Pad using the Run manually option. This limit doesnāt apply to scheduled collection runs in the Postman cloud. A collection run with multiple iterations counts as a single run.
(Just my opinion) ā¦ Itās worth considering that the collection runner, while it appears to āonly run locallyā actually syncs up behind the scenes so that the run results are accessible between client and browser but also cross team members etc. These underlying services and run histories etc. cost money to build and maintain and with Postmanās ever-growing user base, it was inevitable that they would change their pricing at some point.
I know a lot of people probably wonāt agree, but thatās just my viewpoint.
It isnāt just client-side functionality that runs on user-owned hardware. There is synchronisation and data storage that happens for the collection runs that allow you to move between browser and client / other team members being able to see the test results etc.
For exploratory API testing Iāve been using ThunderClient in VSCode for a long time now. For formal API Automation I tend to go with Requests/RestAssured and others.
