I’d love to hear I’m reading this wrong, but it sounds to me like they’re going to limit the number of local runs available. I’m guessing this will be via local client updates.
To better align with the value Postman’s test automation offers for professional testers and quality engineers, we will make the following changes effective February 15, 2023, for free customers, and effective March 15, 2023, for new paid plan customers. Existing paid plan customers will see these limits effective upon plan renewal after March 15, 2023 (except Enterprise users, who have unlimited runs per month):
Local Collection Runner: Runs will be available with Free and Basic plans at 25 runs per month, Professional plan at 250 runs per month, and Enterprise plan at unlimited runs per month
Cloud-based Collection Runner: Runs will be billed on a consumption basis as part of the monitoring calls entitlement
I’m sad to see they make use of the vendor-lock-in.
I hope more automation people will make it into coding so they can use libraries like RESTAssured.
Also I fear the are not given the time and will pushed to similar tool when the company do not want to pay the fee.
I can not stand such UI tool for API automation as I see myself very limited.
I was wondering about that too, the only times I used collection runner was when I needed to create something multiple times, so I’d just run the collection multiple times. Other than that I use it for exploratory testing 99% of the time, I do hope that won’t be affected as I’m very much used to Postman after using it for many years.
“Limits apply only to local runs via the Collection Runner, not via Newman or Postman CLI.”
I guess it depends how heavily you use. the collection runner, as opposed to other options. Personally, when I occasionally use Postman instead of say Insomnia, or HTTPie, I use it for exploration and then if I move onto automation I’ll use a coded solution. So I’m not worried at all about limits to collection runners.
This is of grave concern for me. A recent Webinar with them on the source control aspects is for sure geared to the high tier of subscription.
We do a lot of API validation testing with Postman - and not necessarily running a whole collection - but folders within it. If the limitations apply there as opposed to full collection monitoring, then as a small business we would have to look elsewhere.
Will keep monitoring this chat for anyone who is able to settle my fears!
Another alternative for testing APIs exploratory, calling them however you currently need them (‘chaotic’ emergent) instead of a fix order for a specific test case:
VS Code with the plugin “REST Client”
You have to create .http files in which you note the URL, authentication and arguments.
Then the plugin displays in the file a little ‘Send Request’ button above your request.
Once executed the response is show in another tab.
Also you can use variables for e.g. taking value form a response as input to another request.
What bothers me the most about this is that it is local, client side functionality that runs on user owned hardware. Yes, it’s specifically the Collection Runner. For now. If they’re willing to heavily restrict this functionality there’s no logical barrier, no obviously protected cut off point from locking down other parts of the application in local-only use.
PostmanCLI is unaffected. I would be surprised if it remains that way.
I contacted Postman Support and they were able to provide information on our monthly usage and if this change would impact us. It has been determined that this will be an issue and cause us to change our subscription to the level. I encourage others to reach out to Postman Support.
We stopped using Postman cloud because it was limited to three users. Those collections are now out of date, and TBH I never felt comfortable storing credentials there anyway. It is my understanding that they are planning to limit local runs using the test runner, which we also don’t use. Postman is a quick way to share and understand API definitions (and maybe poke it a little bit) but we do most of our API testing using other tools.
To clarify. The limitation has been applied only to running the entire collection via the ‘Collection Runner’ within the app.
You can still run individual calls locally, and your entire collections via Newman, Postman Cli, or scheduling (monitors/webhooks) without being impacted by this imposed limit. (You are only limited when running the entire collection via ‘collection runner’).
The Postman staff have (from what I have seen) been forthcoming on the community forum etc. by providing information about the new limits but also providing details on how else you can run your collections to workaround the limitations.
@woolsm if you are referring to ‘scratch-pad’ then yes I believe this is also affected;
Your Postman plan gives you a limited number of collection runs you can use each month. This limit applies to collections that you run in a workspace or the Scratch Pad using the Run manually option. This limit doesn’t apply to scheduled collection runs in the Postman cloud. A collection run with multiple iterations counts as a single run.
(Just my opinion) … It’s worth considering that the collection runner, while it appears to ‘only run locally’ actually syncs up behind the scenes so that the run results are accessible between client and browser but also cross team members etc. These underlying services and run histories etc. cost money to build and maintain and with Postman’s ever-growing user base, it was inevitable that they would change their pricing at some point.
I know a lot of people probably won’t agree, but that’s just my viewpoint.
It isn’t just client-side functionality that runs on user-owned hardware. There is synchronisation and data storage that happens for the collection runs that allow you to move between browser and client / other team members being able to see the test results etc.