What are your strategies for working with and maintaining large Postman collections?

Discussions
, ,
1

I just logged off of work for the day, and I am having #QAThoughts. In particular, #PostmanThoughts, #SDETThoughts, and #AutomationThoughts.

Here’s the scenario: I had, what I would consider, a reasonably routine update across the handful of API test suites that exported from Postman, running via Newman and Github Actions. I make the updates in my local Postman app, export the JSON, update via Github, verify, rinse and repeat until sqeaky clean. The challenge was an assertion update for all of the requests. Not just a few requests. All of them.

But Jen, you think to yourself - I’m sure that was like, maybe 20 copy/pastes, right? No. It was a lot more. And it was compounded because I was making improvements to error handling, so as I copy/pasted, I thought more on it, and had better ideas, and then went back and copy/pasted again. I did this a few times instead of leaving the “good enough” code alone, like I should have.

Did Postman have a Find and Replace feature? Yes.
Did it work for multiple lines of test assertions? No.
Was it useless to me? Yes.
Did I manually open up every request in every collection, feeling my relationship with this program that had brought so much to my career direction dry up with every +C and +V? Yes.
Is our relationship status changed? It’s complicated.
Did I only just now consider finding/replacing in the exported JSON collection files and then exporting it back in? Sure, but that’s gross, and you and I both know that. That’s dumb and no one should have to do that.

I’m not sure how I can keep making updates across multiple requests going forward without manually going into each one without a workspace-wide find and replace, for example. This has made me realize that there are likely other issues that will come up as I make more test suites and need to maintain them within this ecosystem.

What strategies have helped you with creating and maintaining your Postman API test suites, in particular if they are larger sequences of requests (ie. end to end tests)?

Some of the challenges I have:

  • End to end test scenarios that require 15+ request prior to set up each test scenario, therefore a collection can be on the larger side
  • Non-prod environments that have flakey performance, so I’ve abandoned speed testing per request because it’s so inconsistent and there’s no prioritization to improve it regardless
  • Difficulty finding time for new automation creation or time to maintain existing test suites in addition to current manual testing workload - in other words, feedback that helps with maintenance efficiency is also appreciated here also!
5 Likes
2

Why didn’t it work for multiple lines? Works fine for me :person_shrugging: even for improving code.

Let’s say you use request Y and test Y on multiple sections in your suite.
You can just add 1 request to a folder, in your previous request you can do

let name = request.name;
pm.globals.set("requestName", name );

postman.setNextRequest(requestY)

This will store your ‘unique’ request name in a global variable.

On your requestY you also have a test script:

var nextRequestName = pm.globals.get("requestName");
postman.setNextRequest(nextRequestName )

Making you only have 1 request and not having to copy paste into multiple of the same requests.

Shouldn’t be a problem when you use something like the above setup, where you use setNextRequest. This will reduce the amount of requests you have and making your collection smaller.

I agree on the choice of giving up the speed testing. If you have issues with tests failing because of performance issues, you can always introduce a pull mechanism into your code to retry the request 3x or 5x with a bit of delay in between to reduce flakyness and making your automation more viable and not having to look at false positive failing tests.

The above code also REALLY helped me with maintenance, since you only have 1 requestY and if the body changes you only need to change it once.

1 Like
3

Hi @kristof, thank you for taking the time to reply. I appreciate it.

For the first point, I’m wondering if the issue was find/replace with comments vs. without (this works per Tests tab, but not with the Find and Replace across Collections, for example). I would much prefer to have this work, so I had to mention this here in case there were caveats I wasn’t aware of.

For your suggestion with the global variable and the request name, I find this interesting. I’m unsure if this would apply to the particular scenarios I have now as I have one test scenario per collection given how different the requests are per scenario. It sounds like this would work if I had multiple scenarios per collection with more overlapping or similar requests, though, and somehow manually set the order of the requests with either an iteration data file or other setup. This is an interesting take that I hadn’t considered, but you’re correct that it would reduce the maintenance per request. Unfortunately, I have scenarios where requests would potentially have different assertions depending on the scenarios, which I think would complicate the Tests tab a bit - it’s not impossible, it’s just a considerably different approach than what I’ve done so far.

You’ve given me a lot of food for thought, though. Thanks for the tip.

4

alright I got something else that I did for this but it will require some rework. I made ‘hierarchical tests’ .. at least that’s what I call it.

I write 0 assertions on request level and I write assertions on Collection Level. YES COLLECTION.
What I did was a very big If if if if statement.

Based on request method and url you know what request you are firing off.
So in the pre-request script (collection) you clear all global variables or set a value ‘untested’.

Then in the test script (collection) you write something like this:

If POST & URL == /customer/details/XYZ {
Get global value A B C
If A = untested, skip (etc…)
Assert: A B C}

Else IF GET & URL == /customer/details/XYZ {
Get global value O P Q
If O = untested (skip…etc)
Assert: O P Q }

On the Pre-request script you fill in your expected value doing a set global value A B C & set global value O P Q.

If that makes sense?

I find it super easy to maintain since you only write your assertions once for your whole collection, per request. If you don’t want to assert a specific field, you can just check the ‘untested’ value.