Considerations for API Test Frameworks

Questions
, ,
#1

Hello all, long time no see.

I’ve started looking at standardising our API test framework (C#).
We currently use a restsharp wrapper, allowing us to swap out when required to other http clients.
We use a service/repository model with NUnit and fluent assertions on top of that.

We decided against using BDD as it didn’t seem to sit very nicely, lots of storing scenario context variables etc. seemed to make invisible spaghetti code, has anyone had a better experience with this?

Thinking aloud…

  1. I’ve had some ideas about using CURL somehow to allow our tests to be more maintainable, and able to directly base our tests off of documentation, maybe storing it in a CSV or DB…has anyone seen this done?

  2. I’ve thought about just using postman + Newman, which we currently use for documentation, but I see limitations as our test environment data is unfortunately rarely predictable, so we currently seed/destroy data in test setup/teardown. The main drive for this is to make our tests more accessible to people who don’t do automation, has anyone had much success driving that?

  3. Do I not need to make any changes?

1 Like
#2

I would treat any changes to Test Frameworks the same as other changes you might make to your core products.

E.g make sure to:

  • Allocate sufficient time and effort, remember automation is software
  • Carry out a requirements gathering stage
  • Identify stakeholders and understand what success looks like
  • If you have some options but don’t know what will work best, work them into experiments and try them out. Make sure to go in small cycles and get frequent feedback to allow for course corrections. As they say, failure is always an option.

It’s easy to get lost in the details of how you could change things, but forgot to take time to understand why you are changing them.

I hope this helps!

Explicitly in terms of using CURL script, I don’t find CURL the best for even basic testing. This is mainly because it can’t be made to throw errors for 4xx and 5xx return codes. Where as HTTpie can.

Getting people in involved who wouldn’t normally write Automation is always a worthy goal, given they have sufficient support. I wouldn’t exclude the idea of training them up to use existing C# automation suites if they are willing, if those are already working well. But it can’t be a lazy side project. to succeed it needs time and resource to train people up. Pair programming is great as part of this, but depending on what support is needed other training is possibly desirable.

Good luck, let us know how you get on!

#3

The --fail flag will do exactly this (i.e. return a non-zero exit code on 4xx and 5xx responses)

1 Like
#4

I lean towards not making any changes. You haven’t stated what problem you’d be solving by taking on this work?

You mention making “tests more accessible to people who don’t do automation”, but in my experience, that rarely works. i.e. you could create some kind of runner that reads a CSV file that contains a URL and a status code and verify that happens, but then the limitations of your runner become problematic, with people then asking if they can have multiple responses, multiple requests, changing payloads, etc. You end up spending more time trying to make all these things easy, but there’s no documentation, and then you end up having to support everyone who tries to use this. You’re much better off finding testers who are motivated to learn automation and level them up than trying to dumb down your test framework.

1 Like
#5

I’ve found this doesn’t work the way I I want it too in all cases. But thanks for the clarification might be more useful for others.

Anyway, there are much better options using HTTP clients and programming languages.

Such as Axios in JavaScript or Requests in Python. To name just two examples.

#6

Thanks for all the replies.
Regarding curl I was going to look at having a service that simply creates a http client request based on a curl string, then allowing us to assert using fluent assertions etc.
I have had similar experience as yourself Ernie, time never seems to lend itself when trying to upskill others unfortunately!

Thanks for the feedback :slight_smile:

1 Like