Do you know of any example test suites for learning?

The upcoming MoT API automation challenge caught my eye recently. I was intrigued by the restful-booker app and API that will be used for it.
The app is open-source and contains a good number of tests. Which is awesome - I’ve come across a good number of simple apps to write automated tests against, but a much smaller number of example test suites.
My question for you all is: where else are there open-source/publicly available test suites? I’ve learned a lot of my automation skills from just poking around test suites I’ve inherited and think it would be cool to see more.
I’d be especially interested in:

  • A repo that automates the same app but with a handful of different frameworks (Cypress and Selenium for instance).
  • A repo with some bad practices! Would love a repo with some obvious and less-obvious room for improvement (page objects have locators for more than one page; organization is bad; assertions are redundant, things like that).

Are there ‘learning test suites’ that you know of? If there’s not, want to build one with me? :wink:

3 Likes

There are several Postman test suites around Restful Booker:
https://www.postman.com/search?q=restful%20booker&scope=public&type=all

4 Likes

Are we talking about web-service testing and browser based app testing only @john_sand ?

I suspect that in between finding low quality test examples via google, while there are loads of really good suites on github and all you need to do is look at your fave open-source apps or tools and see how they test. But the “comparison” this is interesting. Myself, not used Cypress, mainly out of language binding choices, do share what you can.
To be sure, there really are no good and bad practices, only useful and less-useful tactics within their context, and I’m not sure we point to the cool ideas versus the bad often enough by sharing actual fully worked source code often enough to tell that compelling story. Since I’m more of an Embedded and Desktops test person, it’s not often I get to see external test code aside from the ones I spot on github which are mostly Desktop or commandline app suites. :cold_face:

3 Likes

I had in mind web app and API testing, but would be curious to see any and everything. Command line app suites? If you’ve got links, I’d love to have em!
Your question does clarify for me though: I think more than just example test suites, I want learning test suites. Like, ones that provide a lot of contextual information on: why we included this test but not other possible tests; why we automated this step with an API call rather than through the UI; how we dealt with this difficult element.
And yes, I especially want to see the less-useful tactics. I’m imagining a project you could fork that has some brittle tests, with some idea of how you might improve it. Honestly, I think I’m describing that I’d like to work on refactoring a bunch of test suites with a mentor and we’ve got the lee-way to take our time with it. Maybe I’ll get that in my dreams, but irl I’d settle for some learning-focused test suites :slight_smile:

2 Likes

Would a test automation portfolio meet your need?

I’ve seen a bunch of articles lately on that so surely some people have made theirs public to view (just gotta figure out how to find them)

2 Likes

Pretty sure that if you google for it you can find all of the WINE test code. https://www.winehq.org/ WINE is a wrapper created years ago to allow Windows apps to run on unix, it did (does, still today) this by fooling a windows app into thinking it was on Windows by redirecting every single windows api into a posix call and mapping these into real filesystems and networks. It’s a test suite that will make sense only to someone who has actually written a Windows program that did not sit on top of overly fancy frameworks, because it covers most of win32. One of my more fun tester jobs was to use that WINE test suite (all written in C/C++ and using CXXtest) for clues and re-purposing all the tests to test our own windows driver hooking product. The project got scrapped, but I learned a lot from that suite, probably more about api testing and about the product implementation than about test suite design, but I did learn about using test fixtures at large scale too.

  • I’ll never think of a test fixture as “just a little piece to set things up” ever again.
  • Not long after that api hooking project I also fully drank the cool aid on writing tests that check the tests. As Pratchett might have put it, “it’s turtles all the way down”.
1 Like

That would be fantastic!

I love that. I was thinking along the ‘test the tests’ line as well as I was thinking through what I’d want in a ‘learning test suite.’ If I were to set one up where people could do self-learning with a code base, there’d be a tests’ test suite along with it

1 Like

Unless you have used C and C++, the CXXTest framework will not make much sense to you, but it’s one of the most lightweight test engines in the world because it only uses header files and MACROS. But here is the steps needed to run it’s self-test CxxTest User Guide basically you just go

python test_cxxtest.py

Using Python, to test your C++, so it’s not uncommon to test your tests :slight_smile:

1 Like

For Api and UI testing, the Cypress Realworld App is a useful project that contains a lot of variety of testing types. cypress-realworld-app/cypress/tests at develop · cypress-io/cypress-realworld-app · GitHub

2 Likes

Awesome, I have not stumbled upon this til now. Thank you for sharing!

Just found out that Mark has an example API suite for restful booker in several different languages! GitHub - mwinteringham/api-framework: A group of templated API test frameworks based upon this series: http://www.mwtestconsultancy.co.uk/category/api-web-service-testing/
(highly recommend his book Testing Web APIs that the suite belongs to)