The upcoming MoT API automation challenge caught my eye recently. I was intrigued by the restful-booker app and API that will be used for it.
The app is open-source and contains a good number of tests. Which is awesome - Iāve come across a good number of simple apps to write automated tests against, but a much smaller number of example test suites.
My question for you all is: where else are there open-source/publicly available test suites? Iāve learned a lot of my automation skills from just poking around test suites Iāve inherited and think it would be cool to see more.
Iād be especially interested in:
A repo that automates the same app but with a handful of different frameworks (Cypress and Selenium for instance).
A repo with some bad practices! Would love a repo with some obvious and less-obvious room for improvement (page objects have locators for more than one page; organization is bad; assertions are redundant, things like that).
Are there ālearning test suitesā that you know of? If thereās not, want to build one with me?
Are we talking about web-service testing and browser based app testing only @john_sand ?
I suspect that in between finding low quality test examples via google, while there are loads of really good suites on github and all you need to do is look at your fave open-source apps or tools and see how they test. But the ācomparisonā this is interesting. Myself, not used Cypress, mainly out of language binding choices, do share what you can.
To be sure, there really are no good and bad practices, only useful and less-useful tactics within their context, and Iām not sure we point to the cool ideas versus the bad often enough by sharing actual fully worked source code often enough to tell that compelling story. Since Iām more of an Embedded and Desktops test person, itās not often I get to see external test code aside from the ones I spot on github which are mostly Desktop or commandline app suites.
I had in mind web app and API testing, but would be curious to see any and everything. Command line app suites? If youāve got links, Iād love to have em!
Your question does clarify for me though: I think more than just example test suites, I want learning test suites. Like, ones that provide a lot of contextual information on: why we included this test but not other possible tests; why we automated this step with an API call rather than through the UI; how we dealt with this difficult element.
And yes, I especially want to see the less-useful tactics. Iām imagining a project you could fork that has some brittle tests, with some idea of how you might improve it. Honestly, I think Iām describing that Iād like to work on refactoring a bunch of test suites with a mentor and weāve got the lee-way to take our time with it. Maybe Iāll get that in my dreams, but irl Iād settle for some learning-focused test suites
Pretty sure that if you google for it you can find all of the WINE test code. https://www.winehq.org/ WINE is a wrapper created years ago to allow Windows apps to run on unix, it did (does, still today) this by fooling a windows app into thinking it was on Windows by redirecting every single windows api into a posix call and mapping these into real filesystems and networks. Itās a test suite that will make sense only to someone who has actually written a Windows program that did not sit on top of overly fancy frameworks, because it covers most of win32. One of my more fun tester jobs was to use that WINE test suite (all written in C/C++ and using CXXtest) for clues and re-purposing all the tests to test our own windows driver hooking product. The project got scrapped, but I learned a lot from that suite, probably more about api testing and about the product implementation than about test suite design, but I did learn about using test fixtures at large scale too.
Iāll never think of a test fixture as ājust a little piece to set things upā ever again.
Not long after that api hooking project I also fully drank the cool aid on writing tests that check the tests. As Pratchett might have put it, āitās turtles all the way downā.
I love that. I was thinking along the ātest the testsā line as well as I was thinking through what Iād want in a ālearning test suite.ā If I were to set one up where people could do self-learning with a code base, thereād be a testsā test suite along with it
Unless you have used C and C++, the CXXTest framework will not make much sense to you, but itās one of the most lightweight test engines in the world because it only uses header files and MACROS. But here is the steps needed to run itās self-test http://cxxtest.com/guide.html#appendix_C basically you just go
python test_cxxtest.py
Using Python, to test your C++, so itās not uncommon to test your tests
