At TestBash Autumn we had a fantastic Panel Discussion with experts in Driving a Culture of Quality, they are: @parveenkhan , @fullsnacktester and @c32hedge .
During the lively discussion, a lot of questions were asked by the audience and a lot didn’t’ quite make it live.
One of those questions came from @karenlewis :
Do any of you use Docker (or anything similar) for feature testing? What would you say are the pros/cons of one of these apps?
How about you? What would you say?
I’ve used Docker in a lot of projects, and the biggest benefit in my experience?
Just how much it works/ doesn't on my machine is removed. With Docker I can run for example a Node application, without having to install and configure it myself.
Being able to to have people working in the same setup removes a lot of friction and chances for subtle configuration issues. If you are using that same docker image to build your application in different environments, you can be extra sure they’ll be the same. And any effort to change it benefits more than one person.
In terms of drawbacks, it is yet another thing to learn and manage.
But it’s far better to put the effort into defining something once, that you can use for local development, testing, and your production environments.
100% @flynnbops answer. It removes hurdles and creates consistency between different phases of software development.
By the way, it’s not about Docker the tool, it’s about containers, the technology.
When it comes to testing with containers it adds a little complexity to the process. In addition to worry about branches (git) when testing locally, you now have to make sure all the containers are running their respective services and each is working.
For example you might be testing a new feature in a container on your local machine but you can’t see the changes. You have the tight branch checked out. What could be the problem? Well did you restart the right service? Are there other dependencies that are working / updated?
So it adds complexity there. It also can add complexity in terms of debugging things. Like if you were accustomed to seeing logs in your terminal, those might now be buried inside the terminal of an individual container. You’ll have to figure out how to inspect those logs to find your problem.
As usual a great point from Chris! That gets to the core point
I haven’t had a chance to try containers yet, but I think we’re moving in that direction. Helps that nowadays I think they can coexist with VM software, which we currently use a lot.
Containers make it easier to model production environments for realistic integration testing. You can deploy containers identical to production with the same base OS, dependencies, etc.
That’s the real advantage of containers, and you can easily “swap” them out.
If you you have an architecture that looks like
[App A] ----> [ Service B] -----> [Service C]
you can swap out App A or any service with a newer version and still maintain most of your tests for the rest of the chain
[App A] ----> [ New Service B] -----> [Service C]
(As long as it’s not a major change which would require to also swap out other containers with new code in either A or C)
This is a good topic, but could be expanded on to consider how you utilize docker / containers in your testing. I wrote a post some years ago on a different aspect of using containers for testing. I think it is more normal/popular now than when I first posed the question as you can find more articles online around this topic if you search around.
On the questions posed in this thread,
pros of dockerizing your tests: self contained, and hopefully more repeatable deployment of test and test runs; ease of deployment of tests if you have docker infra in place
cons: more work to design the setup if not familiar with docker
and if I wasn’t considering docker, my next thought would be how could I manage the test codebase and infra via something like vagrant or terraform.
Your article seems to be about containerizing your automated tests, right? I agree that’s a worthwhile thing to do.
However I separate that from other testing activities which you might do against containerized applications.
Your article seems to be about containerizing your automated tests, right? […] However I separate that from other testing activities which you might do against containerized applications.
Correct, about my article. However, in the context of this thread, how folks would be testing an application or service (or overall the system under test (SUT)) that is containerized, you could further containerize the testing pipeline by containerizing the test codebase / test infrastructure, such that the only thing not containerized in the test pipeline is the executor, which could be CI, jenkins, a script, etc. Although for case like jenkins, that could also just run from a container that you then use to run job to trigger the tests (running in another container) to test the application running in yet another container container.
As for why consider doing that? The pros & cons are already mentioned in my previous comment.