Does it build? Does it run?

When I’m exploring existing software that is new to me, one of the things I like to try and do early on is build it, and run it locally.

I often find some instructions in the README.md for the service, but they tend to be out of date, or incomplete. I typically have to either figure it out myself or ask for help from a developer, who has done it themselves recently.

When I get it running, I try and improve documentation and help others in the team, or me in the future, get running quicker.

The benefits I get from being able to explore the code, running locally, are many and varied. It allows me to Control and Observe the software much more easily. Two key aspects of Testability, part of CODS model.

You can read more about CODS here:

I’d love to know if you run and test your teams software locally, what pain and what benefits that brings.

3 Likes

I do build and test our software mostly locally and I’m very happy about it.

Benefits:

  • switching versions, or resetting the installation, is way faster than on our test servers (longer build time and installation due to no parallelization)
  • I can better control and test nearly any aspect of our software (including build and deployment/migration)
    • About build etc.: I even experience (parts of) the deployment process. Which I otherwise I would not have (at least in not depth and variate).
  • My computer is way more power than our test servers, therefore I can process more/faster data.

Sure, this demanding to learn.
But what is not in our industry.

Pains:

  • With great power comes great responsibility. I can do more things, also I can screw up more things.
    • e.g. I sometimes execute accidentally a wrong command and have then to spent a while to rebuild the system to the very point again.
  • its harder (so far I never tried it) for developers to debug my local system compared to our test servers. The developers always have so far to reproduce the situation on their system (or I do it on a test server).

Disadvantages, for which I have solutions:

  • Having server and client on the same machine is not a realistic system. For any connection related issues I use our test servers.
  • Installing on a clean system (having not a chaotic registry, already different installed frameworks etc.) needs a dedicated server. At the very least as VM on my computer.
3 Likes

Excellent and relatable list of pain and joy!

To add, most of the services I’m testing have the option to run via Docker, so this helps immensely when transferring configuration, if not state, to others.

As for the pain of debugging a local system, I can say pairing with a developer with a screen share, either on your system or theirs, and the use of remote IDE features like IntelliJ Remote Access is very useful. Even test servers have some pain here, if you don’t have debugging tools available, developer still tends to try and reproduce on their local system to dive deep.

2 Likes

“Run”. Is a funny word these days, where the cloud really is just someone else’s own computer. How useful is it to build and run locally, when today, many applications are composed of multiple 3rd party integrations which once you stub them out, mean you are not really observing the system as a whole at all. I think a bit of the Elon Musk massochist has to be in you, because you will be using Rails, Scala, Python, Bash and of course loads of Java(Script), so I’m assuming also multiple databases and VM’s, authentication servers, all of which Melon Tusk famously asserted were surplus to business requirements. So I’d be a bit careful as a tester trying to set that up.

My experience is that it is very helpful to be able to CODs your product, but impractical if your product is non-trivial to deploy. But learning how it is all architected is far more useful, and by starting the discussion with your developers, you can often earn a load of respect, and will have the devs talk to you about their changes in language you share more often.

2 Likes

This itself is a good indicator of a risk to quality. If the product is non-trivial to setup and test, it’s also non-trivial to develop. Meaning changes that are not fully understood are much more likely, as are unintended consequences.

Quality doesn’t come from testing, testing helps us gain information we can use to improve quality.

But yes, I take your point. It isn’t always plausible to run everything you’re testing locally, nor is it desirable to only run it locally and neglect real deployed environments.

2 Likes

100% there , good point Ben. When building and deploying a complex system entirely, if you have to make compromises and it takes too much headspace to do that, you start being better off taking a clear slice through the tech stack. But designing the slice to give you confidence where it matters most to the team.

As a coder I’m biased towards having testers know how to build runnable binaries. And to be honest even if you don’t get runnable binaries (signing keys for example are always an hurdle with local builds.) Merely getting everything set up and trying is super constructive for the team, and normative exercise that lets you stretch.

1 Like