HI! starting out in CI/CD

Hi :wave:

My name is Sarah, I am UK based.

I’ve been a tester since March 2021 (although way back when I did do some website testing as a freelancer)

I joined Ministry of Testing because I was looking for an answer to a question about Provar.

I stayed because I just started a new role in Deployment and Automation (same company I was testing with). They are introducing CI/CD so I would like to know if you already do this what do you wish you knew when you started, and what is the most important thing you learned so far?


First: Welcome to MoT & happy that you joined the testing-life!

I’ve been setting up CI/CD also and there isn’t really something that I can think of that I would have liked to known before. I made a script that create & updates pipelines if needed (to bulk edit 100+ pipelines).
For me it was pretty straight forward but of course that’s a personal opinion I don’t know how technical you are.

So it might be nice to know you can create pipelines through scripting & yaml files and not through the UI ( if you don’t like gui) The most important thing is probably the approval gates & validations of your pipelines. So if you are copying files or something, write a small script to validate that they are actually there. So I would advice not just to make a pipeline that deploys stuff but also validate your stuff :stuck_out_tongue:

Depending on what system you are using, look into variables & variables groups and how to properly create pipeline variables & environment variables.

If you have to make multiple pipelines who use the same steps you can create a task (in azure devops) which you can just use over all of the pipelines, so you’ll only have to change it once.

Are they completely new to CI/CD as in, there have no system yet? Might be interesting to look into the differences between local agents & cloud agents.


Hi thanks for your reply, yes we plan to include validation, we are completely new to CI/CD.

1 Like

It’s always nice to draw a schema beforehand of how your pipeline will look/what steps it has/…/including all the environments.

1 Like

Hi @saraheaton,

Welcome to the community. It’s great to have you here! And congrats on your new role. :smiley:

If not on your radar, perhaps a chat with @christovskia might help. Chris works at Provar and is super kind and helpful. He’s a big part of this amazing community.

Good luck!


Thanks @simon_tomes, and hi @saraheaton, I think we may have met at a conference earlier this year.

I’m always happy to help with anything Provar or other!


Hi yes I believe we did!
Thank you :slight_smile:


We had a bug recently in one of our pipelines such that errors were being swallowed rather than stopping the pipeline. So it might be worth having some way of testing your pipeline[s], to check that they really will stop if there’s a problem in step X (if that’s what you want).

Also, what level of detail is useful, to whom? Do you need e.g. a list of all the unit regression tests that have passed or failed, or just a summary of how many there are of each kind? If you summarise, how can you get the extra detail if you need it, e.g. to investigate build failures? We summarise in the main output, and write the detail elsewhere, and put something in the summary about where to find the details.

Who needs to get notified about what, when and with what info?

I guess some high level advice is: the pipeline is a software artefact in its own right. So it’s probably worth doing at least some of the normal software development stuff (as much as is helpful for you):

  • What are the requirements for the pipeline, who decides on what they are and their priority? Does this priority reflect the effort and risk involved?
  • Are there dependencies between the different bits of work that will influence the order things get implemented?
  • Is stuff under source control, is it reviewed, is it tested (see my point above)?
  • Do you need a separate development environment, or is it OK to tinker directly with production? If you have a separate development environment, how will you address risks of (not) making the same changes to production?
  • I’m sure you can think of other things I’ve missed.

If this sounds like your needing product management for the pipeline, with some kind of backlog and owner, then yes, I think you do. If you don’t do this explicitly, then it will be implicit in people’s heads and probably inconsistent as a result.

So, for instance, you might have requirements that appear to pull in opposite directions, such as: must have fast feedback, and must run loads of tests. One way of resolving this is to throw lots of hardware at the problem. Another way is to have more than one pipeline - a fast one that’s run often e.g. as soon as a commit happens, and a slower one that runs if the first one succeeds or overnight etc. I’m not saying which solution is better for you, except that in general having the pipeline’s requirements explicit and agreed upon will help manage expectations and minimise grumpiness.


Thank you these are all really useful points for me to consider :slight_smile:
I am asking everyone so many questions at present which seems to be the best way to go.
The questions you have posed above will add to my list!
Good point about testing the pipeline also.
Many thanks.

1 Like

I’m seeing a lot of really useful suggestions already which is great of course.

The one thing I’d like to add is to just start building something small. It’s very tempting to try to come up with a design up front that incorporates everything you can think of (which will still be lacking), but CI/CD is also something you need to get a bit of feeling for. It’s one of those things that I think is best learned by just doing. You’ll find out fast enough what works and what doesn’t.
And that also shows you the power of automation and consistency. The amount of confidence you will get from that first tiny test suite with very little coverage, that gets run over and over and over again, is something that I will never forgot. It showed me that consistency > coverage, and therein lies one of the true values of CI/CD.


In case it’s useful, I’ve turned my answer (and pinched @hylke’s excellent point - thanks for that) into a blog post with a bit of editing and a little extra stuff like triggers: CI/CD pipelines are software artefacts – Random Tech Thoughts


Hi @saraheaton , welcome to MoT.

Always fun to work on CI/CD. Just curious, what type of products do you plan on setting up build pipelines for ? That would help understand your situation better. Ta


Hi this will be for Salesforce

1 Like

Thanks the blog post is excellent! Really useful :slight_smile:


In this regarding, everything from Dave Farley is gold. He has a good Youtube channel and book. This video gives a good idea on Continuous Delivery.

Also, Continuous Integration is a practice to implement Continuous Delivery. CI/CD is like saying “a Ball/Soccer game”.


Hey Sara,

CI and CD as you must be already aware are about Continuous Integration and Continuous Deployment practices. It is actually a modern development technique where the frequency of edits and updates in the code is more rapid as the objective is to find the most reliable solution for the given problem or purpose.

In short, there is nothing so special that you may need to learn when starting with CI and CD testing.