TestBash Spring 2023 - Infrastructure Testing Strategy - Mohammad Siddiqui

During TestBash Spring, @mo_sid gave a talk on Infrastructure Testing Strategy.

We’ll use this Club thread to keep the conversation going, share resources and answer any questions we don’t get to during the live session.

1 Like

Questions Answered Live

@the_qa_guy - How do you convince managers that testing the environment is important in a world where testing time is limited and features must be delivered?

@moalkhalil - How can the test environment be used to test the production infrastructure, given that the two environments are typically different?

@goranmaksimovic - Which part of infrastructure testing was most useful to you? What is a good position for a start, if none of the infrastructure testing has been conducted?

Anonymous - How to test the prod infrastructure on a regular basis?

@ivoqa - Based on your experience, how much do Test Strategies differ from project to project, on higher level? How different are client’s demands and expectations?

Question Not Answered Live

@fernandoteixeira - What is your view from the current available tools for testing infrastructure as code? Like Terraform testing or Kubernetes files validation

@fireavley - How large a team do you use for infrastructure testing, and do you specialise, or do you switch to app testing too? Are there frequent changes to your tests?

Anonymous - Feels like a huge waterfall Strategy opposed to infrastructure - like cpu of a server - could that not have been tested in its own module area?

@ben.mueller - Do you use a reporting collector or some dashboard / monitoring for the tests?

Anonymous - What’s your experience been with other infrastructure stakeholders in the team, while setting up infrastructure testing?Did you face challenges & how u managed

1 Like

@fernandoteixeira - What is your view from the current available tools for testing infrastructure as code? Like Terraform testing or Kubernetes files validation

Ans.
My view on using Terraform for testing the infrastructure as code element:
This is a good point, since we are using terraform to set up infrastructure elements like servers, we should be testing the process of generating infrastructure elements.
Some example scenarios I can think of here are as below:
• Verifying whether server and databases are created post running a terraform script
• Static testing, that is review the scripts to ensure they are as per requirement.
• We could be running infrastructure testing post the running of the terraform scripts
However, Terraform on its own cannot be used for testing infrastructure as code. We’ll need the tools mentioned in the talk.

@fireavley - How large a team do you use for infrastructure testing, and do you specialise, or do you switch to app testing too? Are there frequent changes to your tests?

Ans.
Infrastructure testing should be performed by the existing test team. If there is a test environment team they should be involved in infrastructure testing.
Tests for infrastructure changes, if there is a change to the infrastructure i.e. new security policy, servers changed or added, new database created etc. Tests would therefore need to be reviewed regularly.

Anonymous - Feels like a huge waterfall Strategy opposed to infrastructure - like cpu of a server - could that not have been tested in its own module area?

Ans.
I am not sure why this feels like a waterfall strategy. Waterfall strategies mainly involve waiting to run tests only after a phase is completed whereas agile involves flexibility where tests can be performed at every iteration or on a regular basis. Infrastructure testing needs to be performed alongside a testing cycle.
I am not clear on what is meant by testing in its own module area. But assuming it means testing components in isolation, I would suggest it is not a good idea.