Ask Me Anything: DevOps

Tonight we had @ns1 join us for our first Ask Me Anything of 2020 talking all things DevOps.

As always, I’ll share the resources below that I was able to catch on the night. If I missed a resource or you have another question you’d like to ask, please share it on this thread.

If you missed the live session, a recording will be available on the Ministry of Testing website.

An XKCD reference :sweat_smile:

Deciding what to test, recommended reading

Along with works from Deming discussing value.

And of course, quite a lot of questions we didn’t get to as we had so many!

  1. What is your definition of DevOps?
  2. What we need to learn, as Testers, to contributed in DevOps?
  3. Can you describe an outage your team experienced? How could testing have prevented that outage? What type of testing?
  4. Performance testing in DevOps.
  5. My company has not fully embraced DevOps yet, now we are introducing TestOps. Some say DevOps and TestOps are the same thing (the latter only focusing more on testing). Others say these are different, although it is possible to combine them. What is your opinion on this?
  6. It’s easy with topics like DevOps to get stuck on the implementation-level (focusing on tools and process rules), slowly losing touch with the principles behind those. That’s often also how you lose the value of something like DevOps. Do you have any advice on how to avoid this?
  7. How can a tester transition from QA & testing to Devops?
  8. Still talking about Continuous Deployment: covering all possible scenarios upfront is not possible, therefore we need to have a good monitoring, observability and so on. What are your thoughts on testing in Production?
  9. Any advice on improving logging for applications to make them useful or logging strategies? Any books or articles you’d recommend?
  10. What does testing mean in the context of infrastructure? What does it mean to test a physical server? Is it different from testing a virtual server? Is it the same as testing a website?
  11. What are specific examples of processes to test infrastructure (not applications)? Is there any publicly documented example of how teams test infrastructure? (I am interested in anecdotes, not tools)
  12. In any team that you worked with, have testers tested infrastructure (not applications)?
  13. Tools used in DevOps, and what exactly they do.
  14. It is common in an Agile approach for a business person to check if that story delivered what was expected or do showcases for stakeholders, but how would that apply with a Continuous Deployment culture? An alternative that we are exploring is to deploy a new version (without end user access) and do any sign off or showcase there, before releasing to the users. What are your thoughts on that?
  15. What’s a typical day look like for Devops engineer?
  16. How do you find exit criteria of testing?
  17. How can design adapt to DevOps practices and culture? I have heard of the use of DesignOps at AirBNB.
  18. Is there one book you would recommend for someone working in a DevOps team?
  19. What is the most important skill to learn in order to adapt to a DevOps culture?
  20. Have you seen any DevOps leader (blogger/speaker/author) who can publicly test applications or infrastructure? What is their process for doing so? (I am interested in DevOps/Ops people, not testers)? Do they have a github repo focused on testing for a project?
  21. Among application developers there is a strong concept of unit testing. Is there any such practice related to infrastructure? (I am interested in developers, not testers) If not, why not?
  22. In devops you need to integrate often, merge you changes in to trunk. When does exploratory and manuall testing happens? Before or after the merge?
  23. How does DevOps differ from QAOps (or is there any relation between the two), if possible?
  24. Shouldn’t TestDev come before DevOps? In TestDev we involve developers in testing (not test automation).
  25. Would your team benefit of having a tester (or test-minded person)? If so how this person’s work could look?
  26. What skills you would look for when hiring a DevOps engineer?
  27. What are top 3 impovement suggested in CD ?
  28. How do you prepare a team to be ready for introducing a dev ops culture? What are your top 3 things to focus on initially?
  29. What do you think about Google’s SRE model (referring to theirs Book). Which things you liked from it and which you didn’t?
  30. How can DevOps and QA collaborate make things for efficient?
  31. TDD any good advice/practice to start applying TDD in a team where developers and testers are not used to collaborate in test automation, an even testers are skeptic about it (“first I want to test it myself”). Thanks!
  32. Should security testing be part of ada testing?
  33. What are some interesting things you’ve seen Devops teams do for delivery teams which you’d recommend or wow’d you? eg initiate build/deploy/rollback via slack

Is there one book you would recommend for someone working in a DevOps team?

The book you must read is ‘The Phoenix Project’. This is like ‘The Goal’. It is a business novel. Keep in mind that the environment is enterprise IT, not product development. I haven’t seen a lot of testers write about enterprise IT. So not sure if this will resonate.

Don’t expect to see any insights related to testing, i.e., the KanerBachBolton type of testing.

I didn’t find most of the other books exciting, like the agile books. There is a lot of tech.

The Google SRE book is readable and free: https://landing.google.com/sre/sre-book/toc/index.html
That is different from ‘The Phoenix Project’, it is focused on product companies like Google, Netflix.

Even the SRE book has a lot of tech. Unless you work with it, it is difficult to relate. The Phonenix project is a narrative, and I found it easier to understand.

One last note - for most people new to DevOps, testing in DevOps is focused on test automation. That is a mistake. Don’t start with that premise when you read.