So, the 2018 State of DevOps report has just gone live. The report is created by DORA (DevOps Research and Assessment) lead by Dr Nicole Forsgren and co-authored with Jez Humble and Gene Kim.
I intend to share some thoughts on the report here tonight/tomorrow. This is a highly respected report and contains some interesting snippets and insights regarding âContinuous Testingâ out there in the industry, so I encourage folk to read it.
Quick observation from me is that itâs great that âmanualâ (I know thereâs some debate around using this as a way to describe non-automated testing) activities are mentioned as an important part of Continuous Testing.
Given a lot of debate Iâve seen recently on whether we can automate all the testing, I think the fact that exploratory testing is highlighted can only be good for testing!
they define âeliteâ and âhighâ performers based on release metrics (deployment frequency, change lead time, time to restore service & change failure rate) rather than customer happiness⌠It seems as if product quality doesnât play a part in defining âhigh performanceâ.
their definition of continuous testing is different from what I know it as (there is nothing in there that talks about testing the idea of the solution, the designs, the artefacts, the code, etc - all the pre-development parts of the SDLC).
they mentioned the words âexploratory testingâ once in the entire document, and the sentence itâs in makes me think that they donât understand that ET is an approach to testing. Which tells me that there is a huge disconnect between the DevOps community and the testing communityâŚ
the way they talk about âvalue addâ and ânon value addâ feels a bit icky⌠There is work that definitely adds value indirectly (like testing - it definitely affects value!), but it feels like this is being labelled as non value adding.
the gender gap is huge⌠ď¸
I like the section on influencing culture through leadership and autonomy!
âoutageâ is the biggest word in the retrospective map. I donât know what to make of thatâŚ
The âmanualâ activities they mention donât appear to be specifically testing though⌠And they still show less as being more elite and high performers. ET gets a shout out once, but it feels like they donât really understand it, which is a shame.
I was referring to the use of the word manual in the Continuous Testing section of the report where they mention âperforming manual test activities.â
Maybe Iâm over-egging the importance of exploratory testing being mentioned, but with a lot of debates Iâve seen recently on Twitter/LinkedIn about âautomating all the testingâ, it was quite refreshing to see it mentioned in a DevOps report.
Do they understand exploratory testing? Possibly not. But at least theyâve mentioned it as an important activityâŚ
ď¸