2018 Accelerate: State of DevOps Report

Questions
,
#1

So, the 2018 State of DevOps report has just gone live. The report is created by DORA (DevOps Research and Assessment) lead by Dr Nicole Forsgren and co-authored with Jez Humble and Gene Kim.

I intend to share some thoughts on the report here tonight/tomorrow. This is a highly respected report and contains some interesting snippets and insights regarding ‘Continuous Testing’ out there in the industry, so I encourage folk to read it.

You can download it for free here.

Interested to read and see your thoughts on this thread.

Ask Me Anything: CI/CD and Delivery Pipelines
#2

Quick observation from me is that it’s great that ‘manual’ (I know there’s some debate around using this as a way to describe non-automated testing) activities are mentioned as an important part of Continuous Testing.

Given a lot of debate I’ve seen recently on whether we can automate all the testing, I think the fact that exploratory testing is highlighted can only be good for testing!

1 Like
#3

It’s interesting!

A few initial thoughts:

  • they define “elite” and “high” performers based on release metrics (deployment frequency, change lead time, time to restore service & change failure rate) rather than customer happiness… It seems as if product quality doesn’t play a part in defining “high performance”.

  • their definition of continuous testing is different from what I know it as (there is nothing in there that talks about testing the idea of the solution, the designs, the artefacts, the code, etc - all the pre-development parts of the SDLC).

  • they mentioned the words “exploratory testing” once in the entire document, and the sentence it’s in makes me think that they don’t understand that ET is an approach to testing. Which tells me that there is a huge disconnect between the DevOps community and the testing community…

  • the way they talk about “value add” and “non value add” feels a bit icky… There is work that definitely adds value indirectly (like testing - it definitely affects value!), but it feels like this is being labelled as non value adding.

  • the gender gap is huge… :frowning:️

  • I like the section on influencing culture through leadership and autonomy!

  • “outage” is the biggest word in the retrospective map. I don’t know what to make of that…

1 Like
#4

The “manual” activities they mention don’t appear to be specifically testing though… And they still show less as being more elite and high performers. ET gets a shout out once, but it feels like they don’t really understand it, which is a shame.

#5

I was referring to the use of the word manual in the Continuous Testing section of the report where they mention ‘performing manual test activities.’

Maybe I’m over-egging the importance of exploratory testing being mentioned, but with a lot of debates I’ve seen recently on Twitter/LinkedIn about ‘automating all the testing’, it was quite refreshing to see it mentioned in a DevOps report.

Do they understand exploratory testing? Possibly not. But at least they’ve mentioned it as an important activity…

1 Like