TestBash SF 2019 Live Blog: Automation Yoga - Stretching ROI with Testing SDK

This has been such a good line-up. Pressure is on for Brendan Connelly to keep this going. But with a title like Automation Yoga, he has me intrigued.

OH! Brendan is the fellow who cheered me on while I was changing my 3 week old’s diaper in the mens room. I am cheering for Brendan now.

Ok, theme set. Brendan comes in with a soft voice, asking us to think about how we feel about test automation. Super zen. What are the hopes and dreams? How do they differ across your company, across the industry? Across this audience?
What if we used a maturity heuristic? Not a new concept… it was applied for REST. Can we consider this when considering expectations and experiences for test automation

Level 0 - Test without a framework
Tool focused.
Have a problem… want it gone… buy a fix.
The Dream → apply the tool and problems are gone.
It is like being a kid in a candy store… but who is the kid? And who owns the candy store?
Treating test automation as a cure implies testing is a disease.
The tools generally do what they claim to do, but the expectations around the project are inaccurate. The tool does not fix the problem because the problem is misidentified.

Level 1 - Framework Exists
Honeymoon is over… have to maintain the tests
Want Unified automation. Portable, maintainable, reliable.
This requires that we are pattern focused. Design, conventions…
But, though we want our code to be dry and look like dev code, it is not really why we write them. It is hard to find a balance. Patterns and Conventions are process for your code. It increases bureaucracy, complexity… and makes it harder to bring in other testers

Level 2 - Multilayer framework
Now we have a lot of tests. We get into grouping and prioritizing tests. Analogy… If you are not running all your tests all the time, it is like only brushing some of your teeth. User facing teeth first :smiley:
We become architecture focused in our grouping. Leveraging application layers, and carving out UI from testing.
Hybrid Maturity - As we start to pay more and more attention to lower and lower levels of our architecture and tech stack, we need to spend a lot more time focusing on our own testing architecture. It becomes easy to suddenly succinctly answer the question: What are we actually testing here? What is our strategy?
We have to start accepting approximations, understanding motivations and describing complexity.
At some point, this becomes a bit of an ego contest: How big is your test suite? How fast is it? How long does it run?

Issue: We have a rift in quality. Our more extensive testing takes us further and further from our customers. We might be all on team quality, but our automation does not reflect it. Manual testing still represents our customers, but they are never something eng teams brag about.

Niche Market
Framework value is constrained by what tests matter.

Level 3 - A testing SDK
Platform focused.
Set of tools to compose solutions
Libraries and packages that can be consumed outside of our tests. Used outside traditional tests.

Proposal: Reframe how we think about automation
FINALLY Yoga comes into the conversation
Does automation provide the same sort of relief as Yoga does?
Pyramid is useful as a list of layers, but it does not speak to benefit.
Okay… we have now established the Quality Cathedral Quality of the application is at its core. Create and use libraries and tools like buttresses to support our product to immense heights that could not exist without them

Concepts
Relieves Pain - (okay, I missed this while I was still thinking about the cathedral analogy)
Bundles of Joy - for testers and devs alike. Apps, scripts, chatbots… external integrations hitting their api’s, open sourced projects
Raises Consciousness - Expose > Engage > Contribute > Evangelize. Successful automation needs and engaged audience and people need to feel the impact. (Great story about how his father took on yoga in his 70’s. Was not the sort to embrace it, but having experienced the benefit and value, he became one of the strongest promoter of the value. He would invite others to come in and try.)
Reduced Anxiety - Automation currently is an exclusive club. Lots of folks feel they are not invited. How do we make it more inclusive? A more inclusive platform for quality? Trying to do this loses nothing, but has an immense potential upside. Let testers be testers and find a way to support them.
Feeling of Peace - Allow everyone to take care of themselves and do their best work with built in support for quality.

So zen… so cool. You need to watch this one on the dojo if you get a chance