How do you “Build a framework”? - Automation in Testing Curriculum

🗄️ Archive
, ,
1

Hi all,

We’re continuing our work to create a curriculum focused on automation that is created with feedback from the testing community. We’ve already run a series of activities that have helped us identify a list of key tasks that helped us create this Job Profile.

We’re now in the process of going through each task and analysing them to identify the core steps we take to achieve them. For this post we’re considering the task:

Build a framework

We’ve run a series of community activities including social questions and our curriculum review sessions to identify the steps we need to take to successfully achieve this task and have listed them below:

  • Identify framework requirements
    • Reporting requirements
    • Standalone / embedded?
    • Execution requirements (parallel runs, multiple browsers etc.)
    • Test data management?
  • Create an architecture diagram of the type of solution we want
  • Run an proof of concept spike
    • Technical analysis on which tools can and cannot be used
    • Develop a basic framework
    • Test out off the shelf tooling
  • Create a space to version control the framework in
  • Setup framework with required libraries and configurations
  • Create an initial check to demonstrate necessary abstractions
  • Sharing knowledge of framework with others
    • Create documentation detailing how to use the framework
  • Upload framework within version control
  • Integrate framework in CI/CD

What we would like to know is what do you think of these steps? Have we missed anything? Is there anything in this list that doesn’t make sense?

What are your thoughts on these steps? How do you go about building a framework?

2 Likes
4

Looks really good! Forgive me if this is wrapped up in another bulletpoint and you don’t think it needs calling out explicitly (or its in another section) but what about choosing the language?

A lot of frameworks only support some languages, and if its important that you align with the language supported by the wider engineering team then maybe this needs to be factored in.

1 Like
5

I am not sure if this fits here but I have found a controversy in building the framework involving using an object oriented framework or one that uses only functions.

This is something I have found to be critical as tests are added and more projects are connected for end to end test runs. This needs to be decided in the begining. Mixing this two aproaches can be cumbersome and confusing later in the project.

So, OOP or function libraries?

1 Like
6

Hey there Christopher . nice callout.
Welcome to the Ministry of test community.

Language choice
Personally this is the kind of point I love to get more context on, as a coder-turned tester I hold a lot of controversial views on framework self-building, since I’ve worked on a team doing this on more than one fashion or other. I’ve only ever built a tiny Framework on my own, and all the scripting was via CSV files as it made parsing easier. But that obviously did not scale, and hence I’m not a fan of underestimating the amount of work around things like the “language” your test tool talks or DSL (Domain Specific Language.) Since DSL are more mature of late, it’s worth trying to use a DSL framework and not try to invent your own if you do choose to create a new language. OO DSL languages also exist now see What are Domain-Specific Languages (DSL) | MPS by JetBrains . But deciding on a DSL is very very context specific, whatever you do it needs to be extensible at runtime and easy to debug, let alone support OO. Currently I am using a complex python fork of “pytest” mixed with “nose”. So we have OO, but it comes with a load of baggage and my other favourite snag, is a lack of separation and a bit too much cohesion at times.

I can only suggestion designing things layered from the get-go and build skeletons or spikes to check that your layering make sense. An early iteration of one project that had too much bonding in used C#, and I’m ever since become allergic to typesafe and compile-time or early binding test framework languages. A problem which further makes things in multiple branch test-tool development version control more painful by far, because you will quite possibly find yourself unable to easily support multiple simultaneous versions of the framework in deployment or CI/CD side by side if you went down that road. So yes, do spikes.