We’re continuing our work to create a curriculum focused on automation that is created with feedback from the testing community. We’ve already run a series of activities that have helped us identify a list of key tasks that helped us create this Job Profile.
We’re now in the process of going through each task and analysing them to identify the core steps we take to achieve them. For this post we’re considering the task:
Create Automated Tests
We’ve run a series of community activities including social questions and our curriculum review sessions to identify the steps we need to take to successfully achieve this task and have listed them below:
- Discussion with Stakeholders, Product Owners, Development about the scope and purpose
- Determine whether the test in question should be automated or not
- Determine what layer the automated test should be on
- List what state, actions and assertions the check will carry out
- Codify the creation of state
- Codify the actions the automated test will take
- Codify the oracle in the automated test (assertion)
- Run the automated test against the product to debug issues
- Test the automated test by switching the assertion to fail on purpose
- Store the automated test using version control
- Have team members run a code review on the automated test
- Run automated test as part of a build process
What we would like to know is what do you think of these steps? Have we missed anything? Is there anything in this list that doesn’t make sense?
What are your thoughts on these steps? How do you go about building your automated tests?
I would phrase it “Develop Automated Tests”.
Automation is development, a specific type, similar to any feature developed by developers. Including all the thinking, design and architecture stuff.
I’m sorry I was unable to make the session - work got in the way.
- Will it assume the person developing the automated test is doing so from a blank slate?
- Is the order of the points, as they’re presented, meaningful?
– Assuming yes, unless ‘Discussion with Stakeholders, Product Owners…’ is related to working from a blank slate, then I would recommend it being a secondary point for ‘introducing automated testing to your team/org’ otherwise that part of the process is overkill in my experience
- Does this include what are typically called ‘developer tests’ or ‘unit/integration tests’?
- Are there particular patterns that are going to be taught?
– If so, then I’d want to merge
** Codify the actions the automated test will take
** Codify the oracle in the automated test (assertion)
- Does ‘Codify the creation of state’ include stubbing/mocking?
–If so, I’d recommend another section on “managing ‘fake’ state to reduce assumptions”
I’ll add more as I get time
Would you still prefer that term if such tests weren’t written in code and were created in a vendor tool for example?
What do you mean by vendor tool?
Or something like Soap-UI or Postman?
all of the above. Basically a tool where you don’t ‘write code’.
I associate ‘Develop’ with code. I’m not saying it can’t be changed, just trying to fully understand your concern and reasonings.
I thought about this for a while.
I don’t see development only being related to code.
It is a spectrum from “easy” to “hard” and you can develop with and without code at both ends of the spectrum.
Many “easy” development is somethings called “scripting”. But even creating scripts can become very demanding.
Not only the development of an application is development.
Most important is to me the mindset, not so much the concrete tool and tech.
A developers mindset, as of being a creator, is different from being a tester, looking for problems in a system.
This is QF-Test, a tool for UI automation.
You literally click an equivalent of code together.
Sure, it is not typing code as in an IDE, but you have the same constructs.
To create this you need a mindset of a developer.
What do you feel and think?
Does this include what are typically called ‘developer tests’ or ‘unit/integration tests’?
Not to forget, that you can create unit checks for your automation in testing
In general I would argue that typical automation in testing is not (so much) about unit and integration level.
Typical are API and UI automation.
A few years ago I explored TDD in Test Automation.
Test Driven Development is a continuous cycle of Red Green Ractor.
TDD can even be combined with Visual Testing: