We’re continuing our work to create a curriculum focused on automation that is created with feedback from the testing community. We’ve already run a series of activities that have helped us identify a list of key tasks that helped us create this Job Profile.
We’re now in the process of going through each task and analysing them to identify the core steps we take to achieve them. For this post we’re considering the task:
Create Automated Tests
We’ve run a series of community activities including social questions and our curriculum review sessions to identify the steps we need to take to successfully achieve this task and have listed them below:
Discussion with Stakeholders, Product Owners, Development about the scope and purpose
Determine whether the test in question should be automated or not
Determine what layer the automated test should be on
List what state, actions and assertions the check will carry out
Codify the creation of state
Codify the actions the automated test will take
Codify the oracle in the automated test (assertion)
Run the automated test against the product to debug issues
Test the automated test by switching the assertion to fail on purpose
Store the automated test using version control
Have team members run a code review on the automated test
Run automated test as part of a build process
What we would like to know is what do you think of these steps? Have we missed anything? Is there anything in this list that doesn’t make sense?
What are your thoughts on these steps? How do you go about building your automated tests?
I would phrase it “Develop Automated Tests”.
Automation is development, a specific type, similar to any feature developed by developers. Including all the thinking, design and architecture stuff.
I’m sorry I was unable to make the session - work got in the way.
Some questions…
Will it assume the person developing the automated test is doing so from a blank slate?
Is the order of the points, as they’re presented, meaningful?
– Assuming yes, unless ‘Discussion with Stakeholders, Product Owners…’ is related to working from a blank slate, then I would recommend it being a secondary point for ‘introducing automated testing to your team/org’ otherwise that part of the process is overkill in my experience
Does this include what are typically called ‘developer tests’ or ‘unit/integration tests’?
Are there particular patterns that are going to be taught?
– If so, then I’d want to merge
** Codify the actions the automated test will take
** Codify the oracle in the automated test (assertion)
Does ‘Codify the creation of state’ include stubbing/mocking?
–If so, I’d recommend another section on “managing ‘fake’ state to reduce assumptions”
Will it assume the person developing the automated test is doing so from a blank slate?
No. Once we dive into learning outcomes, we’ll aim to cover a number of learner profiles, and context.
Is the order of the points, as they’re presented, meaningful?
– Assuming yes, unless ‘Discussion with Stakeholders, Product Owners…’ is related to working from a blank slate, then I would recommend it being a secondary point for ‘introducing automated testing to your team/org’ otherwise that part of the process is overkill in my experience
That would have been covered in the earlier responsibility of strategy. If you open the job profile link you’ll see the other responsibilities.
Does this include what are typically called ‘developer tests’ or ‘unit/integration tests’?
Yeah, the steps are high level enough they would apply to make layers. So when we create content and learning outcomes, we’d include content on multiple layers/examples. But most of these steps apply no matter the test layer.
Are there particular patterns that are going to be taught?
Unknown yet. But again its high level. Every test has a set of actions. Click this, call this API, create this object, call this method. These are all actions.
Identifying a good oracles and then implementing it is a completely different set of skills.
Does ‘Codify the creation of state’ include stubbing/mocking?
The high level skill is knowing you need to think, manage and create state.
If I were creating content, I would absolutely have a section on mocking/stubs and I’d include how you maintain them. This would be translated via the learning outcomes which is the next step.
all of the above. Basically a tool where you don’t ‘write code’.
I associate ‘Develop’ with code. I’m not saying it can’t be changed, just trying to fully understand your concern and reasonings.
I don’t see development only being related to code.
It is a spectrum from “easy” to “hard” and you can develop with and without code at both ends of the spectrum.
Many “easy” development is somethings called “scripting”. But even creating scripts can become very demanding.
Not only the development of an application is development.
Most important is to me the mindset, not so much the concrete tool and tech.
A developers mindset, as of being a creator, is different from being a tester, looking for problems in a system.
This is QF-Test, a tool for UI automation.
You literally click an equivalent of code together.
Sure, it is not typing code as in an IDE, but you have the same constructs.
To create this you need a mindset of a developer.
