How do you "Create an automated test? - Automation in Testing Curriculum

Hi all,

We’re continuing our work to create a curriculum focused on automation that is created with feedback from the testing community. We’ve already run a series of activities that have helped us identify a list of key tasks that helped us create this Job Profile.

We’re now in the process of going through each task and analysing them to identify the core steps we take to achieve them. For this post we’re considering the task:

Create Automated Tests

We’ve run a series of community activities including social questions and our curriculum review sessions to identify the steps we need to take to successfully achieve this task and have listed them below:

  • Discussion with Stakeholders, Product Owners, Development about the scope and purpose
  • Determine whether the test in question should be automated or not
  • Determine what layer the automated test should be on
  • List what state, actions and assertions the check will carry out
  • Codify the creation of state
  • Codify the actions the automated test will take
  • Codify the oracle in the automated test (assertion)
  • Run the automated test against the product to debug issues
  • Test the automated test by switching the assertion to fail on purpose
  • Store the automated test using version control
  • Have team members run a code review on the automated test
  • Run automated test as part of a build process

What we would like to know is what do you think of these steps? Have we missed anything? Is there anything in this list that doesn’t make sense?

What are your thoughts on these steps? How do you go about building your automated tests?

5 Likes

I would phrase it “Develop Automated Tests”.
Automation is development, a specific type, similar to any feature developed by developers. Including all the thinking, design and architecture stuff.

3 Likes

I’m sorry I was unable to make the session - work got in the way.

Some questions…

  • Will it assume the person developing the automated test is doing so from a blank slate?
  • Is the order of the points, as they’re presented, meaningful?
    – Assuming yes, unless ‘Discussion with Stakeholders, Product Owners…’ is related to working from a blank slate, then I would recommend it being a secondary point for ‘introducing automated testing to your team/org’ otherwise that part of the process is overkill in my experience
  • Does this include what are typically called ‘developer tests’ or ‘unit/integration tests’?
  • Are there particular patterns that are going to be taught?
    – If so, then I’d want to merge
    ** Codify the actions the automated test will take
    ** Codify the oracle in the automated test (assertion)
  • Does ‘Codify the creation of state’ include stubbing/mocking?
    –If so, I’d recommend another section on “managing ‘fake’ state to reduce assumptions”

I’ll add more as I get time

1 Like

Would you still prefer that term if such tests weren’t written in code and were created in a vendor tool for example?

What do you mean by vendor tool?
“Code-less” tools?
Or something like Soap-UI or Postman?

  • Will it assume the person developing the automated test is doing so from a blank slate?
    No. Once we dive into learning outcomes, we’ll aim to cover a number of learner profiles, and context.

  • Is the order of the points, as they’re presented, meaningful?
    – Assuming yes, unless ‘Discussion with Stakeholders, Product Owners…’ is related to working from a blank slate, then I would recommend it being a secondary point for ‘introducing automated testing to your team/org’ otherwise that part of the process is overkill in my experience
    That would have been covered in the earlier responsibility of strategy. If you open the job profile link you’ll see the other responsibilities.

  • Does this include what are typically called ‘developer tests’ or ‘unit/integration tests’?
    Yeah, the steps are high level enough they would apply to make layers. So when we create content and learning outcomes, we’d include content on multiple layers/examples. But most of these steps apply no matter the test layer.

  • Are there particular patterns that are going to be taught?
    Unknown yet. But again its high level. Every test has a set of actions. Click this, call this API, create this object, call this method. These are all actions.
    Identifying a good oracles and then implementing it is a completely different set of skills.

  • Does ‘Codify the creation of state’ include stubbing/mocking?
    The high level skill is knowing you need to think, manage and create state.
    If I were creating content, I would absolutely have a section on mocking/stubs and I’d include how you maintain them. This would be translated via the learning outcomes which is the next step.

1 Like

all of the above. Basically a tool where you don’t ‘write code’.
I associate ‘Develop’ with code. I’m not saying it can’t be changed, just trying to fully understand your concern and reasonings.

1 Like

I thought about this for a while.

I don’t see development only being related to code.
It is a spectrum from “easy” to “hard” and you can develop with and without code at both ends of the spectrum.
Many “easy” development is somethings called “scripting”. But even creating scripts can become very demanding.
Not only the development of an application is development.

Most important is to me the mindset, not so much the concrete tool and tech.
A developers mindset, as of being a creator, is different from being a tester, looking for problems in a system.

This is QF-Test, a tool for UI automation.
image
You literally click an equivalent of code together.
Sure, it is not typing code as in an IDE, but you have the same constructs.
To create this you need a mindset of a developer.

What do you feel and think?

Does this include what are typically called ‘developer tests’ or ‘unit/integration tests’?

Not to forget, that you can create unit checks for your automation in testing :face_with_open_eyes_and_hand_over_mouth:

In general I would argue that typical automation in testing is not (so much) about unit and integration level.
Typical are API and UI automation.

A few years ago I explored TDD in Test Automation.
Test Driven Development is a continuous cycle of Red Green Ractor.

TDD can even be combined with Visual Testing: