Ross,
You’ve received great advice so far. Just thought I’d add a little (hopefully good) advice to tie things together.
I like Will’s approach to determining what to automate, but you first have to decide what to test, which seems to have been handled by whoever takes care of his manual testing. For that end of things, I like to use Bach’s Heuristic Test Strategy Model. It helps me with the initial analysis of the project so I can determine where I need to focus and how I need to approach it. I walk through each piece to get a better understanding of what I’m up against, using mind maps to create things like a product coverage outline that becomes the basis for my test focus areas. Those are then broken down into exploratory sessions/automated tests, or sometimes both. I also use this PCO to track my test ideas, whether it’s general automation, exploratory test, or automated acceptance testing.
At the same time, I’m keeping up with stories that move across the board, which sometimes requires some additional conversation to really nail down the criteria per Joao’s comments. This constant analysis feeds my PCO as well, which is updated on a regular basis. We use a SCRUM/Kanban process, so stories are tracked as cards ala Trello-style, with the acceptance criteria I’m going to automate being one or more child stories that need to be completed for the feature story to be accepted.
That doesn’t work for everything, so stories I can’t get to with automation are verified through hands on acceptance testing and a backlog item is created for writing the test automation, so I keep a separate parent story for all test automation efforts. Everything is tagged so I can report out on the actual development process, since test automation is both a dev project and a test project at the same time.
All of my test automation plus my exploratory testing feeds a simple test dashboard where I can roll up a somewhat subjective score on the amount of effort involved in testing an area, its overall coverage, overall quality, and how much of the automatable test case coverage has been completed. It paints a decent picture for management without having to give them pass/fail numbers which can be so easily misinterpreted.
We get tons of info on automation, but so much of it ignores actual testing as if thinking about it is of secondary importance. Don’t fall into that trap. A million automated test cases aren’t going to help you if you aren’t testing the right things.