For my work we want to go also to use Jira for writing test cases.
Now it is not in place yet.
And as I am the only certificed tester it is my task to find the best solution for the developers and the other testers.
How we want to do it? That the test cases are under a story.
We are working with SCRUM and this gives us the best overview.
Another requirement is that we want to make and fill in the test cases in a table.
In the Jira that we have now that is time consuming.
Are these test cases going to be used by the developers or the testers? Also whatās the objective of them? I would generally write acceptance criteria within the story (Using something like Given When Then) and include what risks threaten the story or that the story is trying to mitigate. Then, during kick-off, I would discuss with the developers any specific areas Iād expect them to write automated checks to ensure these risks actually are mitigated (Though this is just for specific stuff I want to be sure they cover, Iād still expect them to be writing other automated checks beyond the specified ones).
And then when testing, if the story is just a small size, Iāll typically just detail brief testing notes in the story notes, and during sign-off, go through these, the risks, and how weāve mitigated or avoided these risks.
For bigger stories, Iād detail some test charters to assist with exploratory testing, and Iād include my testing notes and the charter in the story notes, and go through these during sign-off.
Personally, Iād stave away from being too prescriptive with how to format testing notes. Some people may want to draw a table, and enter details into each, but others may draw things like mind maps, or bullet point things, and so long as they can add these to into the story notes and can explain them back, I think itās all fine.
These test cases are going to be used by testers.
The objective it to write test cases on a structured way.
When you now want to write a table it is very time consuming. And all the testers are doing this.
So this is very irritating.
Personally Iād move away from writing test cases like that. Itās basically just automated checks, but done by humans in a far less efficient manner, and your developers should be writing those. Do your developers currently write many automated checks?
Hmmmā¦ Again, Iād strongly advise moving away from that model, though if itās entrenched and historical, I understand it can be difficult to do. As a quality person in your organisation though, you should be strongly advocating for it, as it will free up the time of your testers to do better work, and will help improve the quality of your product overall. Particularly if youāve recently adopted scrum, this could be the best time to introduce some additional change.
Although your developers shouldnāt find it as big a change, as itās effectively just writing more code.
Part of the reason youāll struggle with doing things like this directly in Jira, is that Jira doesnāt really account for doing things like this.
Something alternative you could try though, is writing your test cases in a regular online spreadsheet application (Google Sheets, Excel Online etc), and providing the link to them in the story.
But again, the best solution is that your developers are writing automated checks, and your testers are freed up to improve the quality through exploratory testing, pairing with developers, and advocating for quality during stand ups/team meetings etc.
Great plugins for test management do exist in Jira, but a lot of other free tools do exist. I myself am also looking at using jira to manage test cases. Iām sitting on top of just over 1000 test, many of which are thankfully automated already. Iām probably going to stump up the cash for the Jira plugin next year, but until then I am keeping test cases as lightweight as possible and using wordings like the āif then whenā.
The trouble with explicit or detailed wordings, is that when developers run them manually, the follow them monkey-style. They then get the bright idea to automate and make the same monkey-style mistake in the automated test. I thus keep test case capture as brief and vague as to implementation as possible. I donāt mind that automated tests are monkey tests, because they create a āpositiveā path that lets me know quickly if the product ācanā be made to work using the same workarounds the automation script used.
Excel sheets do not scale well though, so I suggest finding budget and biting the Jira plugin rental bullet. Formal test case management has all the obvious overheads that source code has, so word documents and spreadsheets approach requires some supporting tooling if you do go that route.
I havenāt worked with it for years, so I canāt offer advice about if itās still good, but the reviews seem to be positive, and it was good the last time I used it for test-case purposes. Using such a tool is a lot easier than making your own thing, and for small(er) teams, it really doesnāt add a lot of costs.
We use Jira and are moving over to the X-ray plugin for test case management. We have also recently adopted the Gherkin syntax Given When Then for our manual test cases and this can be easily used on Xray. My team and I are really liking it so far and would recommend it.
Hey, we recently started to use Xray and started Gherkin syntax. We were able to load in our Regression suite we had in Excell into Xray with little effort as well
Hi @suuske1981,
Iād recommend trying out Xray (https://www.getxray.app/); you can use it for manual, scripted tests and also for Gherkin (e.g. Cucumber, Specflow, Behave) based ones and even for other automation frameworks, whenever youāre at that point. The interesting part is the ability to track coverage right in your user stories or in your Agile boards.
Your opinion is really the best, so Iād recommend that you try it out for yourself. Please have a look at this recording of a webinar showing it in practice: https://register.gotowebinar.com/recording/9169982352850824461
Side note: you should complement check-based with exploratory testing.
Used Zephyr a few years ago, and find X-ray much better. The only problem I find with X-ray is when executing tests, itās not bad, but you canāt open the individual executions in a new tab and if you navigate away from the execution overview, when you return, the filters are reset. A solution to this is keeping your executions nice and short (i.e. breaking down regressions) and avoiding having more than one tester do the same execution
We did consider Zephyr and XRay (XRay I preferred the look of) but one of our key requirements was the ability to allow changing of tools as we grow independently of each function. So we decided to go for a separate Test Management System that integrated with Jira. TestRail is currently working for us and weāve developed further integrations using both TestRail and Jira APIs to help coverage tracking and reporting. However, if we want to move to a new TMS, we can export our test cases and import them into a new tool even if weāre happy with Jira.
For the last five years Iāve been using synapseRT plugin for Jira as a test case management system. At that time only two plugins were available: synapseRT and zephyr. SynapseRT was better, in my opinion, and cheaper. Itās an intuitive and easy-to-use tool. With its REST API, I automated the test results update when running Selenium. Highly recommend.
Iām currently setting up a system similar to what Iāve used before. Its using Jira tickets, no plugins. I didnāt want to invest in a tool because I hope to move away from test cases and toward agile. Using a tool could establish procedures and expectations that make moving further agile difficult. It does require some Jira admin skills though.
Create a new issue type, call it Test Case - whatever
Add to that a custom field āVerified Versionā, use the multi-select version field type.
A Test Case issue may include several related āchecksā to be executed in a single test session. I donāt make a table, just add a list of checks in the description.
Once the checks have been completed a comment is added which contains any info relevant to that test session, the tested version is added to the Verified Version field and any found issues are reported seperately as bugs which are then linked. The issue is closed (resolved) as complete not āpass/failā. Therefor the project is tracked in terms of:
planned testing completed
issues found
issues fixed, verified etcā¦
At the end of a test cycle the tests can be bulk re-opened ready for the next cycle after reviewing etcā¦ The verified versions and comments remain intact.
Its not perfect, but as a halfway solution towards agile I think its adequate. It would be difficult to manage large suites of tests across multiple projects. Re-usability is limited to cut-n-paste or issue clones. Iām working in a embedded, IoT, cloud services business, the last thing I want is a huge suite of manual test cases; but for now I needed something.
