Test.Bash(); 2021 - Leveraging the power of your testing with Xray - answers

đŸ—“ïž Events đŸȘ TestBash
,
1

I was lucky to deliver a talk at Test.Bash(); 2021, showcasing Xray and how teams can use different testing approaches (test automation, exploratory testing, test cases) and have visibility of what is happening in Jira, where probably the team is managing their project already.
During my talk several questions came; unfortunately, I wasn’t able to answer live as I had recorded a long video. Lesson learned and I will try to save time, next time :slight_smile:
Meanwhile, I wanted to share the answers to those questions here.

Is this a Jira plugin?

Yes, Xray is a Jira plugin that you can install in your Jira instance, no matter if you’re using Jira on-premises (server/datacenter) or Jira Cloud. This is a paid plugin.
During the talk I also showed Xray Exploratory App, which is a different application (free) that we can run locally in our own desktop/laptop to assist in exploratory testing. Xray Exploratory App can (optionally) integrate with Xray, to have visibility of exploratory testing results in Jira for example.

Where does the name Xray come from?

That’s an excellent question. AFAIK, the X comes from Xpand IT (the original company behind Xray); many products from Xpand IT started with an “X”. Xray become more or less a good name (among others that were discussed), I think becaused besides being simple & cool, it was a good analogy for having something that allowed us to track how things were from the inside 
 to track if there are problems - that’s why we make X-Rays. The name became something so natural that we promptly used rightaway.
I think this is the reason for the commercial name.
I actually know more about the origin of the product, as I was part of it; but that’s a different story, more about that here.

Does Xray also support the Cucumber pre-processor plugin from Cypress?

Yes.
Please have a look at this tutorial for Xray Cloud, or at this one for Xray on Jira server/datacenter.

If we don’t have an automated CI/CD pipeline, can we manually run our automated tests and upload JUnit results to Xray?

Sure :slight_smile: It’s as simple as doing a HTTP POST request. The exact details depend on whether you’re using Xray on Jira Cloud or Xray on Jira server/datacenter.
You can find a bunch of code snippets in different languages on this GitHub open-source project.
You can see here a full tutorial using Java+JUnit5+Selenium, along with a GitHub repository with all the code, ready to run :slight_smile:
The previous tutorial is for Xray Cloud.

How hard (or easy) is to combine Jira and Xray and your Azure DevOps pipelines. And what should you consider?

We have a tutorial for integration with Azure DevOps for Xray Cloud and also for Xray on Jira server/datacenter.

Are there any plans to integrate with Linear?

I don’t know Linear; I’m unsure of which tool you mean exactly. Can you please clarify, if possible?

If you have any additional questions or feedback you want to share with me, please go ahead. I would love to have your feedback.

3 Likes
2

It was great presentation. I was watching it later on Crowdcast. Nice subtle tip on creating new Test Type - Exploratory. That way, you give exploratory testing a bit more meaning that it deserves :+1:

In my company we have been using Xray for just few months. Currently we’re in the process of learning best practices. One thing that most puzzles me is searching for optimal categorisation of tests with Test Sets. What I don’t like with them is that there is no clear way to see a lot of tests at a glance - especially because tests in Test Sets are “hidden” away below other JIRA fields, so normally you always have to A) wait for Xray to load inside JIRA issue (takes time) and B) scroll down to “Screen 2” to find Test Details.

We are also in heated debated among different teams of whether to use Test Repository or Test Sets as primary organisational unit - and also whether to use them exclusively or in combination.

So far, my approach is to use Test Repository to group tests where they “physically” belong - by components. For example, login test can only ever be part of a Login component, but can be part of a lot of different features - I put those in Test Sets.

On the other hand, that is a bit over-complication from normal JIRA flow which all of the developers are used to. Simple question by team member can be a hard one to answer: where can I see all tests and to what component it belongs? (they are basically asking for folder-like structure as was in TestRail or Cucumber Studio).

TL;DR: what are your tips and trick for organising tests in some logical units in Xray?

3 Likes
3

Thanks :slight_smile:
First, I don’t know exactly if you’re using Xray on Jira Cloud or on Jira server/datacenter; the links below will need to be adapted accordingly.
Some time ago we’ve written some docs related to the different possibilities of organization.

Some users prefer Test Sets (i.e. flat lists of tests), other prefer Test Repository (i.e. organize using folders).
The capabilities, as detailed on the previous links, are slightly different.
Test Repository scales better and it’s easier for you to do operations with Tests, because you can easily filter them out, select and then choose what to do (like scheduling a Test Execution for them).
Each project has implicitly one and only one Test Repository, therefore you have that birdseye perspective of all your tests in the scope of that project.

Simple question by team member can be a hard one to answer: where can I see all tests and to what component it belongs? (they are basically asking for folder-like structure as was in TestRail or Cucumber Studio).

Well, using the Test Repository, you can filter Tests by the component but since Xray is quite flexible, “by component” can mean different things. Are you assigning the “components” field explicitly on the Test issues? If so then you can easily filter by component field.
But you choose to semanticaly make this assignment by creating a folder, then you can simply use that folder to obtain the tests you want.
How you organize the tests is a bit up to you; many teams do this differently.
Using Test Sets and Test Repository at the same time can become a bit hard to manage.
One thing you could try out for example would be to use Test Repository and have the folders by some relevant semantic, and then use the Components field on the Test issue to identify the component
 and that is also visible on the Test Repository and can be used for filtering (like labels but labels are more dynamic).

https://docs.getxray.app/display/XRAYCLOUD/Test+Repository#TestRepository-FilteringvisibleTests

Hope this helps .
Sergio

1 Like
4

Fantastic answser, thank you a lot!

We use cloud JIRA. I have already read both the tips for organising tests and Set vs Repository pros and cons.

What was really enlightening, however, was the third link you provided - the advanced filtering in Test Repository. It is obviously much more powerful than I initially realised and will take that into consideration and discussion with other members.

As for components - yes, I am encouraging the use of components field and since Test Repository displays this out of the box, I am currently inclined to start using it more. Would leave Test Sets probably for stuff that span multiple components / features, e.g. smoke testing, acceptance testing, feature testing etc.

1 Like