Test Management tool for JIRA - Xray



In an attempt to use JIRA as the test management tool with BDD  using add-ons. Here is a comparison that might help:

Exploring JIRA plugin test management tools
Xray for JIRA
Zephyr for JIRA
Behave PRO for JIRA
Cheaper
Expensive
Expensive
Supports manual test running
Not in BDD format
Supports manual test running
Not in BDD format
Does not support manual test running
Supports writing test cases in Gherkin with auto text formatting
Does not support Gherkin language for test cases
Supports writing test cases in Gherkin with auto text formatting
JIRA REST API to automatically import/export feature files and import/export automation results
ZAPI add-on for RESTful API to JIRA
Supports automated cucumber/specflow/behat test cases, exported to feature files
Supports automated cucumber test cases, exported to a feature file and results imported back into JIRA
Plan and execute test cycles with tracking
Does not support requirements and defect linking isn't very mature
Separate requirements and defects, can associate either or both to test cases to provide test coverage
Mature tracking of test quality metrics for every project, more focused on cycles and executions
No test sets to group test cases
Test sets to group test cases
Attachment of defects to test or test steps
Add multiple scenarios to individual user stories as Acceptance tests
Supports pre-conditions for test cases
Link JIRA issues to steps/stories
Ability to import cucumber features
Export to different formats/templates

Link JIRA issues to scenarios
Link JIRA issues to steps/scenarios

Maven integration for automated tests

Decided to go with Xray since it was more aligned with the BDD theme we had. Here are a few YouTube videos that might help:

Xray YouTube videos:

Xporter YouTube videos:


More about Xray:

XRAY documentation:

Here are some advantages that the team from Xray claims to have over Zephyr:
  • Automated Tests: Xray supports automated Tests (Cucumber and Other) to be defined in JIRA and provides, in the same add-on, REST services for exporting Test definitions and importing execution results to JIRA.
  • Editing and visualising manual Tests and steps is much more user friendly than other competitors. Xray supports the JIRA wiki notation in several fields like Test Steps, Pre-Conditions and execution comments. It also allows you to include attachments in Test step definitions.
  • Xray for JIRA is aware of two common concepts in test management: requirements and defects. It can generate Requirement Coverage charts based on the Requirement - Test Issue links.
  • Xray integrates seamlessly with JIRA. It takes most advantage of JIRA entities like Issue Types, Custom Fields, Issue Link Types and Screens to provide an intuitive and easy to use interface. 
  • Xray for JIRA has the concept of test sets used to group test cases. 
  • Xray for JIRA supports definition of pre-conditions for test cases. 
  • In Xray for JIRA, changing a test definition does not affect past executions. Past executions remain with the initial test definition. 
  • Xray provides integration with Xporter to export test, test sets and test executions to DOCX and PDF reports, based on customizable templates. 
  • Price
  • REST API included
Happy Testing!

Comments

  1. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. How to configure selenium with xray or zephyr? Does it support any?

      Delete
    2. Hi Asha, we are using Zephyr at our work and you can write your feature files and then export the results back into xray once the tests are complete. Let me know if you need details

      Delete
    3. This comment has been removed by the author.

      Delete
    4. Is it possible to configure CI Tools like Jenkins or Automation Tools like Selenium with Xray? I know it's possible with Zephyr using Zapi but not sure bout Xray.

      Delete
    5. Hi Abdul, You can integrate any BDD tool like Cucumber/Specflow which will then call your underlying tool for automation. This video might give you an idea: https://www.youtube.com/watch?v=kigz-xF2-9g

      Delete
    6. Thanks alot for your reply,

      Delete
    7. Krupali6/2/17

      Do you have any sample that helps in integrating other automation tools like selenium etc??

      This will help in deciding if to go with this tool

      Delete
  2. I am evaluating Test management tool (a Jira Add-On) which can used to store Requirements, Manual Test Cases as well as Cucumber feature file for automation.

    I have evaluated Xray and it looks promising but i want to play with more such tools before i show a presentation to client. Zephyr, TestRail and HipTest are others in the list to be evaluated. Can you suggest a good tool which i can dig into.

    Thanks.

    ReplyDelete
    Replies
    1. If you need to compare Xray and Zephyr, you may want to give a look at http://www.xpand-addons.com/xray/xrayvszephyr/

      Delete
  3. Anonymous11/7/16

    Whats the exact difference between JIRA Zephyr and JIRA XRay?

    ReplyDelete
  4. Hey we have written our features files within behave pro for jira ,can we import those features into x-ray??please suggest

    ReplyDelete
  5. Hi, Xray supports both cucumber and behave syntax.

    ReplyDelete
    Replies
    1. That's right we have integrated it with both spec-flow and Cucumber.

      Delete
    2. That's right. We have integrated it with both Spec-flow and Cucumber.

      Delete
  6. Does Xray support integrating with JIRA Cloud instance?

    ReplyDelete
    Replies
    1. AFAIK it isn't there yet but the next release has plans: http://confluence.xpand-addons.com/dosearchsite.action?queryString=jira+cloud

      Delete
  7. they don't have Cloud yet but maybe this year. Also they have a new release, version 2.1: http://confluence.xpand-addons.com/display/XRAY/Xray+2.1.0+Release+Notes

    ReplyDelete
    Replies
    1. the website: http://www.xpand-addons.com/

      Delete
  8. Anonymous28/8/17

    Does xray supports behat, its integration and reports?

    ReplyDelete

Post a Comment

Popular posts from this blog

XPATH for IE / internet explorer

All time good software testing books!

Website's underlying technology for automation