Comparison of JIRA Test Management Plugins

Introduction

 

This article gives an overview of the features as well as price comparison for some of the most used JIRA test management plugins. Comparison is made for JIRA server plug-in versions, and it is based on documentation provided for each of the plugins. The following plugins are compared:

 

Features Comparison

 

The following table gives features comparison of the plugins in several categories:

CategoryFeatureXrayZephyrQmetryAdaptavist
Project configurationIntegrates with existing JIRA projectYesYesYesYes
Supports separate JIRA testing projectYesNoNoNo
DesignCreate, edit, delete test casesYesYesYesYes
Clone test casesYesYesYesYes
Clone/Copy test steps within test caseYesYesYesYes
Reuse/Copy test steps between test casesYesYesNoYes
Versioning/History of test casesYesNoNoYes
Comments on test casesYesYesYesYes
Restoring previous version of test caseYesNoNoYes
Hierarchical/Folder structure for test casesYesNo[1]NoYes
PlanSupport for Test PlansYesYesYesYes
ExecuteCreation of test executionsYesYesYesYes
History of test executionsYesYesYesYes
Support for Test EnvironmentsYesNoYesYes
Automated Time Tracking for test executionNoNoNoYes
TraceabilityLinking User Stories with test casesYesYesYesYes
Linking test cases with bugsYesYesYesYes
Built-in User Story – test case – bug relationship reportYesYesYesYes
Integration of test artifacts in JIRA Agile boardYesYesNoNo
Automation[2]Native Support for BDD (Gherkin/Cucumber)YesNoNoNo
Mark Test Case as Automated testYesNoNoNo
Create Test Cases from automated tests resultsYesNoNoNo
Import test results in Jira from automated test runsYes

Cucumber
Behave
JUnit
TestNG
Nunit
Robot
NoYes

Cucumber
TestNG
JUnit
SpecFlow
No
Built-in REST APIYesNoYesYes
CI/CD integrationJenkins add-onYesNoYesNo
Bamboo add-onYesNoYesNo
CustomizationCustomize Test Execution statusYesYesYesNo
Customize Test Step Execution statusYesYesNoNo
Customize Test Case workflowYesYes?[3]NoNo
ReportingNumber of built-in reports5474
Number of built-in gadgets68120+
PlatformJIRA Cloud or JIRA serverCloud
Server
Cloud
Server
Cloud
Server
Cloud
Server

[1] Folder structure is supported only for TestCycles

[2] Automation and CI/CD integration features for Zephyr are supported by separate JIRA plugin ZAPI that is paid separately from Zephyr. More info on ZAPI: https://marketplace.atlassian.com/apps/1211674/zapi

[3] It is not well documented

 

Price Comparison

 

The following table gives price comparison of plugins for JIRA Server:

UsersXrayZephyrQmetryAdaptavist
10$10$10$10$10
25$500$750$600$700
50$1,000$1,500$1,200$1,400
100$2,000$3,000$2,400$2,800
250$4,000$6,000$4,800$5,600

For more info on prices and conditions please see the following links:

 

Conclusion

 

From the perspective of manual testing all of the compared plugins provide similar features. However, from the perspective of support automated testing and CI, Xray provides built-in support of more testing reporting frameworks, including native support of Cucumber. Taking in consideration also the price for each of the plugins, it looks that Xray should be the choice for a test management solution integrated with JIRA.