Exploratory Testing with Inflectra SpiraTest

Exploratory testing using Inflectra Spira Test allows creating, executing and getting test results very fast. Exploratory testing allows to write and execute test cases simultaneously while we are testing the desired functionality as test cases are not created in advance. The Exploratory Test Case type, supported by Spira Test, supports documenting executed test steps, and also keeps the evidence of the test results.

 

Introduction to Exploratory Testing

 

Exploratory testing tries to find out how the software actually works. The quality of the testing is closely related to the tester’s skill of inventing test cases and finding defects. The more the tester knows about the product and all the different test methods, the better the testing will be. In this article we will discuss how Spira Test can be efficiently used in exploratory testing.

 

Exploratory Testing vs. Ad-Hoc Testing

 

Exploratory Testing vs. Ad-Hoc Testing

(For more detailed comparison of Exploratory Testing and Ad-Hoc Testing, please visit: http://www.softwaretestingclass.com/difference-between-adhoc-testing-and-exploratory-testing/)

 

Exploratory Testing Procedure

 

Exploratory Testing, developed by Cem Kaner, is structured approach that at a high-level can be separated in the following phases:

  • Product Exploration – 1st activity
  • Test Design – 2nd activity
  • Test Execution – 3rd activity

Within the Product Exploration and Test Design activities there is an important activity named as Heuristics that can be simply explained as “brain-storm on how I would accomplish the activity”.

 

Product Exploration

 

Product Exploration covers the following activities:

  • Identify the purpose of the product:
    • Review the product and determine what fundamental service it’s supposed to provide. Define the audience for the product.
  • Identify functions:
    • Function is anything the software is supposed to do. This includes anything that results in a display, changes internal or external data, or otherwise affects the environment. Functions often have sub-functions.
    • Two types of functions:
      • Primary Function: Any function which if it is not working from the user perspective will mean that the whole product is not working.
      • Contributing Function: Any function that contributes to the utility of the product, but is not a primary function.

 

Test Design

 

Test Design should result in:

  • High-level defined Test Cases that will contain:
    • Test Case name
    • Description of the Test Case (what is tested)
    • Test Data
  • Each function can be mapped to one or multiple Test Cases
  • As a rule of a thumb, depending on the available time:
    • 80% of the time should be spent on testing Primary Functions
    • 20% of the time should be spent on testing Contributing Functions

 

Test Execution – Session-Based Test Management

 

  • Session-based test management (SBTM) is an enhancement to Exploratory Testing
  • Developed by Cem Kaner and James Bach
  • SBTM incorporates planning, structuring, guiding and tracking the test effort
  • SBTM has its four elements:
    • Charter – a one- to three-sentence mission for a testing session, i.e. a high level plan that determines what should be tested
    • Time Box – each testing session lasts from 45min to 90min
    • Reviewable Result – as an outcome of the testing activities is a so called Session Sheet
    • Debriefing – conversation between the tester who performed the session and someone else—ideally a test lead or a test manager, but perhaps simply another tester

For more information on SBTM and Exploratory testing please visit: http://www.satisfice.com/

 

How Exploratory Testing can be practically implemented in Agile testing?

 

  • In Agile (Scrum) usually Specification (Requirements) based testing is implemented, where each User Story specifies particular requirements
    • Each User Story defines particular functionalities that need to be verified
    • Depending on the User Story it can be considered as a Product (with Primary and/or Contributing functions) or as a single Contributing function
  • All other aspects of the Exploratory testing approach, ideally should be supported by tools
    • For example, one of the problem is creating the Session Sheet and evidence of the executed tests, but with Spira Test tool, this is no longer a problem, because Spira is providing complete evidence of executed tests, including the name of the tester, the duration of the execution, detailed results with Pass, Fail, Block, Caution statuses etc…
    • Reports can also be exported in multiple formats like Microsoft Excel, Word etc…

 

How to create and execute Exploratory Test Case with Spira Test

 

For the example purpose, we will create Exploratory Test Case for Login functionality on http://www.libraryinformationsystem.org/.

Access the Spira Test tool, navigate to Testing > Test Cases tab and click on New Test Case button. Under “Type” drop-down for the particular test case, select “Exploratory” and enter test case “Name”.

 

Create Exploratory Test Case in Spira Test

Figure 1: Create Exploratory Test Case in Spira Test

 

Once the test case is created, click on the test case name. The test case is opened in a new page, where we can enter description and the test steps. In the Description section, we should write a short description on what is the purpose of our test. After that we can see that one dummy test step is already present and we can start the test execution.

 

Adding description to the Test Case

Figure 2: Adding description to the Test Case

 

After clicking the Execute button on the top toolbar, the test execution form is displayed that allows us to write test steps while executing it.

 

Exploratory Test Case execution form

Figure 3: Exploratory Test Case execution form

 

We will start our exploratory testing by automatically editing the created dummy test step. In our simple test, Step 1 would be to access the http://www.libraryinformationsystem.org/ page and verify that the login page is displayed. As it was mentioned previously, the exploratory test execution form in Spira Test enables editing/writing test steps while executing the test.

 

Write/Edit test steps in Exploratory Test execution form

Figure 4: Write/Edit test steps in Exploratory Test execution form

 

Once we have the first step written, we are executing the actual test step, with navigating to http://www.libraryinformationsystem.org/ page and verifying that the login page is displayed.

 

Executing the first step in our test - Navigate to login page

Figure 5: Executing the first step in our test – Navigate to login page

 

The login page is displayed, so our test step passes. One great feature which comes with Spira Test tool is the ability to put screenshots in the Actual Results field. Putting a screenshot in the Actual Result field is quite easy and fast, we just need to get a screenshot (Alt + PrtSc) and then return to our Spira Test execution form, click on Actual Result field and paste (Ctrl + V) the screenshot. For the Step 1 we will put this screenshot and pass the step.

 

Screenshot as Actual Result in Exploratory Test Case execution form

 

Figure 6: Screenshot as Actual Result in Exploratory Test Case execution form

 

Now we can continue with writing and executing the second step. By clicking on the sign (+) below the first step, a new empty test step is created, enabling us to populate fields in the form.

 

Adding new Test step in Exploratory Test Case execution form

Figure 7: Adding new Test step in Exploratory Test Case execution form

 

The second step would be, entering invalid credentials, with expecting a warning message to be displayed. Once we write the test case, we are going to execute the actual step.

 

Executing the second step in our test - Login with invalid credentials

Figure 8: Executing the second step in our test – Login with invalid credentials

 

Once again, the current test step is passing, so we are pasting the screenshot of the warning message in the Actual Results and click on the Pass button.

Now we can create (by clicking on + button),  write and execute the third and final step of our scenario, where we are entering valid credentials, the login is successful and the home page is displayed. Assuming that while executing the step, the login was successful, we can get a screenshot of the home page, paste it into Actual Result, and pass the step by clicking Pass button. Since this was the last step in our exploratory test, we can now finish our test.

 

Finishing Exploratory Test Case

Figure 9: Finishing Exploratory Test Case

 

Finishing Exploratory Test Case can be done by clicking on the orange button with stop symbol. Clicking this button will give you two options. “Update Test Case” will update the test case execution status, and also change its name, description, and test steps to reflect those on this page (adding, deleting, moving, editing as necessary). “Just Finish” will only change the execution status of the test case only—leaving all details of the test case unchanged. Either option will archive the entire test run (so it can no longer be amended) and the page will automatically exit the test execution page. Because we want to save the created test steps, as well as, the testing artifacts (screenshots) in Actual Result we will choose “Update Test Case” and after clicking it we will be navigated to Test Case list, showing our Test Case as Pass.

 

Test Case list with our Exploratory Test Case

Figure 10: Test Case list with our Exploratory Test Case

 

Conclusion

 

Spira Test supports Exploratory testing methodology from all its aspects, and resolves one of the most important requirements of the Exploratory testing: creating a testing evidence of the executed tests. Exploratory Test Cases, by its features for adding/removing test steps, writing/editing test steps while executing the test and keeping testing results from each test step, enables the tester to implement Exploratory testing in an easy and quick manner, focusing only on exploring and investigating the system under test.