Exploratory testing is defined as “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize… by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project” (Kaner, PhD., 2008). Originally developed for Agile teams, the activities and mindset conducted during an exploratory test match its adaptive style.
The opposite of exploratory testing is scripted testing, the default activity for software testers. These tests are derived from requirements and documentation in order to enumerate every detailed pathway in the application beforehand. There are many drawbacks to scripted testing which exploratory testing can solve.
Pros and Cons of Exploratory Testing
Pros: The main advantage of exploratory testing is that it requires little preparation. When experienced testers perform exploratory testing, it allows them to do what they do best, test an application naturally. Exploratory testing can be crucial for conversion rate optimization, a practice for increasing customer interaction on a website. Exploratory testing uncovers all the issues scripted testing simply cannot uncover.
Cons: Due to the freestyle nature of test activity, there are several disadvantages of this approach. It requires a capability for self-direction and time management for a single tester. Since everyone has a personal bias, there may be some types of issues that remain undiscovered. Mitigate these drawbacks by following the recommendations below.
When performing exploratory testing, consider the following tips:
- Use experienced testers to conduct the test
- Time box activities to avoid wandering
- Pair up testers together to increase the chances of discovery
- Consider all the requirements, scope, and purpose of the application
- Understand the underlying technologies and their programming pitfalls; e.g. is the application a website, API, or desktop program?
- Always consider ways of using the application in an unintended manner
- During testing, record the session or keep a document open to paste commands and screenshots into for reporting
If enabling testers to use the formal exploratory approach all the time is not feasible, consider a hybrid approach. This arrangement can emphasize scripted testing after a short period of exploratory testing.
On Agile teams, this can look like the following:
- Scripted test cases are designed at the start of the sprint for unit and functional tests.
- This feeds into the exploratory testing because it informs the tester of functional areas
- Before starting scripted tests, testers explore the system freely and submit any issues
- Some areas are checked off as the tester progresses through execution
- Focused automated tests are created around the issues
- Priority areas to automate can be sorted using information from previous steps
- The rest of the scripted tests are executed if time allows
- Not necessary to execute or automate every test case, but it is recommended to track them
Examples of Exploratory Testing
Many enterprise-level teams found great success by inviting other employees outside the group to review their application in an event called a bug bash. Before releasing their software to the public, the development team hosts an internal demo and asks others to explore their application as if they were customers. This type of exploratory testing gathers valuable feedback about customer experience and conversions. Releasing software to a small audience of customers outside the company, or beta testing, is another example of exploratory testing. The latest evolution of these practices is using the power of the worldwide web to crowd-test on-demand or 24/7.
Case Study for Community Testing
Putting an open bounty on issues or enrolling in a crowd-sourced/community-sourced testing program represents an exciting new development for digital software teams. These programs leverage the power of a large, diverse pool of exploratory testers. They are motivated experts available to perform many different kinds of tests. Each tester represents a fresh perspective. Consider this case study where “in just 3 days Digivante’s 449 testers performed 183.9 working days worth of work and uncovered 432 issues.” In software quality, the more feedback an application receives, the better. The bottom line is greater “confidence and security in the product” for the client’s customers.
Written by one of our expert community testers, Christopher Canova. Christopher is an accomplished IT professional, with over a decade of experience as an enterprise software development engineer in test (SDET) in the Pacific Northwest. He also enjoys crowd-testing, hunting bug bounties online as a security researcher, or blogging about these topics.