Problem – What is exploratory testing, what makes it different from functional testing and what kind of issues does it pick up?
Solution – We define exploratory testing, specifically highlighting the difference between exploratory and functional. Additionally, we walk through 3 real bugs found during an exploratory test, all identified on industry different sites from e-commerce to financial.
Read Time – 5 Minutes
Exploratory testing is a type of functional testing. However, instead of having expert testers follow specific or scripted journeys, exploratory testing goes beyond what can be labelled as a ‘normal’ journey. Exploratory testing searches for issues that are inline with natural user flows, these are typically outside of the scripted journeys.
A technical definition of exploratory testing is:
After just under 10 years in the website testing industry, Digivante has come across hundreds of odd and unusual bugs, all found through exploratory testing, on a range of different sites and devices from every possible industry.
Here are three of the weirdest bugs found by the Digivante community to date:
1. Now you see me: One of the most commonly identified, yet unusual, bugs come in the form of cut-off or limited menu scroll for slide-in burger menus on mobile.
Here’s an example of how a slide-in burger menu should look and perform on mobile, however, our testers frequently find bugs within main menus which restrict the scrolling ability of a slide-in menu on mobile devices. This can cut off certain menu options and textual links.
Exploratory testing picked this issue up as it was deviant from the core journeys outlined during a scripted test. Issues like this can occur when a site hasn’t been fully optimised for mobile devices.
2. Not your favourite: While shopping online for a special dinner or choosing an outfit for date night, you’re likely to add your choices to your ‘favourites’ to be picked up at a later date. As useful as the ‘add to favourites’ functionality is, it’s often littered with bugs. One bug that was spotted by one of our exploratory testers resulted in a product no longer being able to be added to the ‘favourites’ section after deletion.
In this case, a tester added a dress to their favourites cart on an e-commerce site after logging into their user profile on the site. The tester then removed this dress from their favourites and proceeded to try to re-add the item to their favourites cart. However, the site did not allow the item to be added to the cart again after deletion and the tester was faced with an empty favourites page.
Even after reloading the page, logging out and logging back into their profile choosing a different size and closing down the browser, the tester still could not add this product to their favourites.
Only exploratory testing could have identified this issue as it was outside the initial functional testing script. Even though an issue such as this wouldn’t be regarded as high-priority, it shows how exploratory testing identifies issues which scripted functional testing wouldn’t be able to locate. Read the Website Testing 2020 guide.
3. At the last hurdle: Payment is the final step for most transactional sites. It’s where users sign on the dotted line and all of the work gone into encouraging these users to convert shows its worth. However, for one of Digivante’s previous clients working within the jewellery industry, our testers found a bug which wouldn’t allow users to complete payment using Paypal if they proceeded to the checkout as a guest.
This specific bug only occurred on specific device and browser combinations across iOS and Android, meaning this bug was harder to pick up by the company's internal testing teams as they had limited time available to test and also lacked the resources required to conduct a full functional and exploratory test.
Exploratory testing is just one part of functional testing and should be approached with the same amount of urgency as a core functional or regression test. Exploratory testing mimics actual human interactions on your site, meaning it picks up all the variations that automated or scripted testing simply can’t identify.