As a testing company that relies entirely on the brilliance of humans, it may surprise you to hear us say this, but we will anyway: human beings are not naturally suited to software testing.
Scrutinising a website that looks almost identical on lots of different screens and browsers is not something that nature has equipped us to do particularly effectively. But given that software testing is a very young profession, however; perhaps one day we’ll evolve.
Until that day though, you face the problem of needing your sites and apps to work on as many different combinations of device, browser and operating system as possible. In order to explore how best to achieve that, it’s a good idea to think about why humans aren’t that great at software testing, and what can be done to get around our shortcomings.
Where's the Proof?
The main reason for those shortcomings is amply demonstrated by a now-famous psychology experiment conducted at Harvard University some years ago.
Christopher Chabris and Daniel Simons, the 2 academics that performed the experiment, made a video of a group of people passing a ball around and then screened the video to an audience. The audience were asked to count how many times the ball changed hands during the video, and 50% were so engrossed in their task that they completely failed to notice a person in a gorilla suit walk into the middle of the ballplayers, dance around a bit, and then walk off.
Chabris and Simons characterised this strange phenomenon as ‘selective attention’, explaining that when we are concentrating intently on one thing, we can miss something else that would ordinarily be blindingly obvious.
This is why we aren’t naturally adept at software testing: when we view the same site over and over again on 10, 20, 30 different devices or web browsers, we begin to concentrate only on what we expect to see from previous experience, missing critical bugs in the process. The phenomenon is being increasingly recognised within the testing world, and is often referred to as ‘browser blindness’.
Given that selective attention affects all of us, even the most skilled, experienced testers are fallible. How, then, is it possible to ensure effective testing of your site?
The answer is by minimising the amount of testing that testers have to do. We may be ill-suited to spotting new bugs after having seen the same defects on 30 different browsers, but skilled testers who aren’t expecting to see certain bugs are highly effective.
The solution to the problem of browser blindness, then, is to use a large tester base to reduce the amount of testing done by each individual. Community-sourced testing offers this solution, drawing in the talents of thousands of professional software testers from across the world.
In a community-sourced testing scenario, in which hundreds of testers compete to find the most valuable bugs on a single website or app, no one tester has the opportunity to test on more than 1 or 2 device combinations. As a result, the testing that is done is extremely effective and the negative impact of browser blindness is avoided.
We may not be that great at software testing on multiple devices and browsers, but the recent innovation of crowdsourced testing means that we don’t have to be. It is now possible to have your site tested effectively and feel confident in releasing it into the live environment.