Reputation and revenue depends on friction-less, error-free, e-commerce experiences. And new website or app deployments can be make or break moments for customer trust and future brand loyalty. No matter how thorough your testing, putting an application live for the first time, or releasing updates and new functionality always represents risk.
One of those risks is that a simple configuration mistake made in the release process could introduce a bug that wreaks havoc with your customer experience and conversion rate - and that you don’t notice. Which is the whole point of post-implementation testing - and why it is so vital for a stress-free website or app release. Read our Grow Your Website Conversion Rate ebook.
So, what is post-implementation testing?
Post-implementation testing - or deployment testing - is a concentrated period of testing that follows the deployment of a website or app into the live environment. In this process a variety of compatibility and exploratory checks attempt to identify any issues in the live environment which may have been obscured in the test environment or otherwise missed.
But shouldn’t pre-release testing have covered all this?
Many brands undertake a deployment process in a test environment that exactly mirrors live, as a way of satisfying themselves that a release will be safe to deploy to the live environment. And, of course, it is absolutely essential to do this. But most errors found after the release of a site emerge as a result of deployment itself - issues with configuration settings, database connectivity, network configuration and the like. You need to test for these as part of your standard release process.
Welcome to the jungle
It has also been said that comparing a test and production environment with each other is like comparing a zoo to a jungle. They may have similarities but there are important differences.
The fact is, no matter how closely the environment you tested in has mirrored production, there are always going to be things that happen ‘in the wild’ that you cannot predict or simulate effectively before release. Take, for example, one e-commerce website experience of post-release reality:
“In practice, unexpected things kept happening in deployment. Users sometimes did unexpected things, like launching a flash sale that inundated the site with a huge number of users at once instant. Or there could be some unexpected interaction between the actual hardware and the software."
From The Age of Agile, by Stephen Denning
The lesson is, you need to be confident in the hour’s post-release that you are testing for configuration issues, or bugs that could not be replicated outside of live.
When updates and releases are coming thick and fast you need confidence that your conversion rate is not going to be suddenly and dramatically impacted by some previously unseen or introduced defect - and that you can quickly determine the cause of them when they’re noticed.
Spending time and resource checking for post-release problems is more than worth it, and leaving them un-rectified will cause the costs of fixing them in the future, to balloon:
“As bugs age in the system, the cost can increase as the developers have to reacquaint themselves with the code to fix the bug or as dependencies to the code in the area surrounding the bug introduce additional complexity and risk
to the fix.”
Alan Page author of How We Test Software at Microsoft
Is there room to fail?
In a world where customers are accessing your websites and apps 24/7, as well as active and all too eager to comment on social media about their experiences, for many brands there’s no such thing as a low profile failure.
Of course, brands should continue to shift left with testing at every stage of the pipeline enabling faster feedback from testing that informs our development. But testing in production represents another vital quality guard around your application.
Thorough post implementation testing ensures new functionality on your website or app is successfully deployed to the live environment as seamlessly as possible with the least possible impact on your customers.
In this way new functionality can immediately begin delivering a better customer experience, minimising the risk of bug-driven reductions in revenue and costly, time consuming investigations further down the line:
- Intensive post-launch exploratory testing protects revenue and reputation
- Uncovers compatibility and other errors quickly before customers do
- Reduces impact on conversion and costly investigations later on