Defining, developing, launching and testing new functionality on websites and apps should be a central activity of every e-commerce team. After all, the continual improvement of customer experience to drive sales and conversion is the entire reason for being for a digital, retail brand.
But how can you ensure the quality and velocity of new website and app releases keep pace with each other?
What is new functionality testing?
New functionality testing is the process of ensuring that any release of website or app code which power new features, e-commerce journeys, APIs and payment gateways are all working exactly as they should across the range of devices that are likely to be accessing them.
New functionality testing protects your revenue and reputation by picking up on usability issues and bugs in new features as early as possible in the release cycle.
It should include targeted functional, usability and regression testing to reduce the risk of damaging bugs being introduced into a website or app that will be even more costly to identify and remove later on. Read the Website Testing 2020 guide.
Why do new functionality testing?
E-commerce software release cycles are becoming increasingly more frequent as companies seek to steal competitive advantage, convert more visits into sales and set new high standards in digital customer experience.
Adding new features is vital to grow market share, increase customer dwell time, loyalty, basket size and spend in the hyper-competitive e-tail sector.
The pressure to perform
You may not, like Amazon does, be deploying updates at the rate of 1 release every second, but like the online giant your team will be focused on responding to the needs of your customers through a constant stream of improvements that continually add value to your offering.
As Jeff Bezos points out, this kind of approach is an internal imperative to innovate and release more, that creates the natural momentum for growth and success:
“A customer-driven focus… aids a certain type of proactivity. We don’t wait for external pressures. We are internally driven to improve our services, adding benefits and features, before we have to!”
Jeff Bezos, Amazon
But of course, this innovation needs to be matched by a commitment to quality, because customers are highly sensitive to change and inconvenience - and are not afraid to share their experiences online and elsewhere:
“If there’s one reason we have done better than of our peers in the Internet space over the last six years, it is because we have been focused like a laser on customer experience, and that really does matter, I think, in any business. It certainly matters online, where word-of-mouth is so very, very powerful.”
Jeff Bezos, Amazon
New functionality testing should first and foremost ensure that the newly released features are not buggy and causing system crashes and slowdowns. Secondly, it should help you understand, very quickly, whether the features you are releasing are bringing the intended value to your customers. If not you need to decide whether they can be optimised for better performance - or, if they need to be killed off altogether.
In an agile world, new functionality testing should help you deliver higher quality releases more quickly, but also give you the data and insight you need to ‘fail fast’ ensuring you are investing time and effort in the right places.
How to do new functionality testing
New functionality testing should be conducted pre-release in staging environments that mirror live as closely as possible. This should include:
- Scripted functional and usability testing to confirm the new features work properly and as intended - ensuring they achieve their objectives and do not contain any bugs, issues or glaring UX problems.
- Targeted manual and automated regression testing in these environments will also reassure brands that the new features will not cause issues elsewhere in the website or app.
- In the hours following a new release, these tests should be repeated in live, to ensure that the deployment of code has not introduced any new bugs into the system. This is a moment of particular risk, because bugs are often introduced when a website’s core code hasn’t been updated to be compliant with new functionality or add-ons.
- In the weeks following launch, exploratory testers should continue to work to understand how the new functionality is operating in the live environment and how it can be optimised to increase effectiveness.
In a fragmented digital world, where users are spread across an almost infinite array of device types, OS and browser combinations, testing on a range of real-world hardware and software is essential. Issues are often specific to certain browser and hardware combos that cannot be predicted or understood except when they are tested in the real world against real user scenarios.
Results from these tests need to be fed back into the business, but in a way that is accessible to commercial and technical teams alike. The new functionality testing tools adopted by an e-commerce business should use video clips, imagery and clear language to describe bugs and usability issues, so that fixes can be prioritised and the team as a whole can decide what next steps for optimisation.
Who should do new functionality testing?
Should your developers be doing their own testing? In an agile context, of course, quality is deemed to everyone’s responsibility. But the fact is, your dev team will likely be too close to the code and the experiences they have created to be able to root out issues and critique it properly.
At the same time, an internal QA team often will simply not have the resource, time or hardware to cover the full range of required regression, usability and functional tests. A QA team need to spearhead testing initiatives, but they need the resources to complete these tests as quickly and exhaustively as possible.
The challenge of new functionality in the agile world is keeping pace with new release schedules and the attendant regression and usability testing they require in a changing digital landscape.
Only a real world testing solution that can be mobilised quickly and followed up with a range of ongoing, exploratory tests post-release, can help you understand the impact new features are having on your revenue and where you should take them next.