/Opinion/ Testing ecommerce sites
02/02/2009 | Filed under Discover > Opinion
Effective performance testing is vital if you want to provide your visitors with a smooth online shopping experience, says Andy Buchanan
Online retail is increasingly big business. The latest IMRG Capgemini e-Retail Sales Index saw UK sales reach a record £15.2billion last Christmas, up 50 per cent on the previous year. In this increasingly competitive environment, websites need to provide users with the best possible visitor experience, one that will keep them coming back for more. But how can etailers achieve this, while coping with surges in online traffic?
Although a lot of time and consideration is usually spent on preparing the creative side and content aspects of websites, functionality and web application testing is often neglected. This can be an oversight for which a company’s reputation may pay dearly. According to the research group Gartner, bugs found after release can not only spell doom for a project, but also cost 80 to 1,000 times more to fix than if they were found during pre- deployment testing.
Usually the best way to ensure web applications meet the demands of visitors, while meeting internal deadlines, is to keep testing simple. Here’s an analogy: in order to check the plumbing of a major sports stadium in the US on the eve of opening day, the entire workforce got together and flushed the toilets simultaneously. This simple, labour intensive approach is often used to load-test new web applications, with lots of people asked to log in at the same time. However, not only is this approach logistically challenging, it’s often impossible to realistically simulate the number of users who might hit the site.
This is where Rapid Bottleneck Identification (RBI) comes in. Every web application has at least one bottleneck – an element of hardware, software or bandwidth that places defining limits on data flow or processing speed. In addition, applications are only as efficient as their least efficient elements, and it’s here that bottlenecks directly impact on performance and scalability.
While the best testing should be quick and simple, it needs to be thorough, which is why RBI is a great testing methodology for web developers.
Back to basics
Performance testing often begins with scenarios that are overly complicated: exercising too many components makes it easy for bottlenecks to hide. By starting with basic system-level testing you can check the web application’s performance before it’s even deployed.
Furthermore, applying a modular approach helps simplify things. You start by testing the simplest possible test case and gradually build in complexity. If this works, testing moves on. If the next stage fails, you can identify where your bottleneck is.
What’s useful about the modular method is that after uncovering each bottleneck, you’ve already ruled out previously tested components. For example, if you hit the homepage and encounter no problems, but then decide to execute a search, which shows poor performance, you’ve established that the cause of the bottleneck is the search functionality.
All performance testing should begin with assessing the basic network infrastructure supporting the web applications. If this cannot uphold anticipated user load, then even an infinitely scalable application will bottleneck.
Next, it’s time to focus on web applications. The approach, again, should be to start with the simplest possible test case and then to add complexity. For typical ecommerce applications this means testing the homepage first, then adding pages and business functions until complete transactions are being tested, first individually and then in complex scenario usage patterns.
Once this is complete, transactions can be put into scenario concurrency tests. These must reflect real user behaviour (for example, 50 per cent just browse, 35 per cent search, 10 per cent register and log in and five per cent add to the shopping cart and make a purchase). But the virtual users who are testing the site must execute the steps of those transactions using the same pace and behaviour as real-world visitors.
Testing is a bit like insurance – usually you don’t realise it’s not good enough until it’s too late. So, whether or not you conduct performance testing in-house or via a managed service, it’s important that it’s done methodically and rigorously.
Andy Buchanan is European director of the Web Business Unit at Empirix, a leading provider of web application testing that works with the likes of Comic Relief and John Lewis.
Bookmark with:
Comments
Alistair Campbell / 24/02/2009 / 14:08 / http://www.web-energy.co.uk
Andy, I agree with the sentiment of your article and the need for diligence and robustness in both the planning and execution of e-commerce websites. My difficulty is that the majority of e-commerce sites are in the SME marketplace and working within budgets of the £10k - £20k region. Which doesn't allow much time or budget for the depth of testing you describe.
As an e-commerce web designer, we always do at least wireframe testing with prospective customers at a pre-build stage to ensure that we are communicating the key USPs, and confirmed that the likely user experience will get them in to the product catalogue and hopefully engaged with the purchasing process. After that, it is very much about looking at the sales conversion ratio (number of transactions in a given period / number of site visitors in a given period) to gauge the effeciency of the site (4% on average being acceptable?) and also looking at other metrics such as Google Analytics to get a feel for how the site content and functionality is performing.
At the end of the day, it's got to be the right product for the customer at the right price and no e-commerce site will succeed with that being in place. Cheers.
James / 16/11/2009 / 20:49 / http://www.theimmediagroup.co.uk
Testing is a bit of a bugbear at the lower end of the market. And let's face it - that's where most business owners are. They neither have the time, money, nor the patience for proper testing. It's hard to change the mindset that a website doesn't have to be tweaked, changed, modified, added to throughout its lifecycle - it's basically finished when signed off!
I think the rise and rise of the e-commerce professional, though, will in the end annihilate the smaller fry in the less popular niches, and pretty soon most online retailers will be delving into their pockets to pay the monies required for a premium ecommerce site.
Until then, it's basic testing and tweaks!
MageWorx / 11/12/2009 / 00:47 / http://www.mageworx.com
Andy,
Thanks for the article. I have been working as quality assurance specialist for 4 years. We performed the Load / Stress tests and performance ones. We had different types of scripts which emulates the behaviour of real users (browsing the pages, perform some actions etc). All our projects were tested using this.
As for the ecommerce perfomance, now we use Magento platform for our business. But frankly speaking it's so slow and requries rather good server configuration.
Have you used Magento or tested it regarding performance matter?
Thanks
Kate Sudarkina / 05/02/2010 / 10:54 / http://www.ecommercesupermarket.co.uk/
Testing is crucial to e-commerce because e-commerce sites are both business critical and highly visible to their users; any failure can be immediately expensive in terms of lost revenue and even more expensive in the longer term if disaffected users seek alternative sites. Yet the time pressures in the e-commerce world militate against the thorough testing usually associated with business criticality, so a new approach is needed to enable testing to be integrated into the development process and to ensure that testing does not present a significant time burden.
The very familiarity of much of the technology means that tried and true mechanisms will either be suitable or can be modified to fit. Rapid Applications Development (RAD), in particular, suggests some promising approaches. Like most new ventures, though, e-commerce must find its own way and establish its own methods. In this paper we have suggested some testing principles that have stood the test of time and intermingled them with some lessons learned from similarly challenging development environments to give e-commerce testers a staring point for their journey of discovery.
