#performance testing approach

The stiff competition in the online environment has made the customers of today choosy, demanding and vulnerable at the same time. Business enterprises, with the purported aim of extending their market footprint, are launching multiple products for different user groups (based on age, gender, geography, and purchasing power). The product launches consider user preferences, predictions, and analytics to drive better customization and personalization. The rapid pace of product launches by businesses is driven by the objectives of retaining the existing customer base, adding new ones, and staying competitive.

The products should possess certain characteristics – available 24 x 7, scalable to accommodate any future growth, reliable, and intuitive. Moreover, high speed has become the new normal with customers expecting their software products to respond in milliseconds. If these were not enough, the advent of new technologies or methodologies such as DevOps, Agile, SMAC, VPN, IoT, and Big Data among others have created new challenges for enterprises to integrate them in their workflows. The challenges are mostly in terms of achieving a seamless performance of the application across digital touchpoints. The realisation has dawned upon enterprises about executing performance testing (PT) beyond the realm of load and stress testing. Let us understand how businesses can bring out the best in their performance testing strategy in an Agile environment.

#1. Performance to be the focus area: In the Agile-DevOps development and testing environment, testers should be tasked with executing a range of performance testing services. This can only be ensured should performance be included in the Service Level Agreements (SLAs). In other words, an application cannot reach the delivery stage unless the SLA parameters pertaining to performance are met. Moreover, a detailed test strategy should determine the test types that are needed to make the application future proof, scalable, robust, and dynamic. The test strategy should be in accordance with the real-time environment.

#2. Focus on performance engineering: Since the performance of an application encompasses non-functional requirements such as throughput, memory usage or latency, the testing team should make performance engineering (PE) its focus area, preferably at the beginning of the design phase. The test team should stay ahead of the curve by engaging with the developers. This can be done by being a part of scrum like meetings where updates related to development are planned and discussed. The test team should analyze if the updates or changes need new load or stress tests, for the team can deliver a positive outcome should it deal with such changes in a proactive manner. Moreover, the test team should build an Agile-based performance engineering framework to cater to the ever-changing dynamics of technology and market.

#3. Component-based testing: Since performance related issues can derail the functioning of an application not to speak of hitting the end user experience negatively, the test team should focus on individual components post finding the errors. The performance testing strategy should aim at isolating the erring component and dealing with it rather than looking at the overall module. AI-based automation can play an important role in driving this strategy, for the test script can zero in on the exact component causing the error or deviation of the outcome from expected lines. In the ultimate analysis, such an approach can save the organization money and effort.

#4. Optimization of technology and infrastructure: Application performance testing deals with issues such as scalability or performance of a software application when the latter undergoes integration (with legacy systems or cloud) or upgrades. Since the emphasis is more towards achieving cost rationalization, the performance engineering approach looks at embracing cloud and virtualized environments. This leads to the provisioning of on-demand resources and sharing of hardware systems. Also, in view of the popularity of Web 2.0, the front-end PT/PE should deliver personalized user experiences across devices, networks, browsers, and operating systems.

#5. Real-time tests: Since the applications will be used in a real-time environment with constraints of bandwidth, the typical PT/PE exercise should emulate a real world network. Here, virtual users shall download the software application in an environment having a weak signal strength. Thus, by simulating latency the test team can ascertain the requirement for parallel connections to achieve faster response times.

Conclusion

To sustain and compete in today’s online environment, a software application should be scalable, available, customizable, secure and predictable. However, should the application features not function according to the user expectations, the business can lose its customers - not to speak of its brand equity. It is only a robust PT/PE routine incorporated at all layers of the SDLC, which can ensure the performance to be seamless and top notch.

With businesses getting onto the online bandwagon in a big way, users have started using the online medium to access various products and services. This often leads to situations wherein a website (read eCommerce) or mobile app experiencing a sudden rush of traffic leading to consequences such as system latency, disruption, or downtime. Such an eventuality leaves the users frustrated, businesses losing their brand value, and competitors stealing a march ahead. For example, the eCommerce giant Flipkart suffered a loss in its brand image when users flocked to its site on October 6 to be a part of its Big Billion Day sale. The traffic was huge given the sales pitch promising products at a low rate of 1% of the actual selling price. So, when users responded to the call, the site collapsed with users greeted with broken links and a 404 error message.

So, what does a business do when faced with such an eventuality?

The answer lies in conducting a robust performance testing exercise beforehand. However, it is easier said than done given the number of factors businesses should keep in mind while approaching the job of performance testing.

Factors influencing performance testing strategy

#Identifying the goals: The approach of testing the performance of a system or application should be in alignment with the business goals. The QA tester should be in the know about a few things such as system architecture requirements, similarities in the test and production environments, and the type of users among others. Before testing, the tester(s) should take into account factors such as business requirements, key performance indicators, hardware architecture etc.

#Simulating the test environment: Before writing the script to test the performance of an application the real world scenarios should be simulated as well. The scenarios would include the type of users and their browsing patterns, user expectations, peak user loads, touchpoints and their impact on traffic, and technology constraints etc.

#Testing for mobile responsiveness: With mobile usage growing by the day, eCommerce and other businesses selling products or services should make their sites mobile responsive and apps robust. Efforts should be made to enhance the scalability of the sites or apps by simulating their peak user load using load generation tools. The challenges surrounding a performance testing strategy can be immense given the multiplicity of digital elements. In other words, the concerned mobile app should be tested for performance across a range of devices, operating systems, browsers, frameworks, and networks. The aim of application performance testing should be to validate each and every functionality across digital platforms.

#Testing for cloud adaptability: Cloud provides a suitable platform to test the scalability of a product or service. This is done by simulating the number of users and testing their impact on the functioning of the application. On a cloud platform, it becomes easy and cost effective to conduct application performance testing across geographies and other diverse media.

#Optimizing performance testing services: If time and cost constraints hinder the testing (performance) of an application, then select a few high volume or business critical functionalities for testing across devices and browsers. For example, since users are wont to conduct product searches more than any other form of digital interaction in an eCommerce site or app, the performance testing services should validate the same across channels. In the absence of the same, critical operations of the website or app can fail impacting the users in a negative way.

#Testing workflows across geographies: Users of a specific geographical location should not feel let down by the poor performance of a site or app. This calls for conducting performance engineering over testing to validate the software’s architecture. The QA team should see if the functionalities of the application perform well across devices and browsers. Moreover, it is important to validate the quality of an application in terms of its performance before the onset of any peak holiday season. 

# ROI: Any business website or app should generate revenue either in the short or long term. However, the same is underpinned on users receiving a great browsing experience. If the features and functionalities of a site/app perform well even when the user load exceeds beyond the normal levels, the impact can be positive on the brand.

Conclusion

Businesses should create a robust performance testing methodology to ensure their sites or apps perform to their optimum levels while witnessing a surge in the user base. This helps the site or app to gain brownie points from users and the business to stay ahead of its competitors.

Along with many online services, eCommerce too has found favour with customers due to its convenience, multiple options, and cost effectiveness. Customers are often confused when it comes to choosing one eCommerce store over others as each and every store more or less looks the same with similar features and functionalities. And among the many factors that can differentiate the leader from the rest is its performance when there is a sudden spike in the customer count. This is especially true during the holiday season when customers go on a shopping spree and put the eCommerce site or app under a lot of stress.

The sudden spike in the workload can lead the site or app to face downtime, disruption, server issues, and downright crash resulting in a bad user experience. And if not handled in right earnest, the bad user experience can lead users to move to other portals or apps by completely jettisoning the former. To salvage the situation or pre-empt such a scenario, the performance testing of the eCommerce site or app becomes important.

What is performance testing?

It is testing the responsiveness, reliability and stability of a software system when subjected to a certain threshold of workload. It checks and validates the features and functionalities supported by the software when put under stress. The application performance testing even tests the scalability of an application to ensure it meets the performance criteria when needed. A performance testing approach would include executing a range of tests such as load test, stress test, soak test, spike test, breakpoint test, configuration test, isolation test, and internet test.

Empirical evidence suggests that a dip in the performance of an eCommerce site or app can result in the following:

Decrease in customer     satisfaction

Loss in page views

Loss in conversions

Abandonment of the site

Key reasons for considering performance testing services

#1 Scalability: An eCommerce site or app should be able to function to its optimum level for a particular number of customers. The numbers should be decided after considering the possible load the site or app could experience depending on the geography, demography, range of products/services and server capacity. To ensure an eCommerce site/app functions seamlessly without facing any latency or downtime, it should undergo tests such as load and stress among others. These simulated tests will give the threshold up to which the application can handle the workload and the consequences of exceeding the same. By executing application performance testing before deploying the site or app for actual use, the stakeholders can ensure the features and functionalities of the software work to their optimum. The capacity of the eCommerce software should be scalable enough to handle the increased workload, as and when needed.

#2 Mobile optimization: The present trend among customers is to use smartphones for carrying out eCommerce transactions. So, apart from making the eCommerce site mobile responsive or the mobile app functional, it should be tested for its capacity to handle peak loads. This can be done by simulating the peak load by using a load generation software. Moreover, since a mobile works on a range of platforms, operating systems, networks, and screen resolutions, the challenge of testing an eCommerce app or site’s performance is immense. The performance testing strategy should ensure the app or site does not crash under peak load conditions.

#3 Day and service specific transactions: Certain services on the app or site are more intensive and widespread than others. For example, customers visiting an eCommerce site or app first carry out product searches. And only when they are satisfied with their search, they may end up buying one. Thus, the product search functionality should work seamlessly across browsers, devices, platforms and networks to drive conversions. The other functionalities like add to cart, payment gateway, and checkout should be validated for optimal performance as well. Also, tests should be conducted to ensure the site or app performs seamlessly on certain days of high usage (read Sundays and holidays.)

#4 Achieve ROI: The cost of building an eCommerce site or app can only be recovered if the site or app lives up to the expectations of the users. Although factors such as the pricing of products or the robustness of after sales services play an important role, the site or app should provide the best user experience. Remember, a satisfied client will spread the word around and can potentially bring more customers.

Conclusion

In view of the growing competition, business stakeholders should ensure their eCommerce sites or apps should function to their optimum. This calls for validating the features and functionalities of the site or app in handling peak workloads.

Businesses per se are going through challenging times given the fast changing market dynamics, the advent of new technologies and shifting customer preferences. This need to deliver quality products in double quick time to beat the competition have brought in the concept of Agile-Lean. According to this approach, testing a product for glitches should be done during the development stage only instead of the traditional practice of doing it later. Also, hastening the delivery of quality products can be achieved when waste from the system is systematically removed. Incorporating such changes often needs the overhauling of legacy systems and developing a quality culture in the organization. DevOps steps into the Agile-Lean approach as an all encompassing concept – right from the conceptualization of an idea – production- testing – delivery – receiving customer feedback – and working on the feedback to further enhance the quality of the product.

What is DevOps all about?

The DevOps approach stems from integrating each and every department in the organization especially the Development (read Development and QA) and Operations. It further emphasises on creating a quality culture in the organization where every stakeholder will be responsible for upholding the quality of a product during the SDLC phase and beyond. DevOps is underpinned on coordination, communication, automation and integration of processes across departments. It comprises of components such as Continuous Integration and Delivery. In the former, codes are checked simultaneously along with writing thereby identifying and eliminating glitches quickly and easily. The Development and QA teams carry out the integration of codes on a daily basis by using automated tools. Continuous Integration (CI) ensures the codes in the form of a product are ready for release in the quickest possible time.

On the other hand, Continuous Delivery or (CD) takes the concept of product release a step forward. It stresses on enhancing the product quality based on the customer feedback. This helps the organization to stay in competition by fine tuning the product to the customer preferences of the day. It ensures short iterations, an accelerated time to market and reduced cost. However, it has been seen that organizations that embrace DevOps without incorporating a quality culture fail big time. Another important aspect that can ensure the success of DevOps is executing performance testing on the product on a continuous basis.

Why should you settle for a sound performance testing strategy?

Today’s customers are a restless and choosy lot. They are aware of the new technologies and mindful of the experiences in specific domains. For example, if it is about an eCommerce portal, customers expect their feedback to be taken seriously and queries attended to promptly. They are aware of the AI-based chatbots that have brought about a significant enhancement in customer services. So, should you own an eCommerce portal, you better incorporate technologies that enhance the customer experience a great deal. Hence, your performance testing strategy should incorporate load testing, stress testing, scalability testing, spike testing, and volume testing.

These types of performance testing services ensure the product works fine when there is a spike in customer demand. For example, during the holiday season, there are spikes in the shopping pattern of customers. And if your eCommerce portal is not tested to address these, the system may hang or crash leaving the customers frustrated. In such cases, you may even lose your customers for good.

Moreover, since today’s software has multiple interfaces with devices, operating systems, frameworks, and networks, it should undergo a rigorous performance testing exercise. The end-to-end testing on performance metrics can validate the software as far as meeting customer preferences is concerned.

How to execute DevOps through continuous testing?

This can be achieved by following three steps.

The first step is to streamline the processes and let DevOps strike a root in your organization. The silo-based Development and Operations teams should be assimilated. The Operations team should listen to the customer feedback and pass the same to the Development team for quality enhancement by being a part of the development scrum. The CI/CD pipeline should be streamlined and made efficient for each and every stakeholder.

The second step is about breaking a large test into smaller components mostly at the API level. The performance testing approach should be to test various services independently of each other in a specific timeframe.

The third step is about testing the dependent services by using virtual dependencies to test various scenarios. This can save time as well as not impact the UX of customers during testing.

Conclusion

A robust performance testing approach should be undertaken in the DevOps environment to validate the scalability, functionality, and usability of software by repeatedly testing its various components. This can help identify and eliminate the glitches, thus making the software more customer oriented and competitive.