Performance Testing is an important non-functional test phase that gauges system stability and response time under different workloads.
Frequently referred to as Load or Stress Testing, Performance Testing in fact encompasses both types of testing.
Load Testing determines the ability of infrastructure to support the expected usage of a system, while Stress Testing examines the ability to cope with exceptional transaction volumes.
At Prolifics Testing, our dedicated Performance team use cutting-edge technologies, such as our in-house accelerators, custom dashboards, and pre-deployed Cloud injector machines, to measure software reliability at load. These techniques enable us to rapidly deliver a quality service to our customers at significantly reduced cost.
Why Performance Test?
There's no doubt that Performance Testing is absolutely critical to successful business operations and end user satisfaction. If below par, the reliability of your websites, apps, and systems will not go unnoticed, with the potential to seriously undermine your commercial objectives.
There are three core reasons for Performance Testing, which we examine in detail below:
- Preventing system crashes
- Achieving acceptable response times
- Mitigating scaling risks
Preventing system crashes
No doubt, the nightmare scenario for most companies is when high levels of traffic bring systems grinding to a halt.
According to Atlassian, website downtime incidents 'are not only potentially toxic to customer trust and loyalty […] – they're also the financial grim reaper'. Crashes aren't just harmful because of the opportunity cost of downtime, but also due to the very real possibility of reputational damage.
Good Performance Testing therefore aims to identify and remedy problems that would otherwise cause a website to crash, prior to go-live.
Achieving acceptable response times
Similarly, as technological standards continue to rise, users' tolerance for website waiting times is falling drastically. According to research carried out by Google, when websites fail to load within 5 seconds for mobile users, the probability of these users 'bouncing' - abandoning the site to find a different resource - increases by an average of 106%.
Again, good Performance Testing works to mitigate this risk by achieving exceptional response times and thereby streamlining the overall user experience. In turn, this means more online traffic for your business, with an increased likelihood of conversions.
Mitigating scaling risks
Finally, as your business grows, new customisations and interfaces between systems have the potential to create performance bottlenecks. Applications need to be able to scale, and their performance requirements met well into the future. Manual regression testing is one solution to this problem, but the process is often slow and calls for close monitoring to ensure meaningful results.
When implemented as an automated solution, Performance Testing means these issues are caught as the application grows, so that fixes can be quickly deployed and tests maintained for future use. The risk that comes with scaling up is effectively mitigated, leaving you free to focus on other aspects of the business.
Interested so far? Let us know your requirements via our short contact form, and we will be in touch to organise a free consultation:
What sets us apart?
All too often, the vital job of Performance Testing is overlooked during projects due to it being seen as expensive, time-consuming, and highly technical. We're here to show you that this needn't be the case.
Here's a brief overview of how we achieve exemplary results for our clients:
- Ready-made frameworks for both open source (Apache JMeter, Locust) and enterprise tools (LoadRunner, Silk Performer, NeoLoad)
- Pre-built environments for Amazon Web Services (AWS) and Microsoft Azure, to quickly simulate large amounts of worldwide traffic
- Internally-developed Accelerators that reduce testing times and costs by up to 40%
- Integration with Docker for containerisation, reducing reliance on expensive hardware
- Provision of live metrics and real-time reports for every test via our Grafana dashboard
- Ability to analyse trends over time thanks to improved Performance Analytics and integration to Power BI
Unlike other Performance Testing providers, we avoid the need for complex scripting and drawn-out creation of complex scenarios, helping your organisation to Shift Left with ease.
And although we have particular expertise in designing and delivering performance test suites based on the JMeter open-source platform, 'one-size-fits-all' is never a part of what we do. Our UK-based services adapt to suit your business, not the other way around.
We offer all of our clients free use of our Performance Testing Accelerator suite as part of their service. These accelerators help to automate what would ordinarily be time-consuming and expensive manual procedures, as well as increasing reliability of test results.
Our three most popular Performance Testing Accelerators are:
Script Enhancer Accelerator
Using the robust VB.Net programming language, the Script Enhancer Accelerator is specifically built to reduce Performance Testing times by automating the customisation of the script. This is achieved by inserting fields such as Transaction Names, Error Handling, Think Times and Check Points.
Test Data Mining Accelerator
Performance tests can require significant amounts of data to be generated manually. Our approach boasts full automation of this process by using test data mining techniques from any available database and automatic generation of the remaining data using scripts.
Infrastructure Monitoring Accelerator
Our Infrastructure Monitoring Accelerator is an agentless tool that uses versatile Python scripts to capture raw data from logs across applications, web servers, and databases. These data include response times, throughputs, memory utilisation, CPU load and more.
Our Accelerators integrate with all the major test management tools, including Micro Focus ALM and Jira. We also offer automated test case design for a range of major COTS applications, including SAP, Oracle EBS, and Workday HCM.
Depending on the systems under test, your software architecture, and possible security or access limitations, we run our UK-based testing projects either onsite (COVID permitting), remotely, or in a blended delivery model, to reap the rewards of onsite consulting and offsite delivery. However the project is delivered, we always work closely with you, involving your teams in the process, and frequently inviting you to view the tests running in real time.
In over 20 years of business, we have carried out Performance Testing within a range of environments, from full Cloud, to Hybrid Cloud, to fully on-prem. For example, where cloud-based injectors are not suitable for a client, perhaps due to security concerns, we have used machines on clients' networks so that the entire test cycle is kept within your private infrastructure.
It's also worth mentioning that we’re not tied to any testing tool providers, so are able to offer a completely impartial service that pairs the best tool for each project with your desired outcomes.
Performance Testing in the Cloud
When it comes to Performance Testing in the Cloud, our consultants have significant expertise in Micro Focus' market-leading LoadRunner Cloud solution.
Using LoadRunner Cloud enables us to:
- Run Performance Testing as part of Continuous Deployment and DevOps models
- Identify problems faster with efficient root cause analysis
- Test earlier and faster with open source and third-party tool integration (supports reusing JMeter, Gatling, and Selenium scripts; integrates to Jenkins, GIT, AWS Code Pipeline, and Docker)
- Reduce hardware maintenance with flexible testing models
- Optimise application performance prior to deployment
Moreover, as a Gold Partner to Micro Focus, we offer discounted licenses for the full LoadRunner suite - Professional, Enterprise, and Cloud, as well as end-user training to provide your team with all the skills they need.
Our approach to Performance Testing is tailored to the match the requirements of each client and project, but is always based on a rigorous profile of tests designed to quickly apply high volumes of load and identify bottlenecks.
The service we provide is completely flexible, ranging from one-off engagements completed in as little as two weeks, to major outsourced projects where we integrate Performance Testing into your CI/CD process. Whether you're overseeing the implementation of a new website, app, or ERP/CRM system, we have experience carrying out Performance Testing at all stages of development for projects of all sizes and functionalities.
A typical Performance Testing engagement for us looks something like this:
Step 1: Planning
The first stage of a performance test engagement is to document a comprehensive test plan, detailing the scope of testing, transaction details and frequency, the number of users, transaction times, assumptions made, load generation environment, testing objectives, entry and exit criteria, test data, reporting structure and frequency, defect management, schedule, and contact details for all personnel on the project.
Step 2: Build
Once the plan is agreed, user journeys can then be recorded through the application under test using the selected tool. Session variables need to be correlated to ensure valid activity is simulated and data items are parameterised, in order to allow the entry of different data for each user journey through the system. Doing so maximises the realism of the simulation. User journeys are then grouped together to form scenarios to measure the expected use of the system.
Step 3: Environment
Server monitors are put in place and data is populated in the target database to realistic levels. Once complete, dry runs are carried out using the generated scenarios to ensure all elements of the test are working before proceeding to the next step.
Step 4: Execute
The tests that were laid out in the planning stage are run. This is likely to include normal load, peak load, soak, and stress tests. Typical problems at the unit level are identified, such as deadlock detection, memory leaks, query optimisation, architecture issues, etc. The team ensures that API response time meets the associated e2e SLA requirements.
Test Setup in Apache JMeter
The graph below shows the setup for a Peak Load test in Apache JMeter, using Ultimate Thread Group. The graph shows the ramp up, steady time, and ramp down of the users. In this example, the test is set to run with 200 users with a ramp up time of 300 seconds (5 minutes), steady state time of 3600 seconds (1 hour), and ramp down time of 240 seconds (4 minutes).
The graph below shows the setup done for a stress test in JMeter, again using Ultimate Thread Group. Displayed is the incremental user load injection to the application, carried out to find the system breaking point. Continuous monitoring of the server and SLA stats is undertaken during this test.
Step 5: Analysis and Reporting
The final stage in the testing process is analysing the results, including transaction times, server resources, and network and database statistics, to provide a final report. Note that interim test results are also normally provided after each test run, in order for any bottlenecks or defects to be fixed and for the application to be fine-tuned.
Step 6: Support
Following completion of the test cycle, we actively encourage skills transfer to your teams. Our services include full handover, training, and ongoing support to suit your needs.
This is especially advised if you are looking to integrate Performance Testing with your development lifecycle in the long-term. In this case, our team within the UK provides an ongoing consultancy service, where test suites are maintained as applications are developed, regular tests are triggered by code deployments and run remotely, and results are provided automatically.
Commonly known as Performance Engineering, this service emphasises continuous testing and continuous deployment, in line with DevOps methodology. In this case, our consultants effectively design tests that evolve with your applications, enabling earlier visibility of potential performance issues at the Unit and System levels. This approach dramatically reduces the likelihood of application rework when performance problems are discovered further down the line.
Get in touch
Whether you are interested in fast-tracking Performance Testing to achieve results quickly, or wish to implement it as part of your CI/CD pipeline, we have over 20 years' experience of delivering Performance Testing services to clients spanning a wide variety of industries.
Following a successful consultation session, we would provide you with a detailed proposal covering scope, timescales, deliverables and SLAs, along with a fixed price / time-and-materials quote, depending on preference.
We look forward to working with you!