Successfully Implementing Tribal SITS for University of Sheffield

Testing Requirements

Initially, there was a need to assess the programme and provide a schedule for testing, indicating areas of most risk so that the test phases offering the most benefit could be prioritised and estimated. Prolifics deployed a Principal Consultant to carry out the work, which involved organising workshops with the stakeholders responsible for each of the 6 modules of the new system, representing 63 work packages, in order to develop a test strategy and estimates. We developed a custom estimation model based on our knowledge of SITS, the HE sector and the in-scope integrations, which took into account a wide range of factors to estimate time and cost. The approach we took to identify the areas of most risk was based on the relative criticality of each area, business function, level of customisation of the core product and where there were integrations with other applications. We also provided recommendations to improve overall quality from requirements through to unit testing, integration testing, tools, test case reviews and regression testing.

As well as overall estimates, the strategy identified several key elements of a testing managed service that would be supplied by Prolifics, utilising a hybrid model of UK-led and offshore-delivered testing services, using our in-house tools and accelerators.

• Test Automation of a core set of Academic Model regression tests
• Performance Testing of the Academic Model interface
• Data Validation testing of an interface with the current Student Records System
• Performance Testing of the SITS – SAP Interface
• Validation of the Reporting Suite – checking data, formatting and complex queries

Strategy and Execution

Once these areas were agreed as in scope, our team worked directly with project team members to capture the details and provide accurate, fixed price/delivery costs for each one.

Test automation was designed and developed remotely by our offshore team, via connecting to machines within the UoS network and using our in-house developed WebDriver and Java automation framework. 38 complex SITS UI test cases were automated, which allowed the University to have confidence in the implementation and re-run the tests on demand. These tests were linked to the University’s Jenkins server, to allow automated execution and continuous integration. Prolifics handed over all test assets to the UoS test team to continue to use and maintain once our work was complete. The 38 test cases were also scripted using JMeter and tested for performance, to ensure the on-premise system would continue to function under the load of the expected number of concurrent users. Our team performed load, stress and soak testing of the application and identified several areas for improvement and removal of bottlenecks before tests were repeated.

Data validation was performed using our in-house Effecta Data Validation Manager, which was used to write a series of queries based on data mapping documentation. The transformation and mappings logic defined was developed into custom queries in Effecta. Data was within the application and output from the SITS database to Talend, via XML. Effecta ran queries at the Talend level for the data matching each of the specified test cases, generated by the automation framework already developed. Using this approach, our offshore team developed a comprehensive data validation framework that was subsequently used to check several thousand records and save significant amounts of time and money.

There was also a need to ensure the SITS–SAP interface could handle large volumes of data, prior to making the system live. Prolifics designed a performance test framework using JMeter, to simulate the expected peak volumes of data across the interface, which included staff and student data.

As a separate stream, another Prolifics team performed functional testing on a set of 25 reports operational available from SITS, checking the correct data was displayed and that security permissions were correctly populated, against both data and the reports themselves.

The activities above were performed in parallel with the University’s own testing function and were overseen by a Prolifics Test Manager, who provided regular status reports. Our team joined daily stand-up calls on progress and a regular weekly call with the entire team so that all streams were aligned and building towards the same goal, as well as coordinating activity across test environments, sharing data and understanding release schedules /defect fixes.

Benefits achieved through rigorous testing

Our team's technical test activities yielded substantial benefits, including:

Cost Savings: By identifying and rectifying defects early in the testing phase, we prevented potential issues from escalating, saving the University considerable resources.

Enhanced Quality: Our thorough testing approach ensured that foundational elements of the SITS application were nearly free of defects before entering production, guaranteeing a smoother and more reliable user experience.

Long-Term Reusability: The automation, data validation, and performance code we developed can be used by the University in the long run, leading to sustained savings and ongoing benefits.

Increased Confidence: Our testing activities boosted the University's confidence in the system's stability and functionality, reducing the risk of unforeseen issues post-launch.

Efficient Resource Allocation: By addressing defects and performance concerns early on, the University was able to allocate its resources more efficiently and effectively.

We are accredited partners of Tribal and can develop a free, ready-made Framework that enables Performance testing for universities. We’ve gained extensive experience in SITS and have worked with over 20 universities, as well as institutions in Singapore and Australia. If you would like to learn more about how our testing services can benefit your organisation or if you’d like to receive a free Proof of Concept exercise, please don’t hesitate to contact us.