A key strength of a TCoE is that knowledge (both testing and business), information and experience is built up centrally by the team, which can then be deployed onto programmes or projects of work using a central set of infrastructure. As well as the establishment of a team, these individuals are supported by process, documentation and tools, with the aim of establishing best practice and measurable, predictable results across engagements in the organisation.
Key Elements of a TCoE
The three elements here that must be in place are Team, Process and Tools.
The most important element of a TCoE is having a team in place to service incoming project requests. It is important that the individuals in the team have the right skills and a consultancy like mindset, in order for the testing function to provide lasting value to its customers. The following roles are typical in a TCoE, although not all of these roles will be appropriate for every implementation, it does depend a lot on the organisation, and particularly the projects:
– Head of Test / Test Service / Centre Manager
– Test Managers
– Test Architects / Principals
– Test Leads / Co-ordinators
– Test Automation Specialist(s)
– Performance Specialist(s)
– Information Security Expert(s)
Process / Documentation
Establishing a core set of standard documentation and a common approach to testing for different project types is a key part of the TCoE. First and foremost is the Test Strategy. Depending on the size of the organisation and the amount of projects it has, will dictate whether multiple test strategies are necessary, or whether testing can be defined in a single document. Some organisations have a top level Test Policy document, which sets out the high level objectives of testing, this can then be supported by one of more Test Strategy documents. Ideally there needs to be a single test strategy for each group of similar projects in an organisation, for example web developments, or SAP applications, etc.
Having a clear Test Strategy in place is core to achieving success in testing. The strategy should define the approach to testing on different types of project based on the changes made. Examples of changes could be patches from third party suppliers, business as usual Agile or Waterfall development, change requests, bug fixes, hardware changes etc. Each change should have a corresponding entry in the strategy to define what the organisation’s usual approach should be, which is essentially a decision made based upon risk. This doesn’t mean an inflexible strait jacket of test processes, there are always exceptions – but it is important to define the rule of where test phases are appropriate, what is the process, where can documentation be found and what tools to use. A strategy should be accessible to all, having it based on a wiki or SharePoint site works well. This will not only help the testing team, it also helps project managers understand what testing will be needed on their projects, so that the right level of time and cost can be made available, up front.
Typical test phases that should be covered by the Strategy include the following:
– Unit Test (carried out mostly by developers, but it is useful to provide guidance / standards)
– System Test – Manual testing carried out in sprint
– Test Automation – Regression packs, retrospective or developed in parallel / in sprint, tools, frameworks, data, coding conventions, continuous integration links
– Performance – When relevant, core systems in scope, approach, tools and monitoring
– Acceptance – User involvement, co-ordination, guidance and management rather than active participation.
The Test Strategy will also define quality gates and entry criteria at a high level, which can then typically be used in more detailed levels of documentation on projects, for example test plans. The methodology defined by the strategy will include a set of standard templates to be used, which are likely to include the following:
– Test Plans
– Test Charter (for exploratory testing)
– Test Scripts / Specifications
– Test Completion Reports
Note that there are likely to be different versions of the templates needed for different phases of testing, e.g. system / performance. In addition, a feedback form should be completed after each project as part of the wash up / lessons learned process, in order to feedback experience gained to the central team, so that the process can be improved.
Metrics are also very important to collect, statistics like estimate vs actual timescales, cycles of testing, defects opened / fixed / closed / per sprint and by cycle and release. Recording and analysis of metrics will help with measuring return on investment (ROI) and the success of the centre.
The usual testing activities are performed within each phase according to the workflow strategy – including test case design / execution and reporting. It is vital for the TCoE to keep a track on resources, projects in flight and the project schedule, to ensure resources are allocated efficiently and that all the projects in the pipeline are resourced, so that external assistance can be organised in plenty of time if needed.
One element that is sometimes overlooked when setting up a TCoE is establishing a process for customers in the organisation to request testing, but also making sure everyone knows how to engage testing and what service can be provided from the TCoE. This can be resolved by circulating the test strategy and high level approach to the project sponsors / programme management / project office, but also running workshops to increase visibility and to educate customers about what is available, how / when to request it and what to expect.
Choosing the right tools is key for any testing team, but arguably even more so when implementing a TCoE. The types of tool likely to be needed include:
– Test Management Tool – a single tool capable of recording defects, test cases and results and reporting on them across customisable views as a minimum, but also potentially scheduling, release management and mapping of requirements to test cases and subsequently defects. It is very important to select a tool that meets the TCoE’s requirements and allows the recording of metrics in line with those required to provide ROI and cost savings over time with metrics like the Defect Detection Rate.
– Unit Testing – if test automation is to be used at the unit test phase, the tools should be standardised across development teams, including integration into Continuous Integration (CI) tools.
– Functional Test Automation – for the automation of regression tests on systems where regular releases are likely and risk of failure would be significant. This category would usually include device emulation, need to fit in with the development lifecycle in use, technical fabric of the application(s) under test and the skills of the team who will be using it.
– Performance Testing – for testing non-functional attributes of applications once functionality has been proven. It is vital to ensure that systems perform as needed when accessed by the expected level of users, at a range of levels – usually normal, peak, spike and soak. The best tool for the job usually depends on the technical architecture of the applications under test. Open source tools can be appropriate when considering single protocols, e.g. web, but there are some excellent Enterprise grade tools available that can handle multiple applications / protocols as well as integrate with automation and test management tools and allowing full lifecycle management.
– Security Testing – Security is often a discipline that is outsourced and executed remotely, however there are some tools available that can be integrated into the development process, to improve the code and identify vulnerabilities as they appear, thus reducing the cost of remedial action when they are found later.
– Others – other tools to be considered include data management / masking, database query tools, CI tools etc.
Is it for me?
One of the main aims of a TCoE is the need to shift left and to eliminate defects early on, thus saving money. However, there do need to be enough projects and there does need to be an acceptance in the organisation that costs will not go down right away. Only once established and bedded in will overall costs be lower. Included in these calculations should be the number of defects reaching production, both before and after implementing the TCoE as here is where the real savings lie, as well as the increased confidence in the IT department.
The benefits can be summarised as follows:
– Shared resources mean more specialist skills are affordable
– Centralised expertise and knowledge development – both of business / system knowledge but also of testing knowledge
– Consistency across projects
– Measurable, predictable results across projects
– Increased confidence in QA.
Jonathan Binks, Principal Consultant
If you would be interested in hearing more about establishing a TCoE, or would like help or advice on your current implementation, we would be happy to help, please contact us.