Supplementary_File – Supplemental material for Development and testing of a checklist to assess compliance with the faculty of pain medicine’s core standards for pain management services: experience in a new national tertiary pain service
GUID: 9A4F1A3E-0520-47C9-B13B-DEA4242BFBD7Supplemental material, Supplementary_File for Development and testing of a checklist to assess compliance with the faculty of pain medicine’s core standards for pain management services: experience in a new national tertiary pain service by Asma M Torkamani, Rachel Atherton, Martin Dunbar and Hamish J McLeod in British Journal of Pain
The Faculty of Pain Medicine recently published the first UK-focused Core Standards for Pain Management Services (CSPMS). We present an audit checklist tool developed to map compliance to the CSPMS, which offers a practical method of auditing any pain management service against the standards.
The checklist tool was developed and its utility was field-tested in the Scottish National Residential Pain Management Programme (SNRPMP), a newly established service offering residential service to people in Scotland.
The checklist tool developed provides an easy and practical approach to evaluating any pain service against the national standards. Its application to evaluate the SNRPMP indicates that the service meets the majority of CSPMS standards and highlights aspects of the service requiring improvement.
The layout of the developed checklist tool offers an alternative format for the structuring of the national standards in possible future revisions. The audit checklist tool enables evaluation of services with a numerical score, enabling monitoring of their compliance with national standards as well as comparisons between pain services.
Keywords: Chronic pain, pain management, faculty of pain medicine, residential pain management, audit
In the United Kingdom, approximately 14 million people suffer from chronic pain, and in Scotland, 18% of its population is affected. 1 Chronic pain is not simply an unpleasant sensory and emotional experience; it is a condition that impairs many aspects of functioning, impacting a person’s ability to engage in simple activities of daily living and impairing their quality of life. 2 –4 Interdisciplinary Pain Management Programmes (PMPs) are one of the recommended treatments for chronic pain and are well established in the United Kingdom. 5,6 However, pain services, including PMPs, predominantly operate independently and are guided by profession specific guidelines and service specifications, making uniform ascertainment and monitoring of care quality a challenging task.
A growing emphasis on the provision and improvement of services for chronic pain in recent times has resulted in a number of recent developments, including the publication of the Core Standards for Pain Management Services (CSPMS) in the United Kingdom, 7 which provides a national benchmark against which the provision of care can be assessed and areas for improvement identified across pain services. The CSPMS offers a comprehensive list of standards in relation to seven areas covering: service design, physical facilities, team members, assessment pathways, interventions provided, appraisal and revalidation for medical staff, and staff involvement in service improvement and clinical governance. These include general and specific standards that apply differentially to different types of pain services (e.g. primary/secondary/tertiary) and subgroups of pain (e.g. acute, cancer related). While the standards are undoubtedly a useful development, they are written in a narrative format that makes it difficult to judge the extent to which a service complies with them.
We therefore identified a need to develop an audit checklist tool, based on the CSPMS, that could support the implementation of the standards in different pain services. We developed and field-tested the tool in the Scottish National Residential Pain Management Programme (SNRPMP), to evaluate the delivery and organisation of the service in relation to the national standards. The SNRPMP is an intensive 3-week group-based programme that was developed in order to support patients and their carers in managing chronic pain and learning to cope with the effects of the condition. The service was developed following the Getting to GRIPS report 1 which highlighted the lack of access to a specialised residential programme in Scotland. The service opened in January 2015 and ran its first group in November 2015.
An audit tool was developed in a checklist format by incorporating the CSPMS as specific questions. Data for completing the audit checklist and evaluating the SNRPMP were collected from a number of different sources. Data collection took place between July and August 2016.
The CSPMS outlines a total of 251 standards relating to different pain services across. The audit checklist was divided into seven parts to reflect the seven chapters within the CSPMS. Close inspection of the 251 CSPMS highlighted multiple sub-parts within some standards and a degree of overlap with others across the different chapters. Thus, 15 standards warranted consideration in more than one part of the checklist tool, resulting in a total of 279 standards being considered in the developed audit checklist tool (see Supplemental Material 1).
To accommodate for the multidimensional nature of some standards (e.g. ‘There must be appropriate accommodation… administration support… team should work closely together through joint clinics…’), the checklist tool offered the option to indicate whether a particular standard was clearly met (all/most parts met), partially met (some parts met), or not met (all/most parts not met). Furthermore, some CSPMS standards were not relevant to the SNRPMP as they were service specific (e.g. ‘Acute pain in children…’) or difficult to establish without direct observations (such as the nature/quality of staff interaction). Thus, two additional options were introduced in the checklist: not applicable (different service/pain group/interventions) and not determinable (observation required). The modal or mean scores for standards containing multiple parts were used as appropriate to indicate whether a specific standard was met clearly, partially, or not met.
The final audit checklist items were reviewed by two consultant clinical psychologists specialising in chronic pain management as part of a two-stage reviewing process. The checklist items were initially screened against the original multidimensional standards from the CSPMS to ensure all standards were included and appropriately divided, and any overlapping statements were subsequently grouped together.
To audit the SNRPMP and gather information for completing the checklist tool, a number of data collection strategies were identified in consultation with the service clinical lead and another consultant clinical psychologist in pain management. Service-related documents were used to obtain information related to access to the SNRPMP, description of the service and information concerning the assessment and intervention strategies offered. A site inspection allowed investigation of the standards related to consultation facilities and the use of equipment. All patient case records (N = 33) since the establishment of the service were reviewed to evaluate the process of patient consultation, outcome measures used and treatment information. Service records were accessed to determine the content of the programmes offered and the activities of the service. Two staff questionnaires were developed to seek information concerning team structure and functioning. One consisted of identifiable information such as training/qualification (see Supplemental Material 1), and the other completed anonymously sought sensitive information such as experiences of working at the SNRPMP (see Supplemental Material 2). All clinical staff (N = 7) completed and returned the questionnaires.
This audit did not have a direct implication on the routine management of patient’s care, and therefore, formal NHS ethics approval was not warranted. Information Governance was approached and Caldicott Guardian Approval was granted for accessing the patient records. In line with the Data Protection Act, 8 data were anonymised and patient identifiable information was removed.
The audit checklist tool consisted of 279 standards relating to different pain services, of which 121 (43%) standards were not applicable to SNRPMP and a further 18 (7%) were not determinable. This resulted in 140 (50%) standards against which the SNRPMP was evaluated.
The SNRPMP was audited against the CSPMS using the checklist tool. The results of this audit indicate that on average the organisation and delivery of SNRPMP meets 93% of the relevant CSPMS standards. The service met majority of the standards clearly (73%), and some partially (20%), and a few standards were not met (7%) (see Figure 1 ). The outcomes for each part of the checklist are reported below.
Audit outcomes for the seven parts.
Standards relating to the service description were met clearly (50%) or partially (50%). The SNRPMP seeks sufficient informed consent from patients and offers assistance to those requiring additional support. The number of people in each cohort for pain management interventions is sufficient and the timeline for assessment and treatment was mostly met. Discharge is appropriately planned. Performance outcomes are collected. The service is ‘properly’ resourced with time and personnel. The service was relatively new at the time of this audit, and as the service matures, information relating to follow-up ratios or waiting times should be formally recorded and any such information provided to patients should be documented.
The majority of standards relating to the physical facilities of the SNRPMP were partially met (65%) and some were met clearly (33%). The SNRPMP offers residential accommodation near the clinic facility. The hospital-based premises comply with most aspects of the Equality Act (2010), and has a sufficiently large room to accommodate group activities. Identified areas for improvement include additional consulting rooms and workstations to ensure clinical activity is not compromised. The heating and ventilation of the building can also be problematic and should be evaluated.
The majority of standards related to the service team were clearly met (77%) while some were met partially (10%). There was a 100% response rate from staff. The service is made up of a multidisciplinary team (MDT) with appropriate experience and qualifications. Staff offer joint clinics, attend meetings, engage in annual appraisals and continuing professional development (CPD), receive mandatory resuscitation training as part of their NHS employment and adhere to appropriate legislations and guidelines. Staff seek and provide support when necessary. Clinical activity is well documented and files are kept up-to date. On average, staff rated time/resources for attending MDTs meetings and time/opportunities for CPD moderately. Despite a great range, on average, staff reported moderate satisfaction with the amount of training, knowledge and support they have for carrying out their duties efficiently. The SNRPMP does not have a dedicated pharmacist.
The majority of standards relating to the assessment of patients were clearly met (95%) with the exception of one standard that was partially met (5%). A bio-psychosocial assessment is offered at different time points, including before and after intervention. Alongside the clinical interview, validated and common measures are used and outcomes are recorded. The assessment structure offers an opportunity for the patient’s story to be heard and for individual goals to be identified. The duration of assessment is sufficient and rated well by staff. There are opportunities for involvement of family/carers. Staff seek advice when working outside their own competence. Patients accessing SNRPMP are not offered a shared individualised management plan; however, the group programme offered is intended to meet the patients’ needs and appropriate formulation of this is communicated.
All aspects of treatment offered by the SNRPMP met the standards clearly (100%). The group intervention encourages peer support and promotes social and physical functioning, well-being and self-management skills. Appropriate psychological and behavioural interventions are offered including physiotherapy and clinical psychology input. Family/carers are included. Access to the service is based on clinical need and the service does not discriminate based on demographic variables. The patients’ general practitioner is kept informed. No areas for improvement were identified.
The results suggest a number of strengths in the service. Standards for education and evaluation of staff were clearly (65%) or partially met (33%), and all staff reported engagement in relevant CPD. The staff teams rated their confidence in their level of training and knowledge moderately, and thus will benefit from additional support in increasing this.
Standards for service improvement and clinical governance were clearly (40%) or partially (60%) met. Staff engage in clinical governance and audit. All staff took part in this audit and the clinical lead was actively involved in the development of the checklist tool. Staff use Datix for critical incident reporting. Due to the recent establishment of the service, internal auditing is currently not in place. However, relevant information is being collated for this purpose and should be utilised for auditing purposes in the future. Clinicians rated having the ‘time’ and ‘opportunities’ to engage in such activities moderately.
The CSPMS is the first nationally recognised document outlining the core standards for specialist pain services, and the checklist tool developed here provides a generic and useful tool for evaluating any pain service in the United Kingdom against the CSPMS. As shown by the outcomes of this audit, the CSPMS contains a large number of standards, many of which may not be relevant to specific services, such as PMPs, and others that can be difficult to operationalise. For instance, some of the standards in this first version of the CSPMS are somewhat vague and unclear, making general or broad statements, and other standards contain multiple parts within them. Future revision of the standards could structure the standards as specific questions to facilitate their application to services.
The checklist and its corresponding five response options enable evaluation of any pain service against all standards within the CSPMS, irrespective of their relevance. Furthermore, although using average scores for standards that contain multiple sub-parts can potentially result in important information getting lost, the option to indicate the extent to which a standard is met provides a helpful way of representing how a service meets the national standards. In addition, the range of methods employed in this audit for collecting data to complete checklist tool provides a practical approach for evaluating a service. The staff survey is a useful method of gaining insight into the experiences of individual team members. The use of the checklist tool is therefore likely to offer a comprehensive impression of any service. Future audits could also use observational/interviewing techniques to collect data for the ‘not-determinable’ standards, which was beyond the aim and scope of the current study.
In addition, more data on the checklist tool’s utility could be collected through its application and use across different services. This will enable future refinement of the checklist tool to promote its usefulness in different settings.
This audit provides an insight into the organisation and delivery of the newly established SNRPMP. The results indicate that the SNRPMP meets the national standards to a great extent. A number of strengths for the service are highlighted, particularly in relation to assessment and intervention, and the training and qualifications of most staff. This audit also indicates some areas for improvement, and highlights a number of standards that are not met. However, these were largely related to the SNRPMP being relatively new service at the time of the data collection and as such some aspects of its operation were still in development. While this may have compromised the outcomes of this audit, it offers a timely opportunity for focus areas for the service. Thus, the results of this audit could serve as a baseline for the SNRPMP and future audits using the checklist tool could capture the development and growth of the service as it matures.
The CSPMS document offers a useful range of standards that can offer a reasonable insight into the functioning of any pain service, despite difficulties relating to its organisation of standards. The checklist tool developed provides an easy and practical approach to evaluating any pain service against the national standards. The checklist tool was applied to the SNRPMP and its results indicate that the service meets the majority of CSPMS standards, despite being a relatively new establishment. The CSPMS could best be described as narrative in structure. The audit checklist tool operationalised those narrative standards so that the output from using the tool is a numerical score. This allows for services to be judged to an absolute standard (100% compliance), as well as allowing for comparisons between services. We commend the tool to services wishing to benchmark their performance and to readily identify aspects of the service requiring improvement.
Supplementary_File – Supplemental material for Development and testing of a checklist to assess compliance with the faculty of pain medicine’s core standards for pain management services: experience in a new national tertiary pain service:
Supplemental material, Supplementary_File for Development and testing of a checklist to assess compliance with the faculty of pain medicine’s core standards for pain management services: experience in a new national tertiary pain service by Asma M Torkamani, Rachel Atherton, Martin Dunbar and Hamish J McLeod in British Journal of Pain
Thanks to all staff at the SNRPMP for their time and participation in piloting the audit checklist tool.
Conflict of interest: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Contributorship: A.T., H.M., R.A. and M.D. are the contibutors of this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
Guarantor: A.T. is the guarantor of this article.
Supplemental material: Supplemental material for this article is available online.
ORCID iDs: Asma M Torkamani https://orcid.org/0000-0003-3554-1715
Hamish J McLeod https://orcid.org/0000-0002-4225-1815
1. NHS Quality Improvement Scotland. Getting to GRIPS with chronic pain in Scotland: getting relevant information on pain services . Edinburgh: NHS Quality Improvement Scotland, 2008, http://www.nopain.it/file/GRIPS_booklet_lowres.pdf (accessed 24 June 2019). [Google Scholar]
2. Elliott A, Smith BH, Penny KI, et al. The epidemiology of chronic pain in the community . Lancet 1999; 354 ( 9186 ): 1248–1252. [PubMed] [Google Scholar]
3. Sjogren P, Ekholm O, Peuckmann V, et al. Epidemiology of chronic pain in Denmark: an update . Eur J Pain 2009; 13 ( 3 ): 287–292. [PubMed] [Google Scholar]
4. Toblin RL, Mack KA, Perveen G, et al. A population-based survey of chronic pain and its treatment with prescription drugs . Pain 2011; 152 ( 6 ): 1249–1255. [PubMed] [Google Scholar]
5. Eccleston C, Morley SJ, Williams AC. Psychological approaches to chronic pain management: evidence and challenges . Br J Anaesth 2013; 111 ( 1 ): 59–63. [PubMed] [Google Scholar]
6. Scottish Intercollegiate Guidelines Network (SIGN). Management of chronic pain (Publication No. 136). Edinburgh, 2012, https://www.sign.ac.uk/assets/sign136.pdf [Google Scholar]
7. Faculty of Pain Medicine. Core standards for pain management services in the UK . London: The Royal College of Anaesthetists, 2015. [Google Scholar]
