Advanced Clinical Practitioner Training Support Package Evaluation

In response to pressures on the national health service, significant changes have been made over recent years. Pressures around workforce shortages, have led to the promotion of new roles: these include ACP’s. To enable the development of these new roles training is required to give staff the skills/competencies required. In South Yorkshire and Bassetlaw, ACP training has been commissioned at Sheffield Hallam University since September 2016 and the University of Sheffield September 2017. Funding has also been provided to support ACP training in employing organisations. In 2017, because of perceived challenges in the ACP training process a Training Support Package (ACPTSP) was established in the area to try to better support ACP’s through their training. The ACPTSP project team have put in place interventions to help address perceived challenges with the training and development of ACP’s including perceived issues with employers and wider workplace support. In February 2019 a decision was taken to evaluate the progress that ACPTSP has made to ensure that the ACP training was appropriate and effective and that trainees were well supported throughout their training, both academically and in the workplace. This is the report of the ACPTSP evaluation. Introduction The objectives of the evaluation were to: • Understand what interventions have been developed during implementation of the ACP support package? • Evaluate how these interventions have impacted the ACP training process? • Identify areas where further support can be implemented to positively impact the ACP training process? For the purpose of the evaluation, the overall aim was summarised in the following question: - Has the ACP training support package improved the processes and outcomes for ACP training in the South Yorkshire and Bassetlaw area? Methodology A pragmatic, mixed methods approach was be utilised to evaluate the ACPTSP. The evaluation was structured using Kirkpatrick's four level training evaluation model (1996). Whilst the evaluation was not directly evaluating the impact of training, the Kirkpatrick model was felt to provide a relevant framework to evaluating the impact of the ACPTSP, as its role was ultimately to improve ACP training by providing better support to trainee ACP’s (TACP’s) Data gathered for the evaluation included: • 19 semi structured interviews • A questionnaire to both supplement qualitative data and empirically evaluate the impact of the ACPTSP on TACP’s using validated psychometric instruments. Interviewees were selected from three different groups • Post support package ACP trainees (trainees who started their training on the academic years 2018/19 and 2019/20) • Pre support package (Trainees who started their training in the academic year 2017/18 or before) • Wider stakeholders (other people in the system including, clinical superiors, colleagues, and managers) The above data was also supplemented by thematic analysis of Clinical Academic Support Panel (CASP) interview data. Results and discussion Data from all sources was analysed to identify key themes. Whilst the project was focused on evaluating the impact of the ACPTSP on ACP trainees, inevitably there were many stakeholder comments about wider issues to do with the ACP training regime. The report therefore comments on not just the performance of ACPTSP, but also the ACP training regime and ACP training and development issues within the wider health system. The ACPTSP has made clear differences to the quality of ACP training through a number of innovative measures to: • directly support ACP trainees throughout their training; • ensure trainees receive the support they need from other stakeholders, such as their employers and work-placement supervisors. • address structural weaknesses in the ACP training process. Interventions include: • Welcome/induction meetings for ACP’s and Supervisors • Employer information pack and follow-up meetings • ACP Capability standards for Primary and Community Care, and Mental Health • Allocation of ACP contacts • CASP interviews linked to trainee action plans • Supervisor guidance • Supervisor support sessions • Awareness raising at strategic events • Practice manager information sessions • Support meetings for TACP’s and supervisors • Specific issue employer meetings • Clinical simulations • Extra-curricular educational programme sessions (extra the MSc course). These include, patient led teaching sessions within Primary Care, a 3-day paediatric programme and several other expert and patient led teaching sessions. • Peer support through a buddy scheme in year two of the course. • An optional mental health module with input from employers (50% of primary care attendances are mental health related). • CASP interviews. • TACP training portfolios The qualitative data shows that there were significant differences in the perceptions of ACP trainees had about their training after the ACPTSP was implemented. This is backed up by empirical data. Statistical tests on psychometric questionnaire data show that there is a significant and positive difference in levels of “perceived supervisor support” between those TACP’s who accessed ACPTSP initiatives and those who did not. (Not all TACP’s studying at Sheffield Hallam University and the University of Sheffield are eligible for ACPTSP support.) The wider ACP training regime is still developing however, and significant issues were identified. Participants were generally more positive about their core educational provision than the wider system, such as the support they received from employers. The most frequently discussed issues were: - • Clarity, about training requirements; • Time and workload. Trainees stated that they found the overall workload challenging; • Clinical practice issues; • Inconsistencies in resource allocation; • Employer support; • Governance and structure of the ACP training programme; • Pay and Career Opportunities. Limitations The emergence of the COVID pandemic midpoint through this evaluation made recruitment and data gathering particularly difficult. Despite this a significant amount of data was collected and analysed to give robust results. However, because of the above certain aspects of the research plan were modified. Specifically, the plan to analyse and present data according to the Kirkpatrick four level framework was abandoned. Instead all qualitative data was pooled into a single data set to ensure meaningful analysis and results. Within this care was taken to identify differences in the experiences of pre and post ACPTSP trainees to fully understand the impact of ACPTSP. Conclusions and recommendations. It is clear that the ACPTSP has been effective in both supporting trainees and the wider ACP training programme locally, regionally and nationally. However, there are still many opportunities for improvement. The three most frequent themes in the data were: lack of time, lack of clarity and inadequate support. Despite these issues TACP’s are generally highly motivated and had with lots of ideas about improvements that could be made. It was also clear to see positive evidence that many measures have already being implemented by ACPTSP to address the issues that emerged in the data. Recommendations To conclude we outline some of the suggestions given to improve the ACP programme. These are grouped these together into their main themes. · Clear guidelines should be provided and disseminated to both TACP’s and host organisations about what support, training and other support should be provided by employers. Whilst guidelines do exist, guidance information does not appear to be getting to all relevant people in organisations involved. Action is needed to ensure greater dissemination. · Ensure each frontline service that has an ACP trainee fully understands their obligations and has the requirements in place before an ACP begins their training. · Ensure that those who manage, mentor, and supervise TACP’s are adequately trained and supported. · All ACP trainees should be supernumerary for at least year 1 of their training. · Policy needs to be developed to give clear guidance about ACP terms and conditions and grades paid at particular points of development, linked to specific, roles and job descriptions. · Clear pathways of support are needed for both TACP’s and employers particularly when difficult issues arise (ACPTSP needs to continue to provide this support in the absence of other structures). · Clinical placements should be facilitated by a third party rather than rely on trainees or their supervisors to informally make arrangements. · ACPTSP should continue to act as a champion of the ACP role, advocating its benefits at workforce events. · As more ACPs become qualified and integrated into the workforce, they should become mentors or clinical supervisors for ACP’s in training. · Steps should be taken to achieve more clarity and consistency in pay, support, and career progression. · Provide a framework to clarify how all the different training offered and assessments required fit together · Develop and promote peer support systems and structures to ensure equity and support. · Give clear guidance about the purpose of the portfolio as well as sessions on how to use it effectively. · ACPTSP actions need to be supported by actions at other system levels to maximise improvement of ACP training programme, and integration of ACP’s into the health and social care workforce.