The first virtual ASPIRE 7 and 8 programs: preparing senior vascular trainees for FRCS and consultancy using a Post-Graduate Virtual Learning Environment

Background

The COVID-19 pandemic has had catastrophic effects in the UK. The death toll in the UK currently stands at over 170,000.¹; The COvid-19 Vascular sERvice (COVER) study highlighted the significant impact of the pandemic on both elective and emergency vascular surgery provision, with the focus on minimising exposure and therefore cross-infection of COVID-19 and its associated morbidity and mortality.² The effect of COVID-19 on vascular surgery operative training and Annual Review of Competency Progression (ARCP) is also well documented. A 40% reduction in elective operating and 5–10% reduction in emergency operating was reported when comparing 2019 to 2020 elogbooks. The number of outcome 1s recorded in vascular ARCPs reduced by more than 17% in absolute terms and one in five trainees received an outcome 10. Whilst the majority of these were an outcome 10.1 (not requiring an extension to training), the attainment gap that exists for those trainees still needs to be addressed.³ A key progression point in surgical training is the Fellowship of Royal College of Surgeons (FRCS) exam. During 2020 FRCS exams were cancelled due to the COVID-19 pandemic. These were then re-introduced in vascular surgery, initially in a remote delivery format without patients.⁴ Once trainees have completed this exam and enter their final year of training, it is of key importance to prepare them for both the clinical and non-clinical aspects of a consultant’s working career. Prior to the pandemic, the national ASPIRE-7 and ASPIRE-8 courses run by the Vascular Society would support trainees in preparing for the FRCS exam and consultant life, respectively. These would be face-to-face two-day courses. The COVID-19 pandemic therefore required a new strategy for supporting trainees near the end of their training. Learning Management Systems (LMS) have been used commonly in undergraduate education. In Health Education England West Midlands (HEEWM), a strategy for not just maintaining but enhancing the delivery and governance of teaching programs in multiple specialties across the West Midlands was developed. A Post-Graduate Virtual Learning Environment (PGVLE)5 using Moodle™/BigBlueButton (BBB) open source but commercially hosted software was established to support local/regional teaching programs in the West Midlands. This, however, had not been used to support national training schemes. The ASPIRE program organisers, in collaboration with HEEWM, looked to evaluate the utilisation of the PGVLE in delivering the first purely virtual ASPIRE-7 and ASPIRE-8 national courses.

Aim

The aim of this study was to both establish and evaluate a model for the delivery of the ASPIRE-7 and ASPIRE-8 post-graduate vascular trainee courses using the PGVLE.

Methods

PGVLE

A course page for both ASPIRE-7 and ASPIRE-8 was created in collaboration with the HEEWM PGVLE team. Faculty and candidates were given log-in details, a pre-course guide for navigating the PGVLE and links to two video tutorials on how to use the BBB as a trainer and a trainee, respectively. Timetables for the courses were uploaded to the PGVLE course pages for faculty and candidates to view. Pre-course materials were developed by the faculty and made available to candidates on the PGVLE course pages. Restricted access faculty areas were created on each course for resources that would be used for the mock scenarios, to prevent candidates seeing the resources while being freely accessible to faculty. Separate anonymised feedback forms for candidates and faculty were created on the PGVLE for both courses. Once completed, this unlocked separate candidate and faculty certificates which autogenerated the details of the candidate/faculty member. BBB conference rooms and BBB small group seminar rooms were created to deliver the virtual teaching sessions.

ASPIRE-7

In March 2021 a two-day virtual course was planned by the organising faculty that looked to outline the structure of the new virtual FRCS exam using mock stations. Twenty key vascular articles on aortic disease, carotid disease, venous disease, secondary prevention, peripheral vascular disease, diabetic foot and amputation were made available on the PGVLE course for candidates to access prior to the course. Eight pre-course lectures for candidates were also developed by the faculty, for which the topics were:

– What the exam entails
– How to pass the FRCS
– Managing infected grafts
– Carotids – who, what, when and why
– Deep venous intervention
– Vascular trauma
– Vascular access
– Abdominal aortic aneurysm management

Both days started and finished with an introduction session in a main BBB conference room, which included a briefing and opportunity for questions and answers (Q&As). Time was also allocated at the end of each day for a consolidation and Q&A session. Day 1 included 6 hours and 20 minutes of short and long case mock sessions split into a morning and afternoon session. Ten breakout seminar BBB rooms were created, each containing two faculty members with candidates in groups of four rotating around short and long case stations. Day 2 included a morning group session of half the course candidates in a ‘Journal Club’ with two faculty, discussing the 20 key papers in a BBB seminar. The other half of the candidates were split into groups of twos or threes and rotated around oral viva sessions with two faculty in each of the eight BBB breakout seminar rooms. Both sessions lasted 3 hours and 10 minutes and the two groups swapped for the second half of the day. In order to allow faculty to converse in breaks, a faculty BBB ‘common room’ was also created with access restricted to faculty only.

ASPIRE-8

A two-day virtual course was developed and delivered in June 2021. Several important lectures were delivered in a BBB conference room which included:

– Becoming a consultant and the application process/job planning
– Developing non-clinical roles within consultant practice
– Understanding equality and diversity challenges
– A medical directors’ perspective on “What I want from a consultant vascular surgeon”
– A chief operating officer’s perspective on “What management wants from a consultant vascular surgeon”
– Becoming a consultant trainer
– Legal pitfalls and consent
– The future of vascular surgery
– NHS interaction with industry
– How to survive the first year
– Introducing a new service
– How to run an MDT and aortic practice
– Private practice now and in the future
– Who got the job?

Candidates each underwent a mock interview and delivered a pre-prepared mock presentation on a variety of topics in a BBB interview room.

Analysis

Quantitative and qualitative data were collected from candidates and faculty on both ASPIRE-7 and ASPIRE-8 using the PGVLE integrated feedback system. Faculty and candidates’ previous use of the PGVLE was assessed. A 5-point Likert scale (1=unsatisfactory, 2=below average, 3=average, 4=good, 5=excellent) was used to evaluate the candidates’ perceptions of the course lectures, mock stations and overall evaluation of the two ASPIRE courses. Candidates were also free to give qualitative feedback on the mock scenarios and overall impressions of the courses. The same Likert scales were used to evaluate the candidates’ evaluation of the PGVLE and BBB. Faculty were asked how strongly they agreed with statements pertaining to the ease of use of the BBB and PGVLE as teachers, with data gathered using a 5-point Likert scale (1=strongly disagree, 2=disagree 3=neither agree nor disagree, 4=agree, 5=strongly agree). The same scales were used to assess their perceptions of the pre-course preparation materials and whether a pre-course induction of using the PGVLE/BBB would have benefited them. Qualitative data on faculty experiences were also gathered. Data gathered from the PGVLE users were gained with their consent as an agreement in the terms and conditions of the PGVLE and were compliant with General Data Protection Regulations.

Results

ASPIRE-7

Candidate quantitative results
Thirty-nine UK FRCS vascular candidates attended the two-day virtual course; 36 (92%) completed feedback, of which 26 (72%) were new PGVLE users. Pre-course lectures received a 91% good/excellent mean rating when those who did not watch these lectures were excluded (Figure 1). The four small group mock stations all received a 100% good/excellent rating and the journal club 94% (Figure 2). The overall educational value of the course received 100% good/excellent rating, as did the BBB, and the PGVLE received a 97% rating (Figure 3).

Candidate qualitative results
The key themes were as follows: Level and pitch: five trainees commented that the ‘pitch’ or ‘level’ was correct: “Well structured and pitched at the right level”. Variety of content and examination style: seven candidates stated that the course had a good variety of cases, four stated that there was repetition and one stated “Good short cases and varied, however multiple repetitions in scenarios”. Technology and virtual format: Four comments were made regarding technical issues such as: “The image quality was compromised”. However, several highlighted positive experiences of the virtual format such as: “I was pleasantly surprised at the fact that it seemed to work very well virtually and I think this is probably a feasible model moving forward” and “I would be happy to have future courses in this platform as I saved on travel time, having to make family arrangements and cost. I would like this platform to continue”.

Faculty quantitative results
The course was supported by 27 consultant vascular surgeons spread across the UK, 21 (78%) of whom completed feedback. Sixteen (76%) were new PGVLE users. 100% of faculty felt that the BBB was easy to use for delivering virtual teaching and 90% agreed that the PGVLE was easy to use and navigate (Figure 4). 90% of the faculty used the pre-course guides on the PGVLE and BBB. Of these, 63% found the pre-course guides useful and 52% felt that a PGVLE/BBB faculty induction course would be useful (Figure 5).

Faculty qualitative results
Several consultants gave positive views on the PGVLE and BBB such as: “This course worked very well in this format”. There were a couple of comments asking “bring back face-to-face”. There were 14 positive comments regarding the course from faculty: “I think it was perfectly executed” and “I can’t see how it can be better”.

ASPIRE-8

Candidate quantitative results
Fourteen UK vascular trainees in the final year of training attended, of which 11 (79%) completed feedback. Seven had previously used the PGVLE (64%). Mean lecture good/excellent rating was 95% and the mock interviews and presentation sessions received a 100% rating (Figures 6 and 7). The ASPIRE-8 course received a 100% good/excellent rating for educational value, the BBB and the PGVLE (Figure 8).

Candidate qualitative results
Technology and virtual format: There were six positive comments regarding the technology used to support the course including “thought the tech was excellent” and “easy to use”. There were also comments of ‘glitches’ but also an understanding from trainees regarding issues; for example: “Although there were technical issues with slide control, I believe this is expected with new platforms and will get better with time”. Content: There were 13 separate positive comments such as: “It would definitely help my preparation to become a consultant” and “It has highlighted several areas for me to work on which I would not have previously considered”.

Faculty quantitative results
ASPIRE-8 was supported by 14 faculty members which included vascular consultants, a NHS medical director, a NHS chief operating officer, a legal director and vascular industry marketing manager. Of the faculty, five (36%) completed the feedback and, of these, four (80%) had used the PGVLE before. 80% of faculty found the BBB easy to use and 100% found the PGVLE easy to use (Figure 9). 100% of faculty agreed that the pre-course materials were useful but there was a mixed response regarding the requirement for a pre-course induction (Figure 10).

Faculty qualitative results
There were minimal faculty comments but key points included the future need for a virtual ‘waiting room’. One faculty member stated that “Pre-course training not essential but would be useful”.

Discussion

The COVID-19 pandemic has fast-tracked the uptake and the development of virtual learning and online resources. From an organisational perspective, the PGVLE supports several key aspects for delivering national education programs such as ASPIRE. Firstly, it acts as a central point for faculty and trainees to interact, whether that be through integrated web conferencing software such as the BBB, online forums or through resource sharing/development. In the ASPIRE programs, pre-course materials were developed by the faculty and uploaded to the PGVLE for trainees. Collaborating enthusiastic educationalists over time can go on to accumulate a significant information repository for their learners without the difficulties that can be met with sharing on cloud-based storage systems that commonly fall under the care of a single education lead. This can then be used for future iterations of education programs. Another key advantage of using a LMS such as the PGVLE is reducing the administration around attendance, feedback and certification. This course was set up such that once the course was complete, feedback was made available for a short period of time (to reduce recall bias) and only once this was completed were template certificates that autogenerated faculty/candidate names able to be accessed by the candidates/ faculty. For frequent courses/programs, this saves time and therefore allows more of a focus on course development rather than administration tasks. Feedback can then be automatically downloaded for review by the faculty and graphs of the data autogenerated by the LMS, which again reduces the administrative burden of running large-scale education programs. Lastly, the virtual nature of running national courses means that candidates can stay at home, for which the advantages were highlighted by one candidate on this course: “I would be happy to have future courses in this platform as I saved on travel time, having to make family arrangements and cost”.

Limitations

Firstly, this course was developed during the second COVID-19 wave, when-face-to face interaction was not an option. Educational opportunities may have been sparse and so results could be overtly positive. Whilst there were no comments from trainees stating that they preferred the face-to-face format, there were multiple faculty comments stating face-to-face was a better format. In this iteration of the ASPIRE 7 and 8 programs, it was not possible to perform a comparative analysis of face-to-face versus virtual teaching, but this should be considered in the future. Another methodology that could be considered is a hybrid approach, where pre-course reading/resources are viewed by candidates, meaning that focused educational sessions that some may consider better or easier to deliver face-to-face can be provided. Response bias could also result in skewed positive findings, as candidates feel pressured into giving feedback, particularly as this is associated with auto[1]certification. Finally, in the ASPIRE-8 program, the faculty response rate was low (36%) whereas, in all other aspects of the course, feedback completion was above 75%. This could lead to non-response bias, particularly as the majority of the ASPIRE-8 faculty (80%) had used the PGVLE before and therefore may have found the experience of using the PGVLE much easier than new users.

Conclusion

Using a collaborative approach between HEEWM and the ASPIRE program organisers, the PGVLE was used to support two novel virtual national vascular education programs. Results from candidates and faculty showed that the pre-course materials, functionality of the software and content of the courses were highly rated.