ORIGINAL RESEARCH
How do vascular surgery trainees optimise simulation-based learning? A qualitative study
Maguire SC,1 O’Callaghan AP,1 Traynor O,1 Strawbridge JD,2 Kavanagh DO1
Plain English Summary
Why we undertook the work: Surgeons need unique skills for their job, especially to perform operations. These are based on experience, but getting more experience is limited by how many patients they see. This can take many years, so there are other ways to make this faster. Practising with models or simulators can help. Simulators are used to help train surgeons, but we want to do it in the best possible way. There are a lot of factors that make simulation more effective. We wanted to find some of them.
What we did: We spoke to 10 hospital doctors who all wanted to be vascular surgeons, and who had worked with fully qualified vascular surgeons. We spoke to them one by one about learning with simulators, asking them all the same questions. The questions looked at what they liked in the training sessions, and what they found most useful.
What we found: All doctors said simulation was important, especially for practising operations. They liked when their teachers watched and told them what was good while they did it. A report at the end was also useful, but making simulators look real was not important to them. They wanted to practice what they do most often, nothing rare or complicated. They liked getting information before classes, or homework to help them learn as much as possible.
What this means: Getting the basics right was most valuable to them. Big operations that are done often are most important for training. They said: “Practise only the hardest parts of these operations, give homework before class, and have a lot of teachers”. Teachers should talk to the training doctors during simulation, not just at the end. They welcomed as much feedback as possible.
Abstract
Background: Technical skills acquisition in modern surgical training is augmented by simulator-based education outside the operating room. Aims: This research sought to explore and understand optimal learning conditions for vascular surgery trainees during simulation from their perspective.
Methods: Ten doctors at various stages of surgical training were recruited to participate from various training events. Semi-structured interviews were conducted individually on a date beyond initial recruitment. Topics included perceptions around how participants learn best from simulation and what they value, prioritise and require from simulation-based training. Initial data analysis was conducted shortly after initial interviews took place, and further interviews were conducted until data saturation was reached. Data were extracted using inductive thematic analysis.
Results: Participants believed simulation was important within surgical training, and that progression towards competence in core technical skills was particularly important. They appreciated direct consultant trainer supervision, contemporaneous bi-directional verbal feedback, and summative written feedback. Visual fidelity and complex/rare procedures were low on their priorities, with a strong desire to optimise fundamental skills acquisition. Participants also felt strongly about the benefits of advanced receipt of a detailed outline of session content and relevant reading material to maximise the educational benefit of training sessions.
Conclusions: Core operative skills and commonly encountered major operative cases should be prioritised in vascular technical skills training. In designing simulators, the main focus should centre on specific skill acquisition as a procedural component rather than aiming to replicate an entire procedure. A flipped classroom model of pre-reading, low student-to-trainer ratios, and detailed feedback from consultant trainers should be encouraged.
Introduction
Vascular surgery has significantly evolved since the turn of the 20th century with a much wider repertoire of increasingly complex procedures expected of the modern surgeon, thus leading to increasing specialisation. Traditionally, vascular surgery was a sub-speciality within general surgery but it has now evolved into its own distinct speciality. In tandem, operative exposure and potential experience for trainees has reduced due to many factors, including increased number of trainees, an expectation for more direct consultant-delivered care and minimum rest periods, leaving newly qualified surgeons with less experience than their senior peers at entry to consultant-grade level; yet equally high expectations for standard of care are appropriately expected.1
In an effort to counteract this shift, various strategies have been adopted with a view to maintaining a high standard of surgical care, thereby compensating for reduced training opportunities. Skills workshops and simulation-based training have emerged as one such area in achieving this goal.2,3
One of the first comprehensive descriptions of modern simulation-based training in vascular surgery was outlined by Bismuth et al in 2012.4 He detailed the development of a vascular surgery ‘bootcamp’ in use in the USA. This method of surgical training was developed based on models from simulator manufacturers, who run courses around the world in addition to developing simulation models. By their own admission, standards in vascular education had not been formally defined at that time so they sought to set their own. Over time, educational theory and curricula continue to evolve. The questions now tend not to be ‘what’ to learn rather than ‘how’ to learn.5
This paper explores technical skills simulation training from the trainee perspective. There exists many perspectives from the trainer, but how much the trainee benefits or what might help them further remains to be fully elucidated.6 Using established qualitative methods, strategies to further enhance trainee skills and knowledge acquisition were explored. Much of this research is based on pre-Covid experiences and assumes training is delivered without pandemic-related restrictions.
Methods
The research team believed that, in the context of a rapidly evolving educational and clinical landscape, this study was best undertaken with a pragmatic approach (with an inclination towards constructivism) as many of the research answers sought are based on perspectives and opinions in the context of current resources, rather than fixed objective metrics.
This study used qualitative research methods through semi-structured interviews to gather data, an established research method employed when data sought is in the form of subjective opinion, perceptions and individual views.7
Non-consultant hospital doctors (NCHDs, also known as junior doctors) working in vascular surgery roles in Ireland, regardless of training scheme status, were recruited via gatekeepers using purposive sampling from multiple open training days and conferences and consent was obtained in writing. Such training days are mandatory for those on a formal training scheme, but those in non-training positions are also encouraged to attend. Clinical experience or seniority is not a barrier to participation in such events.
Independent gatekeepers (who had no other involvement in this research) issued open verbal invitations to groups attending these training sessions; volunteers then self-presented to an identified researcher for enrolment. The attendance profile, while inclusive of those not in training, predominantly comprised those on formal training pathways for whom such sessions are designed. All included participants identified a desire to pursue a career in vascular surgery.
An interactive discussion via telephone was later conducted with the first author and individual participants at a time convenient for them, lasting for up to 45 minutes. A theme sheet (see Appendix 1 online at www.jvsgbi.com)) was followed to guide discussion, but deviations from this template, where initiated by the participant, were followed for maximal data yield. All topics on the theme sheet were covered although, where deviation from the structure occurred, omitted topics were discussed at the end.
Discussions were recorded via voice recorders, which were transcribed verbatim into Word documents by the first author at the earliest opportunity. Participants were given the opportunity to review, edit or erase the transcriptions. No participants chose to amend the interview transcripts.
Data analysis took place on the anonymised transcripts using NVivo 12 to ensure that the processing of data was transparent and protected participant confidentiality. Initial analysis began following the fourth participant’s interview and continued in tandem with interviews thereafter. Thematic analysis was undertaken using a six-step framework (data familiarisation, coding, searching, reviewing and defining themes, then production of a report).8 An inductive method of generating codes was used to maximise data yield.
Systematic processing of the transcripts was followed, with iterative analysis of data. Within eight interviews the yield of new codes was very low. A further two interviews took place to confirm the likelihood of approaching data saturation. No substantively new opinions or perspectives manifested in these final interviews, thus no further recruitment or interviews took place.
Results
One core surgical trainee (CST, undergoing basic training, pre-MRCS), one registrar (post-MRCS, not on training scheme) and eight Specialist Registrars (SpRs) took part in this research. There were 18 SpRs in Ireland at the time of writing. These are doctors who have completed basic surgical training and are undergoing formal training in vascular surgery through the national training scheme. A similar number of doctors not in training posts also work as registrars in vascular surgery. The number of CST trainees in vascular surgery at this time was no more than three. Each event during which recruitment was conducted hosted up to 20 doctors, with substantial overlap; many attended the majority of national educational events scheduled. These categories of doctors represent the full range of vascular surgery NCHDs in Ireland prior to qualification as a consultant surgeon. They mirror the experience of residents at all stages of the residency program in North America. They had a combined experience of over 25 years working full-time in vascular surgery, varying from 6 months to just under 5 years (mean 2.5±1.3). The gender split was seven males to three females; the current trainee split is approximately 6:4.9 Individual participant names have been anonymised and are referred to as P1–P10.
Recruitment continued at educational days and conferences until apparent data saturation was reached and recruitment significantly dropped off (most attendees had already signed up). This was achieved after five events, which included almost all formal training days taking place over a 12-month period.
Good understanding of what is meant by vascular simulation was demonstrated, with user-generated definitions putting an emphasis on recreating “a real-life scenario” (P1) with “models to replicate vascular procedures” (P9) in “a practice environment” (P6) to “improve technical surgical skills” (P7) “so the trainee can become familiar with them without risk to the patient” (P3). The type of model or simulator was frequently not mentioned, but all agreed this could include physical models, interactive digital technology (such as endovascular simulators) or biological tissue (animal or cadaveric).
Six themes and 13 sub-themes were identified as shown in Table 1.
The training journey
Most participants highlighted the changing context of healthcare delivery in Ireland as a key consideration in the need for technical skills simulation. Exposure to practical experience is perceived to have fallen, especially for open surgery, and this was a significant concern for many.
It was felt that a reduction in live operative exposure was due to reduced “working time with EWTD” (P4) and that “medico-legally it’s harder to hand off cases to junior staff.” (P7) However, they were keen to stress that “there’s no real substitute for the real thing” (P1); simulation needed to “complement and add to real life operating” (P8), not replace it.
Workshops needed to acknowledge training levels without needing to re-run for different levels of experience: “If you’re going to practice on something, start off easy, get it down, then move onto a better model, like a calcified aorta, for anastomosis, so you can perfect and practise the skills.” (P4)
Participants also all stated a perceived need to augment their training through extra paid courses: “… the other ones you hear about from other people and you think they might be good…” (P2)
Gaining value from simulation
Repeated deliberate practice of core skills was valued by participants and identified as very important. “I think until you have the common skills, until it becomes second nature and you develop muscle memory, the complex skills probably aren’t as important … knowing how to troubleshoot your way out of trouble… is more important.” (P9)
Participants felt that the simulators themselves do not need to be high-fidelity to confer benefit, especially early on. “I think you can gain something from each of them … it’s the technique, so the tissue doesn’t have to be very lifelike.” (P10)
Ergonomics were highlighted as an overlooked area, as some felt positioning might be easier compared with real life, which may not be beneficial. “… perhaps the positioning might be a little bit easier than in real life, with patient body habitus and things like that.” (P8)
Other environmental factors such as the atmosphere generated by trainers was highlighted by participants and can be adapted depending on the goals of the session. “The best part of the simulation is that you can afford to make a mistake, so it doesn’t really matter.” (P2)
There were some benefits described in relation to small groups as they afforded an opportunity for trainees to interact with their peers and highlight technical experiences from their training to date. Prior challenges were shared and solutions were proposed.
Technical skills development
With the passage of time, participants further develop their skills and find “there are definitely little tips and tricks … which I wouldn’t have known before.” (P9)
These more subtle elements are the things not available elsewhere which participants really value. “Very useful to learn new techniques and get different perspectives on the same thing.” (P5)
Technical tips to enhance core skills were highly valued and were felt to be more important than moving on to more rare procedures, even if they are more interesting. “I much prefer to go over doing a carotid 10 times while they’re looking at you rather than looking at something weird and wonderful you’re only going to see once or twice.” (P7) “… we don’t need to be spending hours learning how to do something unusual.” (P8)
Trainers
Most workshop faculty were practising surgeons. “I would say 80% would be consultant surgeons, then the rest would be trainees, reps, and radiologists.” (P8)
All but one participant reported that knowing them professionally helped them learn. “It’s a good thing, it doesn’t make you more anxious, puts you at ease.” (P6)
Interviewees felt that small groups with lots of opportunities to practise technical skills created the best learning environment. “I think having more than two trainees on a simulator (at any given time) is a little bit excessive” (P7). “I like it when you have a consultant watching while you are doing a procedure, I think that’s where I learn the best.” (P2) This allows for tailored feedback.
Summative feedback was also seen as useful, but only to compare with the next time. “I would like to receive written summative feedback given to me after. This could be used as a standardised scoring system.” (P4)
Practical elements
Participants felt that striking a balance between didactic talks to provide context and instructions for simulator use while maximising operative time on simulators was of utmost importance. Sessions with excessive didactic sessions describing personal experiences were felt to be less beneficial. “Brief didactic instruction and then prolonged sessions on the simulator is where I gain the most out of it.” (P10) “If the program agenda is rigidly structured, you have to finish it in time, and the consultants are rushing you through everything, I don’t think is beneficial.” (P3)
All participants wished to receive pre-reading material to enhance the educational benefit of the day. “Give them material in advance so they know what’s coming up. Provide resources for the procedures, and the level they’re expected to get.” (P9)
Other areas for further development
Participants highlighted a wide variety of other procedures and skills they would like to learn, but no consistent patterns emerged. These included endovenous skills, ultrasound, rib resections, graft tunnelling, flaps and fistulas, among others.
Participants also wished for greater access to cadaveric material. They felt this was only available on fee-paying courses, which are often based outside Ireland thereby incurring additional costs. They did, however, also acknowledge the financial limitations of training budgets, necessitating additional external funding, often from participants themselves, for this type of educational experience.
Occasionally, more focused days based on training level was highlighted as potentially beneficial to complement the core skills days. “… there’s such a wide audience of every level, including SHOs, that they’re a little broad.” (P8)
Discussion
Core technical skills were acknowledged as one of the most important components of simulation training, reinforcing the view that ongoing acquisition and retention of technical skills competence extends for years, if not indefinitely. Similarly, skills degradation can happen relatively quickly. This is borne out in the literature, where core competence in certain procedure-specific skills is reached quickly but technical skills continue to evolve and develop during one’s time as a consultant surgeon.10 Learning about rarer procedures can also contribute to the overall motivation for attending a workshop; it encourages engagement and motivation during the more ‘everyday’ skills practice.11
Physical and tactile realism is frequently highlighted as an area in which technical simulation is limited.12,13 However, many felt realism was not as important, provided the model was able to teach the specified learning objectives. In this regard, tactile fidelity was appreciated but acknowledged as a bonus, but simulated blood flow in particular was highlighted.
No participant expressed a desire to be able to complete a procedure in its entirety, acknowledging simulator limitations and knowing which parts of a procedure need further practice – for example, anastomosis versus wound closure.
Using simulation within training was universally seen as positive, although participants were cautious not to overstate the potential benefits, especially if it was seen to replace rather than supplement training. This opinion was shared with British and American trained surgical trainees elsewhere, who are positive but warn about potential over-reliance on simulation, especially for certification of competence.14,15
For senior participants, the opportunity to see how trainers perform procedures in ways that are different from what they previously learned was seen as a significant positive component of simulation sessions. Under Kirkpatrick’s educational framework, while approaching the level of mastery for a technique is demonstrated by unconscious competence (level 4), such experiences allow further acquisition of skills at conscious competence (level 3).16-18 This can broaden the proficiency of a future independent surgeon so that they can approach a challenging case with a comprehensive skillset.19
Refining core skills was most highly valued by trainees as a worthwhile use of simulation time. The importance of maintaining core competencies is seen as essential, and can further develop over years. While competence can be achieved relatively quickly, mastery takes much longer, at a stage beyond entry to consultant level practice.10
While Ireland’s small surgical population limits the viability of organising separate training days for different levels of experience, small group breakout sessions within training days serve to reconcile training needs according to levels of experience. While participants felt spending valuable simulator time on rare procedures was not the most efficient use of training time, they still did want to know about them. The degradation of certain technical skills over time likely influenced much of their perspective; it is well established that, in the absence of regular surgical practice, technical competency wanes.20 Participants were keen to know how one might deal with rare situations even if they did not practise them in the simulation.
The question regarding the need for realism is a complex one. Balancing the use of synthetic models, which are durable, widely available, ethically accessible and re-usable, against the higher fidelity but ‘single use’ animal or cadaveric tissue can be challenging. The latter can pose significant ethical and health and safety challenges for the skills laboratory.21 Participants expressed a desire for greater fidelity systems, but also acknowledged the practical limitations therein.
However, they did consistently wish for greater utilisation of models containing pulsatile ‘vessels’. These systems are often available as part of an add-on to synthetic models, but cases have been described where pulsatile perfusion can also be established in cadaveric and animal models.22 These can be effective, but often only last for a limited time before perfusing fluid leakage causes significant tissue degradation, reducing their useable lifespan.
Higher complexity models and greater use of biological material is often not feasible for a training scheme, either due to ethical, facility licencing, economics, or a combination of same. Such training may only be achievable in other jurisdictions and usually requires extrinsic funding, either through sponsorship or supplemental trainee contributions.
The participants appeared to need little external motivation in signing up to additional training independent of the standard curriculum, even when funded by the trainee, some of which can be refundable.11 Part of the motivation for signing up to additional courses was stated to be based on access to cadaveric and animal tissue. Many felt they could not access training with high-fidelity tissue within the existing training structure.
Participants appreciated any feedback they received, with personalised insights into skills and techniques being particularly valuable. The use of summative scoring systems was much less prevalent and something they would be interested to see more of for benchmarking purposes. Universal scoring systems have proven challenging to develop, with Objective Structured Assessment of Technical Skills (OSATS) being one of the primary methods used in many different countries.23-25
In line with the concept of spaced learning and the flipped classroom, many participants expressed a desire for pre-reading and homework. They acknowledged the high quality of material during the session, but felt if difficult to retain due to the high volume. Advance reading and at-home re-enforcement of this information acquisition could promote greater retention and understanding.26 This reflects components of the andragogical model of adult education.27 Within this framework, self-motivation and purposeful learning is key and should be encouraged but requires input from course providers in advance.
Study limitations
This study was limited to the Irish system of surgical training. The educational context should be considered if applying these findings in other jurisdictions. The source of recruitment via conferences and training days disproportionately recruited those already on a training programme and under-represented doctors in ‘non-training’ posts. Training days are compulsory for those on a training scheme but optional for others.
The first author in this study undertook this research during academic time between ‘non-scheme’ and formal training appointments in vascular surgery. While we believe this provides an invaluable perspective and ability to empathise with participants’ experience, it may also introduce certain bias or lack of external perspective due to similar shared experiences. In order to help mitigate these potential biases, an external researcher not involved in surgical training (JDS) took special care to observe for unconscious bias and retain more external objectivity.
Conclusions
As simulation-based training becomes ubiquitous and central to professional surgical education, the shape and delivery of such training needs to evolve.
Focusing on core skills with hands-on experience in small groups with professional trainers is highly valued. The current study identifies many areas for improvement including access to biological material and provision of pre-reading/homework to consolidate and maximise educational yield.
Exactly what to provide, and how best to provide it, remains an ongoing question for all education providers. There is never one right answer, but we can, with experience, audit and self-reflection, refine our strategies to increase educational effectiveness.
Article DOI:
Journal Reference:
J.Vasc.Soc.G.B.Irel. 2024;3(4):239-244
Publication date:
August 27, 2024
Author Affiliations:
1. Department of Surgical Affairs, RCSI House, Dublin 2, Ireland
2. School of Pharmacy, Ardilaun House (Block B), Dublin 2, Ireland
Corresponding author:
Seán C Maguire
Department of Surgical Affairs, RCSI House, 121 St Stephen’s Green, Dublin 2, Ireland
Email: [email protected]
