Comparing In Vivo Versus Simulation Training For Transnasal Endoscopy Skills

Fiberoptic endoscopic evaluations of swallowing (FEES) is as important of a swallowing evaluation as the videoflouroscopic swallow study, but far fewer speech-language pathologists are competent in its use (Ambika, Datta, Manjula, Warawantkar, & Thomas, 2019; Brady & Donzelli, 2013; Pisegna & Langmore, 2016). One hurdle in FEES training is the necessity of practicing transnasal endoscopy on volunteers. The primary aim of this study was to compare the learning effectiveness of practicing transnasal endoscopy via simulation with practice in vivo for a student's first passes of the endoscope. The end goal of this study was to determine the most cost-effective and feasible means of teaching transnasal endoscopy to graduate clinicians. Twenty-one graduate students practiced transnasal endoscopy in one of three conditions: in vivo, high-fidelity lifelike simulation, low-fidelity non-lifelike simulation. The learning outcomes assessed were speed of endoscopy, student confidence, and simulated patients' comfort and perception of student skill. There were no significant differences between conditions found for any of these measures. Students in all conditions became more confident after practicing endoscopy, and that confidence was predictive of procedure time. The results of this study indicate that practice with simulation may be an important first step in teaching endoscopy.


Introduction
Fiberoptic endoscopic evaluation of swallowing (FEES) has been considered a "gold standard" in dysphagia evaluation alongside videofluoroscopic swallow studies (VFSS) (Brady & Donzelli, 2013;Heijnen et al., 2020;Hiss & Postma, 2009 ). VFSS is the more commonly used assessment and has a longer history (Cimoli et al., 2019;Langmore, 2017). However, FEES may be a preferable and more effective option for certain populations, such as the critically ill, those postextubation, or those for whom laryngeal physiology is in question (Ambika et al., 2019;Brady & Donzelli, 2013;Noordally et al., 2011). FEES is more sensitive to certain aspects of swallowing function, such as stasis (Pisegna & Langmore, 2016), which may be beneficial for these populations.
Despite its comparable importance as a diagnostic tool, the use of FEES lags behind VFSS, and fewer speech-language pathologists are competent in this assessment. In a 2015 study conducted of Australian speech-language pathologists (SLPs) practicing in the field of dysphagia, 24 percent had completed post-graduate competencies in VFSS, but only eight percent were competent in FEES (Vogels et al., 2015). A smaller, more recent study found that fewer than half as many SLPs perform FEES when compared with VFSS (Cimoli et al., 2019). SLPs attribute the restricted use of FEES exams to the limited number of clinicians trained to complete the assessment (Rumbach et al., 2018).
These Australian findings are similar to a 2003 study conducted in the United States, which found that while 87.5 percent of speech-language pathologists practicing in dysphagia reported competency in completing VFSS, only 3.1 percent were qualified to complete FEES (Mathers-Schmidt & Kurlinski, 2003). In a more recent study of Virginian SLPs, again practitioners stated that their reduced use of FEES was restricted by the dearth of SLPs qualified to complete these studies (Dailey, 2019).

Current FEES Education Methods
As Robinson and Dennick note in their 2015 study of transnasal endoscopy training techniques for SLPs, the traditional method of teaching FEES is not particularly effective in isolation. FEES training for speech-language pathologists is typically provided in one or two-day workshops, followed by mentorship and supervision provided by SLPs competent in FEES. However, Slade (2009) conducted a large survey of speech-language pathologists who went to FEES workshops. They reported that 79 percent did not feel they left with sufficient depth of knowledge to complete endoscopy independently; Sixty-two percent noted that they had little access to mentorship and supervision of experienced speech-language pathologists who were skilled in the technique. As such, the surveyed speech-language pathologists did not feel competent in completing FEES despite their training. Given the paucity of speech-language pathologists who report themselves as competent in FEES, it is no wonder that SLPs have difficulty finding mentorship and training outside of these intensive trainings.
Since SLPs have difficulty learning endoscopy skills on the job, particularly if they do not have easy access to skilled mentors, their graduate training may be the best opportunity for them to begin their supervised practice. The guidelines published by the American Speech-Language-Hearing Association (ASHA) indicate that graduate swallowing courses in speech-language pathology should include education in nasopharyngeal endoscopy for FEES (ASHA, 2007). Although ASHA does not specify whether the information about FEES should be conceptual or through hands-on practice, the graduate setting may be an ideal setting for training due to potential mentorship, supervision, and hands-on practice available. Before an SLP may begin completing FEES with patients with dysphagia, they must first practice transnasal endoscopy on typical individuals (ASHA, n.d.). SLPs hoping to become competent in FEES must therefore find multiple typical volunteers upon whom to practice, placing a significant burden on the student and the mentor. Simulations could feasibly be used to ease the burden of practice on a system where there are already too few competent practitioners (Langmore, 2017;Martino et al., 2004Vogels et al., 2015. The few prior studies of simulation in transnasal endoscopy training have focused on novice student learners (Benadom & Potter, 2011;Berkowitz, 2017;Johnston et al., 2015). Multiple types of simulators have been used to teach transnasal endoscopy, such as medical manikins, task trainers, pool noodles, and glove boxes that are built by the instructors themselves (Bartow et al., 2014;Benadom & Potter, 2011;Berkowitz, 2017). The two prior studies with multiple group designs (Benadom & Potter, 2011;Johnston et al., 2015) have compared high-fidelity lifelike medical manikins, which resemble humans, and low-fidelity task trainers, which do not resemble humans. They have found that the life-like fidelity did not significantly affect students' endoscopy speed, student confidence, or patient comfort (Benadom & Potter, 2011;Johnston et al., 2015). One study, conducted by Johnston and colleagues (2015), compared groups that practiced with high and low-cost simulators with a group that did not practice. However, it is not yet clear whether practice with simulation is comparable to practice in vivo. This study sought to compare the educational efficacy of transnasal endoscopy simulation to in-vivo practice. It also attempted to replicate prior findings that life-like fidelity was not a barrier to learning.
Purpose. Traditional endoscopy training continues to use volunteers for the learner's first endoscope passes prior to supervised practice with real patients (ASHA, n.d.; Leonard & Kendall, 2014). This begs the question whether practice on a simulator may be considered similar to practice with in vivo or merely a precursor to that practice. The purpose of the present study was to assess whether transnasal endoscopy practice with a simulator leads to similar learning outcomes to practice in vivo.
Prior research has indicated that high-fidelity and low-fidelity simulators lead to similar learning outcomes when compared to one another (Benadom & Potter, 2011;Johnston et al., 2015). Therefore, this study compared in vivo training with training using high-fidelity simulators and low-fidelity simulators. Students' speed in passing the endoscope, students' confidence, and simulated patients' self-reported comfort and perception of student skills were compared between groups. It was hypothesized that a student's first few passes of the endoscope would primarily teach them how to handle and advance the endoscope. Since each condition presented this opportunity, there would therefore not be significant differences between groups.

Methods
Participants. Twenty-one graduate students in their second year of a speech-language pathology master's program volunteered to participate in this study. All had completed a prior course in dysphagia and a laboratory prosection of a human cadaver. None of the participants had prior experience observing or completing endoscopy. Manual dexterity was assessed using the Purdue Pegboard Test (Tiffin & Asher, 1948). Results of this assessment were used to divide participants into even groups with equivalent dexterity using summed scores, F(2, 18) = 1.877, p = .182. This controlled for the effects of manual dexterity on endoscopy speed and skill. Eleven participants were assigned to in vivo training (IVT), and ten were assigned to the simulator groups, split evenly between high-fidelity simulation (HFS), and low-fidelity simulation (LFS). The HFS and LFS groups were analyzed together as one "simulation" group to compare simulation with in vivo practice. They were also analyzed separately to replicate prior work and determine whether lifelike fidelity affected outcome measures in the "simulation" group. The Institutional Review Board of Midwestern University -Glendale approved this study, and all participants provided written consent.
Study Design. This study took place over three consecutive days. Each day represented a new phase of the study: teaching, practice, and test ( Figure 1).

Teaching.
On the first day, all participants took part in a two-hour classroom-based lecture. The lecture described nasal and pharyngeal anatomy, correct handling of the endoscope, procedure for completing transnasal endoscopy, a video of transnasal endoscopy being completed, and a live demonstration. Participants were encouraged to ask questions. Each of the participants also had the opportunity to practice handling the endoscope and inserting it into a flexible drinking straw, as described in Benadom and Potter (2011).

Practice.
Participants each had twenty minutes to complete guided passes of the endoscope with instructor guidance and feedback. Participants received feedback on endoscope handling, insertion, and manipulation, as well as body positioning. The HFS group completed these passes on a high-fidelity simulator, the LFS completed them on the low-fidelity simulator, and the IVT group completed them in vivo on a simulated patient.
Test. Participants were randomly assigned to complete transnasal endoscopy on one of six healthy adults acting as simulated patients. Each simulated patient's nasal anatomy was assessed to ensure there was no anatomic variation that would preclude successful passage of the endoscope. No topical anesthetic was employed. Both the simulated patients and the students were reminded that they could end the endoscopy experience at any time without repercussion. A trained speechlanguage pathologist was also present to stop any procedure that could have harmed a simulated patient. In order to ensure simulated patient safety and maximize comfort, participants were allowed a maximum of three minutes from the beginning of the endoscope pass to the visualization of the vocal folds and a maximum of three attempts. Though no student reached either limit, two students did choose to end their endoscopy session prior to reaching the velum and therefore were removed from the study.

Equipment.
High-Fidelity Simulator. The lifelike, high-fidelity simulator used was the METIman ® Adult Patient Simulator (CAE Healthcare, Inc., Sarasota, FL; Fig. 1). This high-fidelity, full-body medical manikin was sat upright in a gurney positioned like a chair. Its nasal cavity opened into a pharyngeal space, and vocal folds and an epiglottis were present at the level of the larynx.

Figure 1: High-Fidelity Simulator
Low-Fidelity Simulator. The non-lifelike, low-fidelity simulator was adapted from Johnston and colleagues (2015) in their study comparing low-cost and high-cost simulators in teaching transnasal endoscopy. Both simulators were created from materials easily found in the home or at a hardware store: plastic pipe fittings for the nasal cavity and pharynx, pieces of plastic drinking straws for the nasal turbinates, and cut thin pieces of plastic to form the septum and laryngeal structures. The simulator in this study differed from that described by Johnston and colleagues (2015) in that the pipe fittings available in the United States of America connect differently from those available in the United Kingdom. However, this difference did not impact the inside of the nasal or pharyngeal space, merely the way the simulator appeared externally (Figs. 2-5). Outcome Measures. Three primary outcomes were assessed: speed of endoscopy, student confidence, and simulated patient comfort and perception of student skill.

Speed of Endoscopy.
Using the video recorded from the endoscope, two timing measures were gathered: full procedure time and successful pass time, adapted from the conventions of Benadom and Potter (2011). Full procedure time (FPT) constituted the total amount of time from first entry into the nare until successful visualization of the vocal folds, inclusive of any false starts or endoscope removal and reinsertion that the participant may have completed due to difficulty passing the endoscope through the nasal cavity. Successful pass time (SPT) constituted the time from nare to vocal fold visualization of the student's successful endoscope pass, excluding any false starts. Timing measures were coded from video using ELAN, version 5.2 (Max Plank Institute for Psycholinguistics, Nijmegen, NL).

Student Confidence.
Participants completed confidence surveys before and after the practice phase, as well as after the test phase of the study. These surveys are labeled survey 1, 2, and 3. Survey 1 and 2 were written in a future tense, and survey 3 contained the same questions in a past tense because it was provided after the test phase. Adapted with permission from Benadom and Potter (2011), these Likert-type scale surveys were comprised of ten questions designed to assess student confidence (Appendix A). Adaptations from Benadom and Potter (2011) included the addition of question 9, the removal of a question that was not applicable to this study design, and a change to future tense for surveys 1 and 2. (2011) (Appendix B) to determine the simulated patients' perception of the students' competence. Simulated patients also completed a visual analogue scale rating of their own comfort during the procedure as adapted from Johnston et al. (2015). Simulated patients indicated their comfort level during the procedure on a continuous 100mm line ranging from "not at all uncomfortable" to "maximally uncomfortable." Figure 6: Study Design Statistical Analyses. One-way analyses of variance (ANOVAs) were conducted to assess for differences in manual dexterity scores, timing measures, student confidence, simulated patient perception of competence, and simulated patient comfort between groups. An alpha value of .05 was chosen for all statistical tests. Repeated-measures ANOVAs were used to analyze the differences in student confidence over multiple surveys. The assumption of sphericity was met for all measures, p >.05, and data were normally distributed. Initially, the LFS, HFS, and IVT conditions were analyzed as three separate groups in order to assess for differences created by the lifelike fidelity of the simulator. The LFS and HFS conditions were then combined into one "simulation" condition for analysis. ANOVAs were run again for each educational outcome between the simulation and IVT groups to ensure any differences between in vivo and simulation training. Both the two-group and three-group comparisons are reported below. Linear regression was used to assess for relationships between the timing, confidence, and patient comfort measures.

Results
Though twenty-one participants volunteered to participate in the study, two participants (one IVT and one LFS) chose not to complete the test phase of the study. Their data were therefore excluded from analysis. There remained no significant difference in overall Purdue Pegboard Test (Tiffin & Asher, 1948) manual dexterity scores between groups, F(2, 16) = 1.98, p = .171.
In vivo training did not lead to significantly different results in comparison with the low-fidelity and high-fidelity simulation groups in speed of endoscopy, student confidence, simulated patient perception of student competence, or simulated patient comfort. This held true whether the HFS and LFS groups were analyzed separately (Table 1) or together as a single "simulation" condition (Table 2).  -df b = degrees of freedom between groups, df w = degrees of freedom within groups A one-way repeated measures ANOVA revealed a significant change in student confidence over the course of the study. Participant confidence levels changed significantly before training, after training, and after the test phase (F(2, 36) = 6.09, p < .005). Post-hoc paired t-tests with a Bonferroni-corrected alpha value of .017 revealed that students became significantly more confident following training (M = 40.32, SD = 4.50) than they were prior to training (M = 37.32, SD = 5.  A simple linear regression indicated that student confidence prior to the test phase of the study did not appear to be predictive of the length of the successful endoscope pass, F(1, 17) = 2.50, p > .05. However, simple linear regression indicated that student confidence prior to the test did predict the length of the full procedure time, F(1, 17) = 8.48, p = .01, with an adjusted R 2 of .29. The model predicts that as student self-reported confidence increased one point, the time of the full procedure decreased by 9.99 seconds. Student confidence predicting the full procedure time but not length of the successful endoscope pass indicates that confidence reduced the amount of time spent in false starts or after having removed the endoscope from the nose after a false start. However, full procedure time and practice confidence were not predictive of student confidence after the test phase (F(3, 15) = 2.51, p > .05) or simulated patient comfort (F(1, 17) = 1.25, p > .05).
Question 7 of the student confidence survey, "The volunteer was comfortable during the procedure," was compared directly with question 4 of the simulated patient survey, "The procedure was comfortable." Spearman's rank-order correlation indicated that there was no correlation between the students' perception of the simulated patients' comfort and the patients' self-reported comfort levels (rs(17) = .33), p = .174). Students' perception of patient comfort is not necessarily indicative of patient comfort.

Discussion
Each study of simulation in this area has indicated that low-cost, low-fidelity simulation models are comparable to high-cost manikins (Benadom & Potter, 2011;Johnston et al., 2015). This study replicated findings from this prior work. However, it is the first of its kind to determine that both of those simulation options provide comparable learning outcomes to in vivo training.
Prior research has indicated that simulation leads to improved endoscopy skill in comparison with not practicing (Johnston et al., 2015). The results of this study indicate that practicing transnasal endoscopy on a simulator may produce comparable outcomes to practice in vivo for a student's first endoscope passes. High-fidelity simulation, low-fidelity simulation, and in vivo practice all produce similar results in terms of student confidence, speed of endoscopy, and patient comfort. Given that the mode of initial endoscopy practice does not appear to affect student confidence or skill, it is likely that the first passes of an endoscope teach the student more about manipulating the endoscope and positioning than about working with patients' individual anatomy.
Additionally, these results indicate that student confidence may have a positive effect on the speed with which they complete endoscopy by reducing the amount of time spent in false starts and reinsertion of the endoscope. Any sort of training, simulation or in vivo, improved student confidence. These results indicate that practice on a simulator may be an important first step in learning endoscopy so as to minimize unnecessary procedure time for volunteers.
The lack of correlation between student perceptions of simulated patient comfort and simulated patients' self-reported comfort was unexpected, although other studies have also found similar disparities between standardized patient and student perceptions (Moineau et al., 2018). It may be that students were so focused on performing their technical skills that they were not attending to cues of discomfort. Another possibility is that the students were not yet sufficiently competent observers of signs of discomfort to make an accurate judgment. Repeated practice and instruction on observation and person-centered care may be an important component in future training.
Not only do the results of the present study indicate that simulation is an effective training tool, but since handmade low-fidelity simulators appear sufficient for training, simulation can also be low-cost and attainable. Therefore, in order to reduce potential discomfort to volunteers, simulation may be considered an imperative first step prior to initiating in vivo practice.

Limitations and Future Directions
One clear limitation of this study is the small sample size. Though this study is comparable with other studies in this area, further research with a larger sample would increase statistical power and better illuminate the generalizability of these results. It is also important to note that these results specifically pertain to the students' first passes of the endoscope. Inevitably, the students will need to learn to adjust their techniques due to differences in anatomy, variations in client posture, and other variables which change from patient to patient. Repeated practice on one task trainer is unlikely to produce the same skills as practicing on a variety of volunteers, just as practicing repeatedly on one person would not.
If the first practice passes of an endoscope truly teach the student more about manipulating the endoscope done about the procedure itself, it stands to reason that there must be a threshold at which simulation is no longer an effective practice method. Unfortunately, that threshold has not been tested. Future research should attempt to determine the limit at which simulation reaches its maximal benefit.
This study only focused on the manual techniques of passing the endoscope. It did not evaluate students' diagnostic or interpretation abilities when completing a FEES assessment, undoubtedly the more skilled portion of the procedure. Indeed, no evaluation of swallowing was completed during this study, only the process of transnasal endoscopy.
Additionally, it is also not clear what role, if any, practicing endoscopy on an unknown person may have had in this study. Since SLPs first practice transnasal endoscopy on volunteers, those first passes are often done on people that the clinicians know. That relationship may have an impact on their confidence and therefore their speed. Future studies should look at whether that relationship is impactful in the learning process.
Implications for Training. Though allowing students the opportunity to practice endoscopy in vivo may be difficult to complete in graduate programs, practicing transnasal endoscopy simulation may be achievable. The results of this study indicate that such simulation is worthwhile. In addition to potential difficulty recruiting volunteers yearly for a training, in vivo training is expensive and time-consuming. It requires frequent disinfection of endoscopes and rotating of volunteers to reduce nasal discomfort. It has not previously been clear, though, whether practicing on a simulator was a similarly meaningful learning opportunity. It appears that such practice is similarly effective in teaching beginning endoscopy skills. If graduate SLP programs choose to take on the cost of endoscopy equipment, they may be able to provide valuable endoscopy education without needing to incur the repeat expense or liability of in vivo practice. Similarly, states that require a specific number of endoscope passes on typical volunteers may wish to consider accepting simulation for a portion of those passes.