ABSTRACT

We ascertained the opinions of residents and faculty regarding technical skills decay during non-clinical training years and evaluated the effectiveness of a technical skills refresher curriculum (TSRC) offered to residents in the month prior to rejoining clinical training years. 32 faculty and 14 residents completed surveys which gleaned opinions regarding technical skills decay during non-clinical years. Six residents completed a TSRC during the month prior to rejoining clinical training. We compared clinical evaluations of the residents who completed the TSRC to residents who rejoined clinical training prior to the implementation of the curriculum. Surveys indicated that residents and faculty believe that non-clinical years have a slightly negative impact on technical skills and residents who completed dedicated research years would require up to 4 months for technical skills to return to the level of their non-research peers. Residents who completed the TSRC reported having significantly higher comfort levels with their technical skills after the curriculum (p < 0.048). Clinical evaluations of these residents were significantly higher than the cohort that completed research years prior to curriculum implementation (p < 0.041). The TSRC is a viable method of preparing residents for rejoining clinical training.

1. Introduction

At our institution and many others, general surgery residents often elect to complete one or more years of research after the PGY2 clinical training year. The reasons for pursuing research may include enhancing one’s curriculum vita, improving one’s chances of obtaining a competitive fellowship, a desire to pursue a career in academic surgery, or for lifestyle reasons. Anecdotally, faculty and residents believe that technical and clinical skills will suffer during these non-clinical training years, however, the degree to which technical skills decay has not been systematically studied.

This study served two purposes. First, we aimed to ascertain the opinions of residents and faculty regarding technical skills decay during non-clinical training years. Second, we evaluated the effectiveness of a technical skills refresher curriculum in terms of self-reported utility and clinical evaluations completed by faculty.

2. Methods

2.1. Resident and Faculty Opinions of Technical Skills Decay during Non-Clinical Years

In order to address the first aim of this study, we issued surveys to residents and faculty to glean their opinions regarding technical skills decay during non-clinical years. The surveys assessed the estimated degree to which technical skills decay during research years (5-point Likert scale) and the amount of time required for technical skills to return to the level of their peers who did not conduct research (7-point Likert scale). Our program has a total of 66 residents, 20 of whom have conducted or are currently conducting dedicated research years. These 20 residents were asked to complete the survey. Residents who have not or did not conduct dedicated research years were not asked to complete the survey because they would not be able to accurately judge the impact of nonclinical year(s) on technical skills. Resident survey questions are displayed in Table 1.

Faculty (n = 32) were asked to estimate the degree to which skills decayed for resident who completed research years, the duration for those skills to return after rejoining clinical training, and whether they felt a technical skills refresher curriculum would be beneficial.

2.2. Technical Skills Refresher Curriculum

The skills refresher curriculum was comprised of two

Table 1. Survey questions.

parts: Dry skills practice and cadaver lab. The dry skills practice consisted of eight elements that were completed in our simulation center. Elements included 12 basic knot-tying and suturing exercises [1], six intermediate knot-tying and suturing exercises, Fundamentals of Laparoscopic Surgery manual skills [2,3], vascular anastomosis, bowel anastomosis, two modules on the METI SurgicalSim virtual reality trainer (retract and dissect, cholecystectomy), and two laparoscopic Nissen Fundoplication part-task exercises (hiatal dissection, cruroplasty).

For the basic and intermediate knot-tying and suturing and the FLS manual skills curricula, residents were required to demonstrate proficiency at established benchmarks. Residents were pretested on these curricula to determine incoming proficiency level. Residents whose performance did not meet the established proficiency criteria for any exercise on a pretest trial were required to complete additional proficiency-based training until the criteria were met. That is, residents were allowed to “test-out” of additional proficiency-based training by meeting the established proficiency benchmarks on the pretest. Residents were not allowed to “test-out” of the vascular, bowel, virtual reality laparoscopic, and laparoscopic Nissen Fundoplication part-task curricula. Training on the dry skills practice was self-paced and selfguided and residents were allotted 4 weeks to complete training. Each resident was required to complete each of the dry skills curricula to be eligible to participate in the cadaver lab.

The cadaver lab was completed in our experimental surgery facility where there were three cadaver stations with 2 to 3 residents per station. Each station was staffed by a faculty surgeon. Each station completed laparoscopic ventral inguinal hernia, laparoscopic gastrectomy, and laparoscopic gastrojejunostomy surgical procedures. Depending on each resident’s proficiency with these procedures and whether time allowed, some residents were able to complete additional procedures including laparoscopic cholecystectomy, laparoscopic appendectomy, open splenectomy, and exploratory laparotomy. The total time allotted for the cadaver lab was four hours.

Upon completion of the curriculum, residents were asked to complete a survey that assessed self-reported global technical skills comfort before and after completing the skills refresher on a 5-point Likert scale. Residents were also asked to rate the usefulness of the curricular elements on a 3-point Likert scale (i.e., “not helpful”, “somewhat helpful”, “extremely helpful”) and whether the curriculum should be offered the following year (“offer the entire curriculum”, “offer only the dry skills curriculum”, “offer only the cadaver lab”, “do not offer any skills refresher curriculum”).

A total of eight residents were scheduled to end their research year(s) during this timeframe. Seven residents completed the curriculum. After completing the curriculum, one resident chose to do an additional research year.

2.3. Technical Skills Evaluations

We evaluated curriculum effectiveness by examining the departmental end-of-rotation evaluations that are completed by faculty, senior residents, and other members of the healthcare team. These evaluations collect ratings for each of the six Accreditation Council on Graduate Medical Education (ACGME) competencies with the addition of technical skills. Ratings are scored on a 4-point Likert scale (i.e., “not acceptable for level of training”, “worse than fellow residents at the same level of training”, “comparable to fellow residents at the same level of training”, “better than fellow residents at the same level of training”). We retrieved evaluations from the April 1 through June 30 timeframe from the PGY2 year (i.e., the last three months of clinical training prior to beginning research years) and the July 1 through October 30 timeframe for the PGY3 year (i.e., the first three months of clinical training after finishing research years) for the six residents who completed the skills refresher curriculum and rejoined clinical training. For a comparson group, we retrieved evaluations from the given time-frames for the seven residents who rejoined clinical training the year prior to the implementation of the skills refresher curriculum. We used only the data from the technical skills item on the evaluation form.

3. Results

3.1. Resident and Faculty Opinions of Technical Skills Decay Survey Data

A total of 20 residents in our program did research (currently PGY3-5) or are currently doing research from 2009 to 2012. Of these residents, 14 (70%) completed the survey. Residents believed that non-clinical training years would have a slightly negative impact on their technical skills (median = 2 ± 0.36). In terms of the length of time required for technical skills to return to the level of their peers who did not do research, residents reported between 1 - 2 months and 2 - 3 months (median = 2.5 ± 1.22).

A total of 16 out of 32 faculty members (50%) completed the survey. Analysis of the faculty survey data revealed that faculty believed that non-clinical training years have a slightly negative impact on residents’ technical skills (median = 2 ± 0.72). In terms of the length of time required for residents’ technical skills to return, faculty believed it took somewhere between 2 - 3 months and 3 - 4 months (median = 3.5 ± 1.93). Thirteen (81.3%) faculty members stated that they felt a skills refresher curriculum should be offered for residents completing research year(s) and that this curriculum would expedite the process of technical skills recovery.

3.2. Technical Skills Refresher Curriculum Data

None of the seven research residents who completed the skills refresher curriculum practiced the basic and intermediate knot-tying and suturing exercises prior to the pretest. All residents achieved the proficiency benchmarks on the pretest for these exercises. Thus, no resident was required to complete proficiency-based training on these exercises.

All seven residents reported practicing the FLS exercises prior to the pretest and all achieved the proficiency benchmarks on these exercises. Thus, no resident was required to complete proficiency-based training on these exercises.

All seven residents who participated in the skills refresher curriculum completed the curriculum evaluation survey (100%). Residents rated their comfort of overall technical skills prior to completing the skills refresher curriculum as neutral (median = 2 ± 0.54) and as moderately comfortable after completing the skills refresher curriculum (median = 3 ± 0). Wilcoxon signed ranks analysis revealed that this difference was statistically significant (p < 0.048).

Figure 1 displays the ratings of the curriculum usefulness items. Residents rated the basic and intermediate knot-tying and suturing as being somewhat helpful (median = 2 ± 0.816). FLS, vascular anastomosis, bowel anastomosis, and the cadaver lab were rated as extremely helpful (median = 2 ± 0.488, 0, 0, 0, 0, respectively). The virtual reality retract and dissect and laparoscopic cholecystectomy modules were rated as not helpful (median = 0 ± 0.894).

All residents recommended offering the entire curriculum in subsequent years, given options of not offering any skills refresher curriculum, offering only the dry skills practice, offering only the cadaver lab, or offering the entire curriculum.

3.3. Technical Skills Evaluation Data

Descriptive statistics for the technical skills evaluation comparison are presented in Table 2 and the gain in technical skills ratings from the end of the PGY2 year to the beginning of the PGY3 year is graphically in Figure 2. Given the small sample size and lack of a normal distribution, data were analyzed using a Mann-Whitney-U test. Results showed that residents in the cohort that did not get the skills refresher curriculum had similar evaluation ratings prior to leaving clinical training to conduct their research years as the cohort that did get the skills refresher curriculum. This finding indicated that the two resident cohorts began their research years with roughly equivalent technical skills ratings.

Analysis of evaluation ratings for the three month period after completing research years revealed that residents in the cohort that received the skills refresher curriculum had significantly higher evaluation ratings (p = 0.041) than the cohort that did not receive the skills refresher curriculum.

4. Conclusions

Overall, residents and faculty view non-clinical training

Table 2. Descriptive statistics of end-of-rotation evaluations technical skills.

years as having a slightly deleterious effect on technical skills. Residents believe technical skills return to the levels of non-research resident peers within a 3-month period, but faculty believe it could require as long as 4 months. However, the perceived decline in technical skills may not be as great as anecdotally reported. This is evidenced by the fact that all of the residents who completed the technical skills refresher curriculum performed at proficiency levels for the suturing and knot-tying and FLS exercises on the pretest.

We attempted to overcome the perceived deficit by offering a technical skills refresher curriculum to research residents in the month prior to returning to clinical training. The majority of faculty reported that a technical skills refresher curriculum would be beneficial and the residents who completed the curriculum agreed. Additionally, these residents reported a significant increase in comfort with their technical skills after completing the curriculum.

In addition to improved self-reported comfort, the residents who completed the technical skills refresher curriculum also received significantly higher clinical evaluation ratings from faculty compared to a cohort of residents who returned to clinical training in the prior year (before the technical skills refresher curriculum was implemented). However, applicability of this conclusion should be tempered given the relatively small number of residents who were available for this study.

The main limitation of the surveys issued as part of this study is the reliance on self-reported comfort. Selfreported comfort can be related to self-efficacy, which psychologist Alfred Bandura [4] viewed as a construct to define an individual’s behavior in novel or challenging situations. Individuals who lack self-efficacy tend to perceive difficult tasks as personal threats to esteem and/or confidence. However, individuals with an abundance of self-efficacy tend to attack difficult tasks as a personal and/or professional growth opportunity. Bandura posits that one of the most effective ways of creating a strong sense of self-efficacy is through mastery experiences. By offering training in a low-stakes and non-threatening environment such as a simulation lab, we allowed residents to achieve and demonstrate mastery of several technical skills, which in turn, served as confidence-building exercises. Indeed, residents who completed the technical skills refresher curriculum returned to clinical training with higher levels of comfort and self-efficacy. It appears that the boost in self-efficacy aided residents in their first three months of clinical training following research years as evidenced by higher technical skills evaluations than their resident peers who completed research the year prior to the technical skills refresher curriculum offering.

Given the observed benefits in technical skills ratings on clinical evaluations and self-reported comfort as well as resident and faculty support of a technical skills refresher curriculum, it is recommended that residents ending research year(s) and rejoining clinical training complete a technical skills refresher curriculum.

REFERENCES