Abstract
Mastery of basic skills is critical for surgical training. Such training is best obtained by experiential learning, which requires an element of self-reflection. Self-reflection is not always an automatic process, however; guidance may be required. This article sought to determine whether guided self-assessment would help facilitate student mastery of learned skills in a veterinary basic surgery course. The course consisted of 18 lectures and eight laboratories. Students were provided with written notes and presentation slides before the course. At the end of each lab, students completed a self-assessment of their skills. Skills were practiced in multiple labs; at the end of the course, each student was given a graded, practical examination to evaluate skills mastery. Statistical analysis was performed to compare students’ mean self-assessment over the eight labs and to determine whether self-assessment scores correlated with examination grades. Results from 80 students were included. Students’ overall self-assessments improved significantly from lab 1 to lab 8, and their self-assessment of two specific skills (closed gloving and simple continuous suture pattern) also improved. Students’ self-assessments after the eighth lab were predictive of their practical exam scores. These results suggest guided reflection in the form of self-assessment could help facilitate student mastery of basic surgery skills. Correlation between self-assessment and practical examination results suggests instructors may use these self-assessments to detect students who need extra practice or instruction.
Learning through hands-on experience is part of the nature of coursework in health profession curricula, and preparation for a career in veterinary medicine is no exception. Experiential learning begins with concrete experience and ideally contains three additional important components.1 As described in David A. Kolb’s groundbreaking work on experiential learning theory (ELT), these additional elements are student involvement in reflective observation of the experience, abstract conceptualization, and active experimentation.1 Reflective observation requires students to contemplate the experience and then integrate and distill their reflections into abstract concepts.2 From there, students actively test what they have learned, which guides them into new experiences.2 In other words, students’ consciousness of their own learning may be used to purposefully enhance learning.2
Kolb’s ELT was incorporated into an approach to teaching in the course Principles and Practices of Surgery, an introduction to surgical skills taught in the second year of the veterinary professional curriculum at our institution. For students learning surgical skills, as discussed in Reznick and MacRae, Kolb’s theory nicely complements Fitts and Posner’s three-stage theory of motor skill acquisition.3 In the initial cognitive stage, the student must think about each step of the task and understand the associated mechanics. With practice and feedback, the student reaches the integrative stage; at this point, the student must still think about the task but displays demonstrable improvement in fluidity and fewer pauses. Once the student reaches the autonomous stage, he or she no longer needs to think about the specifics of executing the task, which allows the student to focus on other aspects of the task. Mastery of basic skills (knot tying, suture placement, instrument handling, etc.) is critical for surgical training, and students must deliberately practice until they attain automaticity in those basic skills.3
In the Principles and Practices of Surgery course described in this article, these theories were combined to deliver content and support mastery. The aim was to facilitate student mastery using reflective self-assessments on learned skills that would be used to build more complex skill sets as the course progressed. Various surgical models (foam; pig feet; and dog, cat, and pig cadavers) were used as the modus operandi. Prior research has suggested that practicing on models before live animal surgery improves veterinary students’ confidence in their surgical abilities and produces better psychomotor and basic surgical skills.4,5
The two-credit, one-semester Principles and Practices of Surgery course is divided into 18 lectures, eight laboratories, two written exams, and a practical examination in which students demonstrate their skills. Each laboratory is divided into two sections with 42–43 students in each section. During the fall 2017 semester, students were given written notes and presentation slides in advance of the laboratories and were asked to read the notes and view the slides before each lab. Written notes for each laboratory included a list of objectives; a detailed description of each task, written in outline form; and illustrations or photographs of portions of tasks. Presentation slides included step-by-step photos and instructions for each task, plus tips and warnings to improve performance or avoid failure.
During each 3-hour laboratory, students were given a list of tasks to complete. Lab 1 focused on sterile (aseptic) technique and was taught by demonstration and subsequent technician evaluation of individual student skills. During each subsequent laboratory, students were expected to incorporate what they learned in previous laboratories to continuously build their skill set toward eventual abdominal vessel ligation, organ removal, and visceral suturing (Table 1) in fresh pig cadavers obtained from a local meat production facility (lab 8). For labs 2 and 3, students practiced using foam models. Lab 4 used foam models and pig feet, and labs 5–7 used cadavers of dogs or cats euthanized at regional animal shelters for reasons other than teaching. All new tasks and some repeat tasks (those that were components of a procedure) were demonstrated live and projected on video monitors. Students were asked to practice sterile technique during pack opening, scrubbing, hand drying, gowning, gloving, and draping in each laboratory.
|
| Lab | Skills taught | Teaching method | Self-assessment scores* | ||
| Overall (N = 80) | Closed gloving (N = 79) | Subcutaneous closure (N = 80) | |||
| 1 | Aseptic technique (including gowning and gloving), surgery pack preparation | Demonstration | 1.74 (0.33) | 1.39 (0.52) | NA |
| 2 | Instrument and suture handling: square knots, simple interrupted sutures, hand ties | Foam models | 1.70 (0.27) | NA | NA |
| 3 | Skin incisions, basic and advanced interrupted suture patterns | Foam models | 1.84 (0.26) | NA | NA |
| 4 | Continuous suture patterns | Foam models and pig feet | 1.74 (0.29) | NA | 1.78 (0.48) |
| 5 | Mass removal | Cat cadaver | 1.58 (0.27) | 2.01 (0.57) | NA |
| 6 | Anatomy, vessel ligation, organ removal | Dog or cat cadaver | 1.79 (0.37) | 2.09 (0.51) | 1.65 (0.58) |
| 7 | Anatomy, vessel ligation, organ removal | Dog or cat cadaver | 1.84 (0.25) | 2.08 (0.50) | 1.70 (0.54) |
| 8 | Abdominal procedure—anatomy, vessel ligation, organ removal, visceral suturing | Fresh, intact female pig cadaver | 1.81 (0.24) | 2.05 (0.45) | 1.96 (0.34) |
* 1 = still need practice or will ask for help; 2 = competent for the practical exam; 3 = excelled at the skill; ready for surgery
NA = not assessed
Four instructors were assigned to each laboratory, and each instructor was assigned to three groups of three or four students. For labs 2–8, the group instructor monitored student progress and provided feedback and instruction during the lab as needed. Before each lab, instructors reviewed the instructional materials, including laboratory objectives and written and illustrated procedural descriptions. For each laboratory, one instructor was identified as lead and provided an introduction and, when needed, demonstrations to the students at the beginning of the laboratory. Lead and group instructors varied with each laboratory.
At the end of each laboratory, each student completed a self-assessment, rating themselves on each required skill task for that lab (1 = still need practice or will ask for help; 2 = competent for the practical exam; and 3 = excelled at the skill; ready for surgery). From these self-assigned ratings, a mean rating was calculated for each student for each lab. Then the mean score for all students for each lab was calculated. Two specific skills, closed gloving and placement of a simple continuous suture pattern, were selected to examine student self-assessment longitudinally; these two skills were chosen because they were self-assessed in at least four laboratories. Closed gloving was taught in lab 1 and was a required self-assessment procedure in laboratories 1, 5, 6, 7, and 8. Open gloving was practiced in the remaining three labs. A simple continuous pattern was taught in lab 4 and was a required self-assessment procedure in labs 4, 6, 7, and 8. Simple continuous pattern placement was not emphasized in the remaining laboratories.
Near the end of the course, students were required to take a practical examination to demonstrate their ability to perform the following 13 tasks that had been covered in the laboratories: identification of suture patterns and instruments, operative preparation, instrument pack opening, hand scrubbing, towel drying of hands, gowning, closed gloving, draping, instrument handling, placement of a simple continuous pattern, changing of contaminated gloves, open gloving, and completing a hand tie. The final course grade consisted of a summary of scores from the two written exams (65% of grade), the practical exam (25% of grade), and laboratory attendance (10% of grade).
At the end of the final written examination, students were asked to complete a confidence survey on which they selected one or more of five statements that best described their feelings about the course and their performance:
I feel I can maintain sterile technique before and during a 1-hour surgical procedure if I set my mind to it.
I feel I am ready to perform a live animal surgery if I have help or supervision.
I would like to practice basic skills more before doing my first supervised elective surgery.
I would probably be more comfortable in surgery if I had practiced more during or between laboratories.
I wish I had been required to maintain sterility throughout the entire pig cadaver abdominal laboratory.
Students’ responses on laboratory self-assessments and the confidence survey were collected in the Canvas learning management system (Instructure, Inc., Salt Lake City, UT) and subsequently downloaded into an Excel (Microsoft Corp., Redmond, WA) spreadsheet, at which time individual identifiers were removed for statistical analysis. The institutional review board of the University of Tennessee approved the project in the exempt category (IRB-18-04610-XM).
Before performing analyses using IBM SPSS Statistics, version 24 (IBM Corp., Armonk, NY), the data were cleaned using the 12-step method of Morrow and Skolits.6 Statistical outliers were identified as any value |3.29| SDs from the mean.7 Outliers were modified in four variables (self-assessment in labs 2, 5, 7, and 8) using the standard method of Tabachnick and Fidell.7 We used pairwise deletion to manage the one missing value in the first closed gloving self-assessment. All data were normally distributed, as indicated by low skewness and kurtosis values.7 Assumptions for each statistical method were addressed, and for the analysis of variance (ANOVA) tests, the Mauchly test indicated non-sphericity; this result necessitated the use of the Greenhouse–Geisser value to examine significance from ANOVAs while reducing the likelihood of a type I error.7
To examine possible statistical differences in students’ mean self-assessment of laboratory skills from lab 1 to lab 8, we performed a repeated-measures ANOVA using a Bonferroni post hoc test. Likewise, we used an ANOVA and Bonferroni post hoc test to examine self-assessment of closed gloving and subcutaneous closure. The alpha level was set to .05 for all tests.
To determine whether a significant correlation existed among the final laboratory self-assessment, practical exam score, or final course grade, we conducted a two-tailed, Pearson correlation test. Additional correlation tests were conducted to examine possible relationships among practical exam score, final course grade, or the final closed gloving and simple continuous closure self-assessments.
To examine possible relationships between final letter grade and the five final performance statements, we conducted a χ2 test of independence for each of the five statements.
Eighty-five students were enrolled in the course for the entire semester. Five students did not provide a self-assessment score for at least one of the two skills in one of the laboratories; scores for these students were not included in the statistical analyses. Table 1 summarizes students’ perceptions of their performance of the two skills and overall for each laboratory.
Results from the repeated-measures ANOVA showed a statistically significant increase in overall self-assessment rating between lab 1 (M = 1.74, SD = 0.33) and lab 8 (M = 1.81, SD = 0.24), F(5, 412) = 15.33, p < .001, partial η2 = .16, power = 1.00 (Table 1, Figure 1). Similarly, a significant increase was seen in overall closed gloving between the first self-assessment in lab 1 (M = 1.39, SD = 0.52) and the fifth self-assessment in Lab 8 (M = 2.05, SD = 0.45), F(3, 251) = 42.74, p < .001, partial η2 = .35, power = 1.00 (Tables 1 and 2, Figure 2). The final repeated-measures ANOVA showed an overall statistically significant increase in self-assessment rating for simple continuous suturing from lab 4 (M = 1.78, SD = 0.48) to lab 8 (M = 1.96, SD = 0.34), F(3, 209) = 9.44, p < .001, partial η2 = .11, power = 0.99 (Tables 1 and 3, Figure 3). Figures 1–3 also show significant post hoc differences between individual labs.
Figure 1: Student overall self-assessment scores (1 = still need practice or will ask for help; 2 = competent for the practical exam; 3 = excelled at skill; ready for surgery) during surgery laboratories
Figure 2: Student closed gloving self-assessment scores (1 = still need practice or will ask for help; 2 = competent for the practical exam; 3 = excelled at skill; ready for surgery) during surgery laboratories
* p < .001
Figure 3: Student simple continuous pattern suturing self-assessment scores (1 = still need practice or will ask for help; 2 = competent for the practical exam; 3 = excelled at skill; ready for surgery) during surgery laboratories
* p < .01; †p < .001
|
| Lab | n (%) | ||
| Still need practice or will ask for help | Competent for the practical exam | Excelled at the skill; ready for surgery | |
| 1* | 49 (61.3) | 29 (36.3) | 1 (1.3) |
| 5 | 12 (15) | 55 (68.8) | 13 (16.3) |
| 6 | 7 (8.8) | 59 (73.8) | 14 (17.5) |
| 7 | 7 (8.8) | 60 (75) | 13 (16.3) |
| 8 | 6 (7.5) | 64 (80) | 10 (12.5) |
* N = 79
|
| Lab | n (%) | ||
| Still need practice or will ask for help | Competent for the practical exam | Excelled at the skill; ready for surgery | |
| 4 | 20 (25.0) | 58 (72.5) | 2 (2.5) |
| 6 | 32 (40.0) | 44 (55.0) | 4 (5.0) |
| 7 | 27 (33.8) | 50 (62.5) | 3 (3.8) |
| 8 | 6 (7.5) | 71 (88.8) | 3 (3.8) |
The Pearson r test, used to examine possible relationships among the practical exam, final course grade, and overall self-assessment on the last laboratory, showed a significant correlation between self-assessment and practical exam score, r(78) = .22, p = .047 (Figure 4), but not between self-assessment and final grade (data not shown). The higher the self-assessment score, the higher the practical exam score. No significant correlation was found between the final mean self-assessment for closed gloving and practical grade or for the self-assessment for suturing and practical grade (data not shown). Grades on the practical examination ranged from 74% to 100% (M = 94.17, SD = 4.63).
Figure 4: Student self-assessment scores during the final surgery laboratory correlated to practical exam mean scores, maximum possible score of 65 (r = .22, p = .047)
Of the 80 students who completed the end-of-course survey, 40 (50%) felt they could maintain sterile technique before and during a 1-hour procedure if they set their mind to it. Thirty-five (43.8%) students felt they were ready to perform a live animal surgery as long as they had help or supervision. Thirty-three (41.3%) students felt they would like to practice basic skills more before doing their first supervised spay or castration. Only eight (10%) students thought they would probably be more comfortable if they had practiced more during or between labs, and seven (8.8%) students wished they had been required to maintain sterility throughout the entire pig abdominal lab (Table 4).
|
| Confidence statement | Final letter grade, n (%) | |||
| A | B+ | B | C+ | |
| I feel I can maintain sterile technique before and during a 1-hour procedure if I set my mind to it | 16 (40.0) | 9 (22.5) | 13 (32.5) | 2 (5.0) |
| I feel ready to perform live animal surgery as long as I have help or supervision | 19 (54.3) | 7 (20.0) | 8 (22.9) | 1 (2.8) |
| I would like to practice basic skills more before doing my first supervised spay or castration | 12 (36.4) | 6 (18.2) | 12 (36.4) | 3 (9.0) |
| I would probably be more comfortable if I had practiced more during or between labs | 3 (37.5) | 3 (37.5) | 0 | 2 (25.0) |
| I wish we had been required to maintain sterility throughout the entire pig abdominal lab | 5 (71.4) | 1 (14.3) | 0 | 1 (14.3) |
Note: Students selected all statements that applied
A breakdown of student confidence by final letter grade is shown in Table 4. The results for ability to maintain sterility and readiness for live animal surgery were similar to the overall final grade distribution: 44.7%, A; 17.6%, B+; 31.8%, B; and 3.5%, C+. Likewise, χ2 results did not indicate a significant relationship between final letter grade and any of the confidence statements (data not shown).
When students participate in directed self-assessment, they are organically taking part in guided reflection.8 Students are able to first focus their cognitive load on the concrete experience; after the experience, students are able to focus on the abstract conceptualization step of experiential learning.8 Ungraded, immediate longitudinal self-assessment as a reflective method in this surgery course seems to be an accurate measure of student performance on the practical exam. Not factoring a self-assessment into the final grade has been shown to encourage students to be more honest and realistic in their reflection.9 Poor performance and uncertainty also promote reflective observation and learning.10 Immediacy ensures students have not yet shared their thoughts with classmates, which could skew their perceptions.9
During the Principles and Practice of Surgery course, we intended to support students in reaching the active experimentation level of ELT by requiring them to practice the same skills in each laboratory. The hope was that this practice would help them extend their learning to new situations (e.g., vessel ligation and visceral suturing), what Schenck and Cruickshank described as bridge building and transfer.8 Such learning and transfer create “richer, broader, and deeper” learning.1(p. 30) In a previous study of surgery performance among veterinary students, skill-based and procedure-based training curricula performed on a simulator improved laparoscopic surgical skills.11 That structured, 10-hour training program resulted in improved skills with minimal proctoring by an experienced surgeon.11 In the current study, overall self-assessment scores significantly increased from the first to the final laboratory, even while tasks became more complex and required incorporation of multiple skills learned individually over the course of the semester. For this course, skills-based instruction was a successful method to transfer basic surgical skills from models to cadavers.
For the simple continuous suture pattern, students’ self-ratings decreased between the first and second self-assessment periods for this skill but increased significantly (p < .001) between the second and fourth self-assessment periods. Several possible reasons for this result exist. Not only were laboratories 4 and 6 separated by 4 weeks, but the simple continuous suture pattern in lab 4 was taught on foam models and pigs’ feet, whereas labs 6, 7, and 8 were taught on cadaver muscle and subcutaneous fat. The passage of time and the use of differing models between labs 4 and 6 could have caused a decrease in self-ratings. However, an additional phenomenon could be at play: the Dunning–Kruger effect. With this effect, students initially overestimate their own abilities, but as they learn, they begin to realize what they do not know. As a result, self-assessment temporarily dips before starting to increase again.12 This same effect might help explain why students initially rated themselves so low on closed gloving—a veterinary technician had provided immediate feedback and constructive critique, possibly influencing the students to rate their skills lower than they really were. Despite the fact that labs 1 and 5 were separated by 6 weeks, the number of students rating themselves as proficient enough to pass a practical examination on closed gloving increased significantly by the end of the second self-assessment period for that skill.
Also of note was the absence of a correlation between students’ self-assessment and final course grade, meaning self-assessment for surgical skills did not predict how well a student would perform in this course as a whole. However, 65% of the final course grade was based on didactic knowledge demonstrated via a written exam; therefore, it is plausible that different weighting of the practical and laboratory components of the course might result in different correlations between these two variables.
Students can gain confidence quickly when learning single skill tasks such as closed gloving, which leaves them more time and mental energy to focus on more complex tasks. Those complex tasks, such as simple continuous patterns, usually require a combination of skills (e.g., instrument handling, knot tying, determination of tissue bite size and depth) and may need to be performed under a variety of different conditions (e.g., on foam, cloth, or other synthetics and on different types of cadaver tissues). Therefore, confidence in mastery of complex tasks will likely take longer. Self-assessment scores can be predictive of practical examination results, which provides an opportunity for intervention by the instructor.
A limitation of the study is that it was conducted with only one cohort of students in one semester at one institution; therefore, the results might not be generalizable to other veterinary students. Student self-assessment in subsequent cohorts, as well as examination of the same methods at other veterinary schools, would strengthen the generalizability of the results and would be areas for future research. In addition, because multiple instructors were involved in the laboratory, variations in instruction and support could affect skills performance or student confidence. In particular, students completed their self-assessments near the end of lab 1 either immediately before or after demonstrating sterile technique to, and receiving feedback from, a technician. Such feedback could have affected students’ perceptions of their skill level. In subsequent laboratories, instruction and feedback were given as needed throughout the laboratories, and self-assessments were filled out at the end of the laboratory.
The results of this study suggest that using student self-assessment enabled students to reflect on their basic surgery skills in preparation for eventually performing their first supervised surgery on a live animal. In addition, students’ self-assessments seemed to accurately reflect their level of preparation, as evidenced by the correlation between the final self-assessment and grade on the practical examination.
| 1. | Kolb DA. Experiential learning: experience as the source of learning and development. Englewood Cliffs (NJ): Prentice Hall; 1984. Google Scholar |
| 2. | Kolb AY, Kolb DA. The learning way: meta-cognitive aspects of experiential learning. Simul Gaming. 2009;40(3):297–327. https://doi.org/10.1177/1046878108325713. Google Scholar |
| 3. | Reznick RK, MacRae H. Teaching surgical skills—changes in the wind. New England J Med. 2006;355(25):2664–9. https://doi.org/10.1056/NEJMra054785. Medline: 17182991 Medline, Google Scholar |
| 4. | Badman M, Tullberg M, Hoglund OV, Hagman R. Veterinary student confidence after practicing with a new surgical training model for feline ovariohysterectomy. J Vet Med Educ. 2016;43(4):427–33. https://doi.org/10.3138/jvme.1015-165R2. Medline: 27487110 Link, Google Scholar |
| 5. | Griffon DJ, Cronin P, Kirby B, Cottrell DF. Evaluation of a hemostasis model for teaching ovariohysterectomy in veterinary surgery. Vet Surg 29(4):309–16, 2000. https://doi.org/10.1053/jvet.2000.7541. Medline: 10917280 Medline, Google Scholar |
| 6. | Morrow JA, Skolits G. Twelve steps of quantitative data cleaning: strategies for dealing with dirty evaluation data. American Evaluation Association Annual Conference; 2017 Nov 6–11; Washington, DC. Google Scholar |
| 7. | Tabachnick BG, Fidell LS. Using multivariate statistics, 6th ed. Upper Saddle River (NJ): Pearson; 2013. Google Scholar |
| 8. | Schenck J, Cruickshank J. Evolving Kolb: experiential education in the age of neuroscience. J Exp Educ. 2015;38(1):73–95. https://doi.org/10.1177/1053825914547153. Google Scholar |
| 9. | Johns JA, Moyer MT, Gasque LM. Planning and facilitating debriefs of experiential learning activities in skills-based health education. J Health Educ Teach. 2017;8(1):61–76. Google Scholar |
| 10. | Miettinen R. The concept of experiential learning and John Dewey’s theory of reflective thought and action. Int J Lifelong Educ. 2000;19(1):54–72. https://doi.org/10.1080/026013700293458. Google Scholar |
| 11. | Chen CY, Ragle CA, Lencioni R, Fransson BA. Comparison of 2 training programs for basic laparoscopic skills and simulated surgery performance in veterinary students. Vet Surg. 2017;46:1187–97. https://doi.org/10.1111/vsu.12729. Medline: 28990691 Medline, Google Scholar |
| 12. | Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77(6):1121–34. https://doi.org/10.1037/0022-3514.77.6.1121. Medline: 10626367 Medline, Google Scholar |
