short simulation training improves objective skills in ... practitioners managing emergencies on the...

ORIGINAL ARTICLE

Short Simulation Training Improves Objective Skills in EstablishedAdvanced Practitioners Managing Emergencies on the Ward and

Surgical Intensive Care Unit

Jose L. Pascual, MD, PhD, FRCPS(C), Daniel N. Holena, MD, FACS, Michael A. Vella, MD,Joseph Palmieri, MSEd, Corinna Sicoutris, RN, MSN, CRNP, BC, Ben Selvan, MD, Adam D. Fox, DO,

Babak Sarani, MD, FACS, Carrie Sims, MD, MS, FACS, Noel N. Williams, MB, BCh, FRCSI, FRCS(Gen),and Charles William Schwab, MD, FACS

Background: Several studies evaluating simulation training in intensive careunit (ICU) physicians have demonstrated improvement in leadership andmanagement skills. No study to date has evaluated whether such training isuseful in established ICU advanced practitioners (APs). We hypothesizedthat human patient simulator-based training would improve surgical ICUAPs’ skills at managing medical crises.Methods: After institutional review board approval, 12 APs completed 1⁄2day of simulation training on the SimMan, Laerdal system. Each subjectparticipated in five scenarios, first as team leader (pretraining scenario), thenas observer for three scenarios, and finally, again as team leader (posttrain-ing). Faculty teaching accompanied each scenario and preceded a debriefingsession with video replay. Three experts scored emergency care skills(ABCs, recognition of shock, pneumothorax, etc.) and teamwork leadership/interpersonal skills. A multiple choice question examination and trainingeffectiveness questionnaire were completed before and after training. Fel-lows underwent the same curriculum and served to validate the study. Pre-and postscores were compared using the Wilcoxon signed rank test withtwo-tailed significance of 0.05.Results: Improvement was seen in participants’ scores combining all pa-rameters (73% � 13% vs. 80% � 11%, p � 0.018). AP leadership/interpersonal skills (�12%), multiple choice question examination (�4%),and training effectiveness questionnaire (�6%) scores improved signifi-cantly (p � 0.05). Fellows teamwork leadership/interpersonal skills scoreswere higher than APs (p � 0.001) but training brought AP scores to fellowlevels. Interrater reliability was high (r � 0.77, 95% confidence interval0.71–0.82; p � 0.001).Conclusions: Human patient simulator training in established surgical ICUAPs improves leadership, teamwork, and self-confidence skills in managingmedical emergencies. Such a validated curriculum may be useful as an APcontinuing education resource.

Key Words: Simulation, Advanced practitioners, ICU, Critical care, Humanpatient simulator.

(J Trauma. 2011;XX: 000–000)

The ever-increasing reductions in resident numbers andwork hours1 have led many intensive care units (ICUs) to

hire permanent advanced practitioners (APs) such as certifiedregistered nurse practitioners (CRNPs), and physician assis-tants (PA-Cs). Even in academic centers where, historically,residents have provided the majority of day-to-day patientcare, APs have been embedded side by side with, and in somecases have supplanted house staff. In general, acute care APshave up to 6 years of graduate training and can provide agreater continuity of care than frequently rotating residentteams.2 Since 1999, the University of Pennsylvania Traumaand Critical Care Health Network has employed APs in theirsurgical ICUs (SICUs). These APs admit patients, provideday-to-day care, and perform all bedside procedures (e.g.,insertion of central venous lines, arterial lines, chest tubes).Although these providers have extensive knowledge andexperience, maintaining the skill set to manage high-riskevents that occur infrequently (e.g., extubation, develop-ment of tension pneumothorax, massive upper gastrointes-tinal bleeding) can be challenging. Currently, continuingmedical education specifically targeted at this group ofclinicians is modest outside webcast and web-based edu-cational resources.

The data-rich environment of the SICU is ideal todesign and implement a curriculum using high-fidelity humanpatient simulators (HPS). This is particularly underscored intwo ways: (1) patients often have simultaneous monitoring ofmultiple physiologic parameters and (2) patient physiologycan change rapidly and by a great magnitude. This setting isthus highly conducive for scenario-based simulation to followtrainee skills before, during, and after the implementation oftraining.

Although much has been published relating to sim-ulation training of ICU medical students, residents, fel-lows, and attending physicians, the role of such training inAPs remains unexamined. We thus developed a scenario-

Submitted for publication January 3, 2011.Accepted for publication April 12, 2011.Copyright © 2011 by Lippincott Williams & WilkinsFrom the Division of Traumatology (J.L.P., D.N.H., J.P., C.S., B. Selvan, A.D.F.,

B. Sarani, C.S., N.N.W., C.W.S.), Surgical Critical Care & EmergencySurgery, University of Pennsylvania School of Medicine, Philadelphia, Penn-sylvania; and Department of Surgery (M.A.V.), Vanderbilt University Med-ical Center, Nashville, Tennessee.

Address for reprints: Jose L. Pascual, MD, PhD, FRCS(C), Division of Traumatology,Surgical Critical Care and Emergency Surgery, Department of Surgery, 3400Spruce Street, Philadelphia, PA; email: [email protected].

DOI: 10.1097/TA.0b013e31821f4721

balt5/zta-ta/zta-ta/zta99909/zta2751-09z xppws S�1 4/23/11 7:20 4/Color Figure(s): F1 Art: TA204578 Input-go

The Journal of TRAUMA® Injury, Infection, and Critical Care • Volume XX, Number XX, XXX 2011 1

AQ: 1

<article-id pub-id-type��doi�>10.1097/TA.0b013e31821f4721</article-id> ● <fpage></fpage> ● <lpage></lpage>

based simulation curriculum targeted at established SICUAPs to provide them the opportunity to experience high-risk/low-frequency events in a continuing medical educa-tion setting without risk to patients. As this traininginvolved time away from their work, the curriculum had tobe succinct enough to minimize the disruption of theirwork schedule. We hypothesized that a short, 5-hoursimulation curriculum emphasizing medical emergencieswould enhance APs’ teamwork, leadership, and clinicalskills. We further postulated that they would find theexercise useful to gain confidence in managing theseconditions.

METHODS

Trainee PopulationAfter institutional review board approval and with the

written consent of each trainee, 3 half-day training sessionswere organized. Trainees were fully practicing APs thatworked in the University of Pennsylvania Trauma and Crit-ical Care Health Network at two sites: The Reading Hospitaland Medical Center (TRHMC) and the Hospital of the Uni-versity of Pennsylvania (HUP). At the time of the study,TRHMC APs (both CRNPs and PA-Cs) were independentand ran day-to-day care of patients with the attendingphysician. APs at HUP (CRNPs only) ran day-to-day care,independently assessing and caring for SICU and postan-esthesia care unit patients and directly reporting to theattending intensivist.

Validation CohortTo validate the curriculum, the complete cohort (n � 4)

of recently graduated surgical critical care fellows wereinvited to participate in all facets of the curriculum to serve asthe gold standard comparison group. All fellows shortlythereafter successfully completed the Surgical Critical CareAmerican Board of Surgery examination.

CurriculumThree half-day sessions were organized to train four

participants (APs) each, at The Penn Medicine Clinical Sim-ulation Center. One additional session was organized for thevalidation cohort. The HPS used were male and femaleLaerdal SimMan with technical manipulation using SimManSoftware version 3.3.1 (Laerdal, Wappingers Falls, NY).Each 5-hour session began with a brief bedside introductionby the faculty and the simulation educator, giving a hands-onexplanation of the simulation process and use of the manne-quins. Each trainee functioned as team leader in a firstscenario (pretraining scenario), observed three3 different sce-narios where colleagues served as team leaders, and thenfinished the session by performing as team leader for a secondtime (posttraining scenario). To ensure performer/observerexperience that was equal for all participants, the scheme inTable 1 was used. During the 15-minute scenario, a facultymember (JLP) served as the bedside nurse, following ordersand performing requested tasks to ensure smooth forwardflow. The simulation educator observed the performancefrom behind a one-way mirror and operated the simulator,

following the scripted scenario choreography. Each scenariowas videotaped using two strategically placed video camerascapable of capturing high-fidelity video and audio of theperformance. At the end of each scenario, trainees (teamleader and observers) would review the recorded scenariowith faculty, each commenting on the favorable and lessfavorable aspects of the performance. Each postscenariodebriefing lasted an average of 15 minutes. Thus, the corecurriculum for each trainee consisted in a scenario performedas team leader (pretraining) followed by three scenarios as anobserver—all accompanied by in-room faculty teaching andby postscenario video review and faculty debriefing. Thecurriculum ended with a second scenario performed by eachtrainee as team leader (posttraining; Table 1).

ScenariosFive different scenarios were constructed by a panel of

faculty including seven surgical intensivists (surgeons andanesthesiologists) and a senior acute care AP. The scenariosincluded relatively uncommon (but not rare) emergent occur-rences requiring a rapid response call or an intervention in theICU. Scenarios were devised on the premise that they re-quired two “major” elements (e.g., tension pneumothorax,anaphylaxis, MI recognition) and three or four “minor” ele-ments (e.g., asks for chest X-ray after intubation, checkshemoglobin in bleeding patient, administers intubationdrugs). Each scenario was vetted and revised by the paneluntil difficulty levels were felt to be approximately the samebetween scenarios. The final five scenarios were (1) anaphy-laxis with tension pneumothorax, (2) septic shock from Clos-tridium difficile colitis, (3) MI with diabetic ketoacidosis, (4)hemorrhagic shock with abdominal compartment syndrome,and (5) deteriorating traumatic brain injury with status epi-lecticus. Although certain procedures were required duringthe scenarios (e.g., needle thoracostomy, chest tube insertion,endotracheal intubation, pulmonary artery catheter insertion),they were not performed physically eventhough indicationsand basic tenants of the procedure were discussed. Labora-tory tests, X-rays, electrocardiograms, and computed tomog-raphy scans required in the different scenarios were availableon request to the trainees at clinically plausible time intervals.

TABLE 1. Half-Day Simulation Session Schedule

Team Leader Session Type Scenario Observer(s)

A Pretraining 1 None

B Pretraining 2 A

C Pretraining 3 A, B

D Pretraining 4 A, B, C

A Posttraining 5 B, C, D

B Posttraining 1 C, D

C Posttraining 2 D

D Posttraining 3 None

Note that there are four trainees, each performing first as team leader (pretraining)then as observer for three scenarios and finishing with a second performance as teamleader (posttraining). All trainees observed three scenarios only and therefore only fivedifferent scenarios were needed. Observers were asked not contribute to the perfor-mance but silently observe in the room.


Pascual et al. The Journal of TRAUMA® Injury, Infection, and Critical Care • Volume XX, Number XX, XXX 2011

© 2011 Lippincott Williams & Wilkins2

T1

AQ: 2

Curriculum Assessment ToolsThe participant’s pre- and posttraining performance

was scored by three separate expert assessors using twoseparate assessment tools: Team Leadership-InterpersonalSkills (TLIS) and Emergency Clinical Care Skills (ECCS)score sheets. The TLIS assessment tool was adapted from thevalidated American College of Surgeons/Association of Pro-gram Directors in Surgery simulation curriculum3 but wasaltered to better conform to situational emergencies. It wassubdivided into (1) teamwork (communication with team,task delegation, leadership), (2) decision making (plan mak-ing, prioritization, timeliness), and (3) situational awareness(calmness/assertiveness, team/distraction management, re-peated reevaluation). Each subsection comprised five objec-tives that were rated on a binomial scale (i.e., adequate/inadequate). The ECCS tool was based on a score sheetdeveloped by the National Registry of Emergency MedicalTechnicians4 and was composed of (1) universal objectives(UN; using universal precautions, calling for help, orderlyprogression through resuscitation ABCs, etc.) and (2) sce-nario specific objectives (SP) (recognition of shock or tensionpneumothorax, initiation of transfusion, etc.).

Additionally, each trainee completed two question-naires before training. The first was a multiple choice ques-tion examination (MCQ) comprising of 18 questions relatedto different technical elements of each scenario. The sameMCQ was then administered to trainees after all scenarioswere completed. Participants were not allowed to consultresources to answer these questions. Although possible an-swers to questions could incidentally enter discussions duringdebriefings or in-room teaching, no specific effort was madeto answer MCQ questions explicitly during training. TheTraining Effectiveness Questionnaire (TEQ) was also com-pleted before and after training and was comprised of sevenquestions asking the participant to rate their own confidence

level at leading different types of medical emergencies usinga five-point Likert scale.

Finally, at the completion of training, a validated feed-back questionnaire was administered to each AP evaluatingtheir opinion on curriculum usefulness.5 This questionnairewas separated into four sections: (1) face validity (scenariorealism, utility for management approaches), (2) use fortraining (scenarios useful for learning leadership and clinicalskills), (3) use for assessment (utility for assessment ofleadership and clinical skills), and (4) quality of feedback(faculty and educator feedback and teaching quality). Com-ments on how to improve or change the curriculum to betteraddress educational needs of APs were also invited in thisquestionnaire.

Expert AssessorsTwo board certified surgical intensivists (DH, AF) and

a senior SICU CRNP (CS) scored all performances (pre- andposttraining). One of the assessors was present in the roomand directly scored trainees whereas the remaining twoscored them off-line using a password-protected web-basedsoftware (SimBridge Version 2.3, B-Line Medical, WA, DC),which allowed Internet access to the Simulation Center’srecorded footage. Through this software, all camera anglescould be accessed together or each in full screen version(Fig. 1). However, the in-room assessor was unblinded toperformance status (pre- or posttraining), and off-line asses-sors were blinded to the training status of the scenarioperformer. All scoring was performed independently by eachreviewer to minimize the possibility of interrater influence.

Statistical AnalysisContinuous data were assessed for normality using the

Shapiro-Wilk test. Initial unadjusted analyses for parametricvariables were performed using either paired samples t test or

Figure 1. Two cameras and a vital signs screen (identical to the bedside monitor) could be manipulated remotely to broad-cast simultaneously from different angles or as a full screen view.


The Journal of TRAUMA® Injury, Infection, and Critical Care • Volume XX, Number XX, XXX 2011 Simulation Training in ICU Advanced Practitioners

© 2011 Lippincott Williams & Wilkins 3

F1

independent samples t test, as appropriate. Wilcoxon signedranks test or Mann-Whitney test was used to test for differ-ences between continuous nonparametric data. Chi squarewas used for categorical nonparametric data. Interrater reli-ability was evaluated using average measure intraclass cor-relation with a two way mixed effects model. Two-tailedsignificance was set at p � 0.05. Data management andstatistical analyses were performed using SPSS Version 17(2010 SPSS, Chicago, IL).

RESULTS

Study PopulationTrainee demographics are summarized in Table 2. APs

from TRHMC and HUP were evenly intermixed amongsessions with twice as many nurse practitioners (CRNPs) asPA-Cs participating. Less than half of trainees reported pre-vious experience with simulator training.

Four recent graduates from the University of Pennsyl-vania Surgical Critical Care Training program were used asthe gold standard cohort to prove construct validity. For allanalyses, there were insufficient numbers to analyze differ-ences between institution of origin (TRHMC vs. HUP) andyears of experience or advanced training degree (CRNP vs.PA-Cs).

Interrater VariabilityInterrater reliability for all scores was excellent (aver-

age measures intraclass correlation 0.91, 95% confidenceinterval, 0.88–0.93; p � 0.001).

Team Leadership/Interpersonal SkillsThe modified American College of Surgeons/Asso-

ciation of Program Directors in Surgery Surgical SkillsCurriculum evaluation was used to assess teamwork andleadership.3 Percent scores of all trainees significantlyimproved in all spheres including team work, decisionmaking, situational awareness as well as total scores (Fig. 2;*p � 0.01, †p � 0.05). Fellows were also evaluated with the

TLIS assessment tool but their mean scores before and aftertraining remained unchanged.

Emergency Clinical Care SkillsEmergency clinical skills were evaluated using an ob-

jective assessment tool scoring clinical skills that fulfilled UNand scenario SP, the sum of which yielded total scores.Improvements in UN, SP, and total scores were not signifi-cant in both trainees and fellows (Figs. 3 and 7).

Multiple Choice Question and TrainingEffectiveness Questionnaires

Trainees were evaluated by a multiple-choice question-naire relevant to the scenarios administered, before and aftertraining. APs scores improved by 5% (p � 0.027), whereasfellows’ scores did not change significantly (Fig. 4). AP TEQself-reported scores improved by 8% (p � 0.048), whereasthat of fellows did not improve significantly (Fig. 5). Evalu-ation of whether posttraining improvements in MCQ scoreswas diminished in later sessions revealed no differencesindicating, perhaps, that trainees from previous sessions were

Figure 2. Team leadership/interpersonal skills sub and totalscores before and after training in APs. Note the improve-ment in all sections. *p � 0.01, †p � 0.05. (TW, teamwork;DM, decision making; SA, situational awareness; total, sumof all subscores.)

TABLE 2. Advanced Practitioner Trainee Demographics

Subject Institution Education Age (yr) SexYears

ExperiencePrevious Simulation

TrainingRapid Reponses

Attended Per Year

1A HUP CRNP 35 M 2 Y 0–5

1B TRHMC PA-C 34 M 5 Y 0–5

1C HUP CRNP 28 F 2.5 N 6–10

1D TRHMC PA-C 26 M 1 Y 0–5

2A TRHMC CRNP 27 F 1 Y 0–5

2B TRHMC PA-C 27 F 2.5 N 20�

2C TRHMC CRNP 30 F 6 N 0–5

2D HUP CRNP 32 F 4 N 0–5

3A HUP CRNP 36 F 2 N 0–5

3B TRHMC PA-C 25 F 1 Y 0–5

3C TRHMC CRNP 35 F 1 N 0–5

3D HUP CRNP 34 M 2 Y 0–5

Note that most APs were CRNP and had 1 year to 4 years experience. Five participants originated from HUP and seven from TRHMC. Half of trainees had undergone someform of simulation training in the past.




T2

F2

F3

F4

F5

not likely discussing MCQ questions with trainees of latersessions outside the sessions themselves.

Feedback Trainee EvaluationsThe Feedback Scoring tool demonstrated that the dif-

ferent curriculum elements were rated universally high bytrainees as follows: face validity (85% � 15%), use fortraining (88% � 11%), use for assessment (78% � 15%),quality of feedback (90% � 10%), and the sum total (88% �11%). Scores given by fellows appeared slightly higherindicating a more favorable perception of the curriculum’s

usefulness (face validity [91% � 8%], use for training[93% � 10%], use for assessment [85% � 15%], quality offeedback [87% � 13%], and the sum total [89% � 10%]).Written comments from both cohorts were overwhelminglyfavorable with occasional indications that it had been a mildlyto moderately stressful experience for APs.

Curriculum ValidationTo validate the curriculum, a complete session was

conducted for four surgical critical care fellows that hadcompleted their fellowship and were about to undergo theAmerican Board of Surgery certification examination. Thiscohort of fully trained intensivists served as a gold-standardvalidation cohort. Their TLIS scores were compared withthose of AP trainees before and after training (Fig. 6).Pretraining scores were significantly lower in APs than infellows (70.3% � 3.1% vs. 86.4% � 1.6%, p � 0.001).However, posttraining TLIS scores were similar in bothgroups. The ECCS section demonstrated a trend to improve-ment in both fellow and AP scores. Fellows demonstratedhigher ECCS scores than APs that approached significancepretraining (83.5% � 2.2% vs. 73.6% � 0.03%, p � 0.067),and posttraining (86.9% � 2.5% vs. 79.1% � 0.02%, p �0.054) with AP scores never reaching fellow scores (Fig. 7).The MCQ and TEQ scores showed no differences betweenAPs and fellows before and after training though scores werealways at least 5% higher in fellows.

Figure 3. Emergency clinical care skills—sub and total scoresin APs. Although an improvement in all scores were ob-served with training this was not significant. (UN, universalobjectives; SP, scenario-specific objectives; total, sum of UNand SP.)

Figure 4. The multiple choice questionnaire administeredbefore and after training was relevant to the discussed sce-narios and demonstrated improvement in advanced practi-tioners but not in critical care fellows. †p � 0.027.

Figure 5. A training effectiveness questionnaire was adminis-tered before and after training to APs to self-assess the de-gree of confidence they felt in managing different emergentsituations in the ICU or ward. Improvements were seen inadvanced practitioners but not in critical care fellows. †p �0.048.




F6

F7

DISCUSSIONAPs are increasingly staffing positions previously held

by residents in trauma and acute care units across the UnitedStates.6–9 Many studies suggest that APs make a significantpositive contribution to patient care,2,10,11 but continuingmedical education resources targeted at these highly skilledclinicians may be disproportionately low when comparedwith those allocated for continuing physician education. Thepurpose of the present curriculum was to provide a continuingeducation resource using HPS simulating acute medicalemergencies that are infrequent but associated with highpatient risk. These conditions require rapid diagnosis andmanagement that, fully trained, practicing APs were reason-ably expected to render in the scope of their current practice.Use of the simulator allowed the opportunity to manage theseconditions in a virtual, yet stressful environment without riskto actual patients. We found that the curriculum was ratedhighly by participants and was qualified as an effective

training tool. Improvement in performance was seen in allspheres, most significantly in leadership/interpersonal skillsand in the written questionnaires. The curriculum was bestvalidated using the Teamwork/Leadership assessment tool(TLIS) where training rendered AP scores indistinguishableto those of a cohort of fully trained intensivists.

High-fidelity human patient simulator training offersthe advantage of prospective and repeated experientiallearning in select short-duration scenarios providing analmost limitless opportunity to practice managing high-stakes events in a risk-free setting.12 This is particularlyvaluable in critical care medicine, which lends itself poorlyto traditional apprenticeship-type bedside teaching. In-deed, when the patient is unstable, the learner is oftenshifted to an observational role and the more experiencedinstructor takes the lead to minimize patient risk. Thestudent then loses key learning opportunities to prioritizetasks, make decisive, time-sensitive management choices in amultidisciplinary setting where leadership is inherently diffi-cult to assume. Medical emergency simulation using HPSprovides the opportunity for real time and subsequent debrief-ing feedback while providing the trainee the opportunity toview and reflect on their own actions and reactions on videoplayback. Although most simulation training in critical carehas centered in procedural skill acquisition,13–17 we choseinstead to focus on the cognitive and leadership skills that areof paramount importance when caring for critically ill pa-tients with time-sensitive conditions.

Although several studies have evaluated clinical decision-making using simulation, this study is the first to evaluate thisin established APs. Simulation of perioperative scenarios hasbeen found to improve anesthesia trainee performance inconducting emergent cesarean section anesthesia.18 In an-other study, attending and resident anesthetists demonstratedimprovement in response time, management techniques andless deviation from accepted protocols after simulating hy-perthermia and anaphylactic shock scenarios.19 In this study,we used simulation scenarios to both train and evaluateparticipants’ clinical management, skills. Additionally, thescenarios afforded the opportunity to assess their leadershipskills and ability to work as a team.

Many simulation curricula evaluate training effect bysubjective participant surveys or by comparing pre- andposttraining multiple-choice examinations. Although ourstudy included this methodology, we also used objectiverubrics. We first evaluated generic universal clinical essen-tials and while there was an improvement trend, this was notsignificant in either APs or fellows. This could have been dueto the fact that trainees felt that these practices were notnecessary in the setting of a simulation. Conversely, thesepractices may have been forgotten in the stress and urgencyof the scenario, which is known to occur in real-life clinicalemergencies.20 Ratings of the ability of APs to rapidly workthrough the scenario’s diagnoses and institute proper man-agement strategies demonstrated an increased trend in post-training scores (81%) that still remained well below those offellows (93%).

Figure 6. Pretraining team leadership/interpersonal skills to-tal scores were significantly lower in APs than in fellows. Af-ter training, scores became similar. Op � 0.001.

Figure 7. Pre- and posttraining emergency clinical care skillstotal scores were higher in fellows than in APs but improve-ment with training was seen in each group. (**p � 0.067;***p � 0.054.)




Other groups have investigated teamwork and lead-ership skills in emergencies; described by some as crisisresource management. Kim et al.21 have described a vali-dated crisis resource management simulation rating scalefor medical residents that is similar to our TLIS check-list— evaluating leadership skills, situational awareness,and decision making (problem solving) but also incorpo-rating resource utilization. In our institutions, APs areoften called emergently by bedside nurses to managemedical crises in the ward or ICU. We demonstrated that 5hours of training with HPS using carefully selected high-stakes, high-risk, low frequency scenarios, resulted in demon-strable improvements in teamwork, decision-making, andsituational awareness skills. Sample size was too small to findsignificant correlation in scores to years of experience orlocation of practice (HUP or TRHMC).

In simulation studies evaluating scoring of traineeskills, validation of assessment tools is sometimes present.One study examining situational awareness using simulationof advanced trauma life support-based scenarios found par-ticipants with a higher level of training to obtain higher scoresrelative to their less experienced colleagues.22 In a morecomplex curriculum, Holcomb et al.23 assessed multidisci-plinary military teams in simulated trauma scenarios anddemonstrated a valid and reproducible evaluation systemwhen comparing scores to those of expert trauma surgeons.Using a different validation scheme, another study assessedeffect of experience of surgical and emergency medicineresidents in managing four simulated critical care patientscenarios.24 Trainees evaluated on completing essential clin-ical tasks, cognitive errors, and directionality of reasoning,obtained higher scores if they had rotated for �10 weeks inthe ICU when compared with counterparts who had rotatedfor a lesser time. In this study, we used fully ACGME-trainedSICU fellows to prove construct validity of the curriculum.Pretraining AP scores were significantly lower than those offellows, whereas posttraining scores were similar in bothgroups using the TLIS assessment tool. This validated thecurriculum as level-adequate for APs. However, improve-ment in ECCS AP and fellow scores were not significant.These latter results may indicate that in the selected charac-teristics evaluated, scores approached each other after someexperience using simulators. Alternatively, posttraining scoreimprovements in both groups may also have been due togreater familiarity and ease in interacting with a high fidelityhuman simulator.

After training, APs felt more confident in managingemergent conditions as they self-reported in the TEQ. Con-fidence in managing emergent ICU situations is an importantquality to rapidly configure a plan and manage rapidly chang-ing conditions.

This study has a number of limitations. The number ofsubjects was small. A greater number of participants mayhave rendered significant some of the improvements seen inECCS scores. Also, many (50%) of trainees had no previousexperience with simulation training and simulation discom-fort may have contributed to lower pre-training scores. Otherinvestigators have described the concept of “stage fright”

when first faced with a simulated patient and note howtrainees may not immerse themselves in the artificiality ofsimulation until repeated exposure.23 We have now instituteda “warm-up scenario” that is performed by all trainees tofamiliarize them with the basic workings of scenario/simula-tion processes before beginning the training curriculum. Ad-ditionally, attempting to rate complex behavior such as thatwhich surrounds the management of a critically ill patient isinherently fraught with difficulty. Some performance mea-sures were rated as either inadequate or adequate; althoughwe attempted to standardize definitions of these ratings; somedegree of subjectivity is doubtless present. In addition, be-cause it was not possible to blind all assessors to the identitiesof participants, the possibility of interpersonal bias in scoringis present. Furthermore, only three expert assessors were usedto evaluate each trainee. The nature of video-recorded per-formances available for web evaluation may allow signifi-cantly more assessment input in future studies. Despite theselimitations, we still found that there was a very high degree ofinterrater reliability in participant scores between the threeassessors, suggesting that the application of the measurementtool was reliable.

In conclusion, we have demonstrated that a short,half-day curriculum evaluating five easily simulatable emer-gent ward/ICU scenarios can be useful in improving the skillsof ICU APs. Such a curriculum may improve leadership andAP teamwork skills whereas at the same time increasing theirconfidence levels in managing high-risk scenarios. Moregeneralized institution of this curriculum may better targetcontinuing medical education needs for this group of experi-enced clinicians.

REFERENCES1. Education ACfGM. Report of the ACGME Work Group on Resident

Duty Hours. Available at: http://wwwacgmeorg/new/wkgreport602pdf2002. Accessed November 17, 2010.

2. Tume L. Remodelling the paediatric ICU workforce: there is a case forimplementing advance nurse practitioner roles into all paediatric inten-sive care units. Nurs Crit Care. 2010;15:165–167.

3. Knudson MM. Teamwork in the Trauma Bay Assessment Tool ACS/APDS Surgical Skills Curriculum for Residents, Phase 3: Module 1.2008. ACS. Available at: http://www.facs.org/education/surgicalskills.html.

4. Advanced Level Practical Examination, Patient Assessment—Trauma.National Registry of Emergency Medical Technicians. 2000. Availableat: https://www.nremt.org/nremt/about/nremt_news.asp.

5. Moorthy K, Munz Y, Forrest D, et al. Surgical crisis management skillstraining and assessment: a simulation[corrected]-based approach to en-hancing operating room performance. Ann Surg. 2006;244:139–147.

6. Howard F, Brown KL, Garside V, Walker I, Elliott MJ. Fast-trackpaediatric cardiac surgery: the feasibility and benefits of a protocol foruncomplicated cases. Eur J Cardiothorac Surg. 2010;37:193–196.

7. Green A, Edmonds L. Bridging the gap between the intensive care unitand general wards—the ICU Liaison Nurse. Intensive Crit Care Nurs.2004;20:133–143.

8. Jastremski CA. Using outcomes research to validate the advancedpractice nursing role administratively. Crit Care Nurs Clin North Am.2002;14:275–280.

9. Lustbader D, Fein A. Emerging trends in ICU management and staffing.Crit Care Clin. 2000;16:735–748.

10. Winkelman C, Maloney B, Kloos J. The impact of obesity on criticalcare resource use and outcomes. Crit Care Nurs Clin North Am.2009;21:403–422.




AQ: 3

AQ: 4

http://wwwacgmeorg/new/wkgreport602pdf

http://www.facs.org/education/surgicalskills.html

http://www.facs.org/education/surgicalskills.html

https://www.nremt.org/nremt/about/nremt news.asp

11. Aragon D, Sole ML. Implementing best practice strategies to preventinfection in the ICU. Crit Care Nurs Clin North Am. 2006;18:441–452.

12. Hammond J. Simulation in critical care and trauma education andtraining. Curr Opin Crit Care. 2004;10:325–329.

13. LeBlanc VR, Tabak D, Kneebone R, Nestel D, MacRae H, Moulton CA.Psychometric properties of an integrated assessment of technical andcommunication skills. Am J Surg. 2009;197:96–101.

14. Parent RJ, Plerhoples TA, Long EE, et al. Early, intermediate, and lateeffects of a surgical skills “boot camp” on an objective structuredassessment of technical skills: a randomized controlled study. J Am CollSurg. 2010;210:984–989.

15. Mayo PH, Hackney JE, Mueck JT, Ribaudo V, Schneider RF. Achievinghouse staff competence in emergency airway management: results of ateaching program using a computerized patient simulator. Crit CareMed. 2004;32:2422–2427.

16. Wahidi MM, Silvestri GA, Coakley RD, et al. A prospective multicenterstudy of competency metrics and educational interventions in the learn-ing of bronchoscopy among new pulmonary fellows. Chest. 2010;137:1040–1049.

17. Kruglikova I, Grantcharov TP, Drewes AM, Funch-Jensen P. Assess-ment of early learning curves among nurses and physicians using ahigh-fidelity virtual-reality colonoscopy simulator. Surg Endosc. 2010;24:366–370.

18. Scavone BM, Sproviero MT, McCarthy RJ, et al. Development of anobjective scoring system for measurement of resident performance onthe human patient simulator. Anesthesiology. 2006;105:260–266.

19. Chopra V, Gesink BJ, de Jong J, Bovill JG, Spierdijk J, Brand R. Doestraining on an anaesthesia simulator lead to improvement in perfor-mance? Br J Anaesth. 1994;73:293–297.

20. Evanoff B, Kim L, Mutha S, et al. Compliance with universal precau-tions among emergency department personnel caring for trauma patients.Ann Emerg Med. 1999;33:160–165.

21. Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J. A pilot study usinghigh-fidelity simulation to formally evaluate performance in the resus-citation of critically ill patients: The University of Ottawa Critical CareMedicine, High-Fidelity Simulation, and Crisis Resource Management IStudy. Crit Care Med. 2006;34:2167–2174.

22. Hogan MP, Pace DE, Hapgood J, Boone DC. Use of human patientsimulation and the situation awareness global assessment technique inpractical trauma skills assessment. J Trauma. 2006;61:1047–1052.

23. Holcomb JB, Dumire RD, Crommett JW, et al. Evaluation of traumateam performance using an advanced human patient simulator forresuscitation training. J Trauma. 2002;52:1078–1085; discussion 1085–1086.

24. Young JS, Stokes JB, Denlinger CE, Dubose JE. Proactive versusreactive: the effect of experience on performance in a critical caresimulator. Am J Surg. 2007;193:100–104.




JOBNAME: AUTHOR QUERIES PAGE: 1 SESS: 1 OUTPUT: Sat Apr 23 07:20:38 2011/balt5/zta-ta/zta-ta/zta99909/zta2751-09z

AQ1— Please specify what “ABC” denotes here.

AQ2— Kindly define MI.

AQ3— Please define ACGME.

AQ4— Please provide accessed date for the refs. 3 and 4.

AUTHOR QUERIES

AUTHOR PLEASE ANSWER ALL QUERIES 1

short simulation training improves objective skills in ... practitioners managing emergencies on the...

Documents