Plastic Surgery residents’ perception of feedback in the operating room – a comparison of structured and non-structured approaches

Y Sardiwalla (BSc), N Joukhadar (MD), D Tang (MD)

Disclosure:

•No conflicts of interest to disclose

Background: •Competence By Design (CBD)

•In-Training Evaluation Report (ITER) •Quality of feedback limited

Background: •Ottawa Surgical Competency Operating Room

Evaluation (O-SCORE) •11-item tool •5-point scale

•Competence to perform procedures independently

Background: •Junior and senior residents

•Internal consistency and evidence validity •Paucity of evidence regarding residents’ experience with

this evaluation

Purpose: •To use this period of transition to more comprehensively

study what benefits and disadvantages residents perceive in each of the feedback tools

Methodology:

•Dalhousie Plastic Surgery residents •PGY 2-5 levels

•Semi-structured focus group •Exit survey rating

Methodology:

Analysis:

•Grounded theory •Constant comparative analysis. •Survey median values and interquartile ranges

qualitatively assessed

Results

•90% residents participated in focus group •60% fully completed survey.

Four Major Themes:

1. Traditional intraoperative feedback evaluation •Informal feedback •Faculty-dependent •Timing

RESULTS:

Four Major Themes:

2. O-SCORE strengths •Quality and timing •Specific •Longitudinal

RESULTS:

Results

•“I think the O-SCORE is definitely a step in the right direction. Since it has been introduced the staff and I actually sit down and review it. You actually talk about the case more. But what I found was even more useful, is when you pick a case that you’re doing your O-SCORE for and decide this a few days before, it encourages playing a more active part in the planning.”

Four Major Themes:

3. O-SCORE weaknesses •Selectivity •Generous evaluations

RESULTS:

Results

•“One thing to note with the O-SCORE is that because we only have 3 that are mandatory per rotation, you can pick which cases to use. I’m going to pick a very standardized case which will get you all 5s so then you look great on your piece of paper.”

Four Major Themes:

4. Considerations for the future •Standardized portion of O.R. •Surgery specific evaluations

RESULTS:

O-SCORE ITER

Results

Discussion: •Resident satisfaction

•Concerns •Adding to routine •Training to how use O-SCORE

Limitations and Interpretation: • Lack of a validated survey • Small sample • Plastic surgery unique challenges • Supervising surgeon preferences

DISCUSSION:

Conclusion • O-SCORE complemented traditional feedback. • Residents state O-SCORE improves:

• Opportune feedback • Comprehensive discussion • Improved progress tracking

DISCUSSION:

References: 1. Maudsley R, Dagg P, Wilson J, et al. Competence by design. R Coll Physicians Surg Canada. 2014;(March).

2. Weller JM, Misur M, Nicolson S, et al. Can i leave the theatre? A key to more reliable workplace-based assessment. Br J Anaesth. 2014;112(6):1083-1091. doi:10.1093/bja/aeu052.

3. Bindal N. Trainee doctors’ views on workplace-based assessments: Are they just a tick box exercise? Med Teach. 2011;33(11):919.

4. Lorello GR, Cook DA, Johnson RL, Brydges R. Simulation-based training in anaesthesiology: A systematic review and meta-analysis. Br J Anaesth. 2014;112(2):231-245. doi:10.1093/bja/aet414.

5. Blum RH, Boulet JR, Cooper JB, Muret-Wagstaff SL. Simulation-based Assessment to Identify Critical Gaps in Safe Anesthesia Resident Performance. Anesthesiology. 2014;120(1):129-141. doi:10.1097/ALN.0000000000000055.

6. Rall M, Manser T HS. Key elements of debriefing for simulator training. Eur J Anaesthesiol. 2000;17:516-518.

7. Okuda Y, Bryson E, DeMaria S, Jacobson L, Quinones J, Shen B LA. The utility of simulation in medical education: What is the evidence? Mt Sinai J Med. 2009;76(4):330-343. doi:10.1002/MSJ.

8.Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ. The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE). Acad Med. 2012;87(10):1401-1407. doi:10.1097/ACM.0b013e3182677805.

9.Budden CR. Using the Ottawa Surgical Competency Operative Room Evaluation (O-SCORE) in a Canadian Plastic Surgery Program. 2016.

10 MacEwan MJ, Dudek NL, Wood TJ, Gofton WT. Continued Validation of the O-SCORE (Ottawa Surgical Competency Operating Room Evaluation): Use in the Simulated Environment. Teach Learn Med. 2016;28(1):72-79. doi:10.1080/10401334.2015.1107483.

11. Sherbino J. The Ottawa Surgical Competence Operating Room Evaluation A Tool to Assess Surgical Competency. Key Lit Med Educ Pod Rev.

Thank You!

Registration and Ethics:

•Dalhousie Research Ethics Board (REB #: 2017-4322)