Assessing the Assessing the practical/clinical skills of practical/clinical skills of

students:students:

Presented at the RITAL Conference 2010Presented at the RITAL Conference 2010By: Florence Davidson, Radiography By: Florence Davidson, Radiography

Lecturer, CPUT, GSH campus & Lecturer, CPUT, GSH campus & Curriculum Officer for RadiographyCurriculum Officer for Radiography

77thth December 2010 December 2010

Type of assessmentType of assessment

SimulationSimulation OSCEOSCE Clinical (practical, in the workplace)Clinical (practical, in the workplace)

Contents:Contents:

Presentation: (15 minutes)Presentation: (15 minutes) Defining “clinical competence”Defining “clinical competence” Types of instruments used for Types of instruments used for

assessing competenceassessing competence Drafting criteria for rating a clinical skillDrafting criteria for rating a clinical skill Aspects of defining “performance”Aspects of defining “performance” Additional considerationsAdditional considerations Round table discussion: (30 Round table discussion: (30

minutes)minutes)

A model of competenceA model of competence

Fitness for purpose: focuses on the immediate requirements of the workplace

Fitness for award: focuses on the educational preparation of the graduate

Fitness for practice: focuses on the legal aspects of the individual practitioner’s capabilities

A model of competence (Williams and Berry, 1999)

CPUTGRADUATE

Defining clinical Defining clinical competencecompetence

1. Novice

2. Advanced beginner

3. Competent

4. Proficient

5. Expert

The Dreyfus and Dreyfus model of skill acquisition (adapted from described Benner (2001.)

KEY:•Novice: 1st time in clinical,

no experience.

• Advanced beginner: some clinical exposure, but unable

to adapt to differentsituations.

• Competent performer:typically at the end of training/mentoring, able to cope, ready

registration.

• Proficient performer: has experience to modify their

response.

• Expert: has an intuitive grasp,can “hone in” on the problem.

Observable Practice

Potential practice

Professional field

•OP: what the HCP is observed doing in practice

* PP: what the HCP does not ordinarilydo (?specialties,? extended scope), but there may be reasonable expectation

• PF: holistic practiceencompassing all the above as well as key competencies

KEY:

Adapted from Melnick et al (2002)

Practice domains and assessment

““Technical ability”Technical ability”

What are we

assessing?

Motor skillsPrevious

experiencesCognitive elements

Adaptability to

new and changing

situations

Ability to work withpeers and

other Health Care Professionals

Procedural

checklistsGlobal

Rating scales

Instruments for assessing “technical” Instruments for assessing “technical” skills:skills:

1. Structured Technical Skills Assessment Form 1. Structured Technical Skills Assessment Form (STSAF)(STSAF)

Instruments for assessing “technical” Instruments for assessing “technical” skills:skills:

2. Global Rating Scales (GRS):2. Global Rating Scales (GRS):

Respect for tissueOften used unnecessary force on tissue or caused damage by inappropriate use of instruments

Careful handling of tissue but occasionally caused inadvertent damage

Consistently handled tissues appropriately, with minimal damage

Time and motion Many unnecessary moves Efficient time and motion, but some unnecessary moves

Economy of movement and maximum efficiency

Instrument handling

Repeatedly makes tentative or awkward moves with instruments

Competent use of instruments, although occasionally appeared stiff and awkward

Fluid moves with instruments and no awkwardness

Knowledge of instruments

Frequently asked for the wrong instrument or used an inappropriate instrument

Knew the names of most instruments and used appropriate instrument for the task

Obviously familiar with the instruments required and their names

Use of assistants

Flow of operation and forward planning

Knowledge of specific procedure

(Krishna Moorthy, Yaron Munz, Sudip K Sarker, Ara Darz:2003)i

Variable 1 2 3 4 5

Drafting criteria for rating a Drafting criteria for rating a clinical skill (STSAF)clinical skill (STSAF)

1.Decide on the type

of student to be assessed

4. Write down the performancecriteria you want to see a student

do to demonstrateacceptable skill

5. Discuss, revise and

select a master list ofcriteria

2. Decide on the setting forthe assessment

3. Select theclinical skill you

want to evaluate

(Adapted from Watts, N: 1990)

Essential elements in the Essential elements in the construction of the instrumentconstruction of the instrument

Inter-rater Inter-rater reliabilityreliability

Construct validityConstruct validity

10/1010/10

10/1010/10

10/1010/10

1st year: 42%

2nd year:

63%

3rd year:

88%

Essential elements in the Essential elements in the construction of the instrument construction of the instrument

Aspects of “performance”Aspects of “performance” ObservableObservable -(things you can see, hear or -(things you can see, hear or

feel someone do);feel someone do); EssentialEssential – not simply desirable, or – not simply desirable, or

characteristics of an excellent characteristics of an excellent performance, but performance, but necessary necessary in order to be in order to be acceptably safe, effective acceptably safe, effective and/or and/or efficient;

Directly related to this skill;Directly related to this skill; Likely to be called for under most (routine)

circumstances when this skill is employed(Watts, N: 1990)

Other considerations:Other considerations:

How often we need to see a student How often we need to see a student attempt the task before we are prepared attempt the task before we are prepared to judge their level of mastery;to judge their level of mastery;

What types or degrees of error would say What types or degrees of error would say mastery is not yet acceptable;mastery is not yet acceptable;

The importance of procedural steps v The importance of procedural steps v speed and smoothness;speed and smoothness;

Routine v unfamiliar situations; andRoutine v unfamiliar situations; and How many times until we are convinced of How many times until we are convinced of

mastery?mastery?

Discussion pointsDiscussion points

What What type of instrumentstype of instruments are you are you using to assess the practical using to assess the practical competence of your students?competence of your students?

What What aspectsaspects (technical/procedural/ (technical/procedural/ professional) are you assessing?professional) are you assessing?