an evaluation of the uk dental foundation training portfoliocopdend.org/data/files/meetings/barratt...

An Evaluation of the UK Dental Foundation Training Portfolio

Dr Linda Prescott-Clements

http://www.copdend.org/default.aspx

Context & methods

Data available

Implementation of the portfolio across the UK

Workplace-based assessments ESPR, ADEPT, D-CbD, MSF, PAQ

Monitoring Clinical Experience, Assessment & CPD / Education Logs

Professional Development Plan

Reflection

Current perspectives from Deaneries

Overview

Context

Instructions

- User Guide

- Educational agreement

- Activities Schedule

Reflection

- Critical incidents

- Interesting cases

- reflection on progress

Monitoring

- Clinical experience log

- Assessment log

- CPD / education log

- Personal development plan

Assessment

- ESPR, ADEPT and D-CbD

- PAQ

- Multi-source feedback

- Appraisal statements

DFT Assessment

Portfolio

Methodology

Evaluation of DFT Portfolio

Deanery Interviews

Raw data analysis

Trainer & Trainee Survey

Surveys

Response: 706 trainers & 311 trainees

All deaneries represented except Defence (Scotland not included)

Trainers: 90% GDS, 8% HDS, 4% CDS, 2% Management

Trainees: 295 had worked in GDS, 54 in HDS & 19 in CDS.

Raw Data

20 randomly sampled, anonymised, e-portfolios representing 6 different UK deaneries

Interviews

All deaneries (PD Dean or rep) within study were interviewed.

Data

Assessments and portfolio fully implemented within DF1 across all UK Deaneries.

Target numbers of assessment & reflection being monitored

Steady uptake of e-portfolio – now implemented across all deaneries.

Implementation into DF2 has been more difficult – most deaneries have ‘patchy’ uptake.

Implementation

Early Stage Peer Review (ESPR)

First 4 weeks DF1

Completed by 93.7% trainers

63% trainers identified skills needing targeted training

Majority of trainers & trainees found ESPR helpful

Workplace-based Assessment

A Dental Evaluation of Performance Tool (ADEPT / DEPs)

Observation of performance (monthly)

Majority trainers completed target number (many exceeded), at regular intervals.

Flexibility appears to support implementation across different services




0

10

20

30

40

50

60

Done on an ad hoc basiswhen time available

Planned in advance atregular intervals

Planned in advance -irregular intervals

Multiple ADEPTs done in ablock

GDS

HDS

CDS


Both trainer and trainee satisfaction was HIGH

Majority trainers & trainees in GDS felt ADEPT is a good use of their time (lower in HDS – 1/3 trainees agreed).

Almost 90% GDS trainers happy to assess using ADEPT.

Good support for design features

Median time observing 40min, feedback 20min.

Considered ‘most helpful’ aspect of portfolio by 28% trainers


Dental Case-based Discussion (D-CbD)

Majority trainers completed target number (monthly).

Flexibility of tool appears to help implementation

High satisfaction amongst trainers

60% trainees considered D-CbD ‘a good use of time’, but ~20% disagreed with this statement.

90% trainers (all services) happy to assess using D-CbD, with majority feeling it is a good use of time.


Dental Case-based Discussion (D-CbD)

Support for design features

Presenting / observing 30min

Feedback 30min


Multisource Feedback

Peer Assessment Tool (PAT)

Team Assessment of Behaviour (TAB)

Not recommended in DF1 due to limited number of assessors & difficulties retaining anonymity

Limited data available – suggests uptake disappointing 31% HDS trainers asked to complete MSF

9% CDS trainers asked to complete MSF


Patient Assessment Questionnaire (PAQ)

Recommended DF1 (x1 –at 6months) & DF2 (x2 –end of post)

Limited data available

GDS trainees: Majority (80%) agreed patients happy to assess them

Most (60%) agreed PAQ a ‘good use of time’ – 33% neutral

Almost half had been anxious about asking patients

Majority (80%) thought feedback was motivating


Clinical Experience Log

Original format – first 5 of each procedure

E-portfolio – all procedures logged.

Majority of trainers (68%) and trainees (59%) found this useful

Support from Deans / managers in enabling breadth of clinical opportunities to be identified

Monitoring

Assessment Log

Monitors areas of curriculum assessed as training progresses

Help ensures content validity

Uptake good, although ‘sign-off’ decreased by end DF1

Raw data sample: 71% trainees assessed across all 11 areas of clinical focus by end DF1. (100% assessed across 10/11 areas).

Monitoring

Assessment Log

Monitoring

CPD / Education Log

Completion better in DF1 than in DF2 posts

Trainer satisfaction generally good – 48.5% agreed it had been useful

Almost 10% trainers identified this as being ‘least helpful’ element of portfolio

Monitoring

Professional Development Plan (PDP)

Designed to help trainers monitor progress of trainees with areas identified as needing further development

Raw data indicates completion variable (individualistic data)

Trainer satisfaction generally high (63.5% considered it to be useful)

Most areas identified using ADEPT or reflection

Monitoring

Professional Development Plan (PDP)

Monitoring

ADEPT

D-CbD

Reflection

Other

Guidance: to complete weekly during months 1 & 2 of DF1, then monthly thereafter.

Some deaneries requested weekly reflection throughout?

Additional reflection on tutorials / study days requested in E-portfolio

Whilst 67.5% trainees found it easy to identify cases on which to reflect – 21.6% disagreed.

57% trainees considered reflection to be a good use of time, but 27% did not.

24.3% trainees had ‘invented’ cases to reflect upon

Reflection

Element / aspect of portfolio % trainers identifying this element as being the most helpful

All aspects of the portfolio 3.6 %

Implementation / design features: Ease of access / online Online messaging system Scheduling of assessments etc

9.9 % 1.1 %

10.0 %

Monitoring section: Clinical experience log CPD / tutorial log Professional development plan (PDP) Assessment log (competencies covered

4.7 % 8.2 % 5.1 % 5.6 % 2.4 %

Assessments: ESPR ADEPT D-CbD

10.9 % 27.8 % 29.8 %

Reflections 16.5 % National standardisation 0.2 %

“Most Helpful” aspects of Portfolio

Element / aspect of portfolio % trainers

None - All aspects of the portfolio are useful 14.8 % Most elements 1.0 % Implementation / design features: No access after completion No dual access / split screen Technology / access difficulties Understanding the system / technology Time consuming (general) Time consuming (assessment) Structure / formality

0.8 % 1.9 % 7.3 % 3.2 % 5.7 % 4.3 % 3.8 %

Monitoring section: Clinical experience log CPD / tutorial log Professional development plan (PDP)

0.8 % 7.5 % 9.7 % 0.8 %

Assessments: ESPR ADEPT D-CbD PAT / TAB PAQ

2.4 % 3.8 % 6.5 % 2.4 % 1.0 %

Reflections 5.9 % Duplication / repetition 4.9 %

“Least Helpful” aspects of Portfolio

Best Aspects

Electronic format (no Deaneries wish to return to paper)

Structured assessments (schedule)

Ease of Access

Monitoring tools

Current Perspectives from Deaneries

Worst Aspects / Suggestions for improvement

Further refinement of e-portfolio

Reduce amount of reflection (especially during later stages of training)

Clinical activity log too laborious


Preferences for future direction

ALL support move to satisfactory completion

Refine e-portfolio (access, HDS)

Revisit curriculum for DF2 / separate competencies

Refine structure (schedule) – ‘terms’

Expand into lifelong learning


Contact details

If you would like further information about this work / a copy of the report,

or to enquire about consultancy services, please contact:

[email protected]

Tel: 07866 522155

Dr Linda Prescott-Clements PhD, MHPE (Hons), BSc (Hons), PFHEA

mailto:[email protected]

an evaluation of the uk dental foundation training portfoliocopdend.org/data/files/meetings/barratt...

Documents