applying the evidence: portfolio assessment in midwifery margaret fisher senior lecturer in...

APPLYING THE EVIDENCE:PORTFOLIO ASSESSMENT IN

MIDWIFERY

Margaret FisherSenior Lecturer in Midwifery/ Ceppl Activity Lead,

University of Plymouth

Alison StoneMidwife, Torbay Hospital/ Ceppl Team member

www.placementlearning.org

Email: mfisher@plymouth.ac.uk

9th Annual Interdisciplinary Research Conference, Dublin, 7/11/08

Background: Triggers for change

• Ceppl research: “Assessment of Practice”

• Literature evidence

• Anecdotal feedback

• Professional body requirements

• Re-validation of Midwifery curriculum

Evidence from the literature

•Assessment of practice is crucial in determining whether or not a student meets the criteria required of their profession, thus ensuring safety of the public

UKCC 1999, Watkins 2000, Cowburn et al 2000

•Defining competence has long been a challenge Cowan et al 2005

•Efforts to ‘measure’ competence and professional abilities have resulted in a wide variety of methods of assessment

Baume and Yorke 2002, McMullan et al 2003

•The student may also be required to fulfil set criteria or outcomes and practice may be graded Caraccio and Englander 2004, Slater and Boulet

2001

•Unless outcomes are clear, the result may be that the student focuses too heavily on completing the portfolio [or other tool] rather than learning from the experience itself

Scholes et al 2004

•Reflections on practice may form part of portfolio assessments, and this process may also contribute to the student’s learning

Mountford and Rogers 1996

Ceppl project: Assessment of Practice

• Longitudinal case studies

• Staff focus groups

• Literature search

• Trawl of websites

• Conference networking

Aim

To establish an evidence-based set of key principles and

resources to guide

Assessment of Practice,

relevant across professional boundaries.

Research Questions

1. What are perceptions of validity and reliability of the practice assessment methods used?

2. What are perceptions of the impact of the practice assessment process on the student learning experience?

Methodology

• 14 participants from Midwifery, Social Work and Emergency Care programmes (nurses and paramedics)

• Semi-structured interviews at the end of each year

• Longitudinal case study approach• Single-case and cross-case analysis

and synthesis of findings – “Framework technique”

Ritchie and Spencer 1984

Methods used

1. Portfolios *

2. Reflections

3. Tripartites/ 3-way meetings *

4. Criterion referenced assessments *

5. Conversations

6. Observations

7. OSCEs

Portfolios

Provide focusEvidence of

capability/ achievement

Encourage student as see their progress

Self-directedMotivate learning

×Prescriptive/ restrictive (“tick boxes”)×Weighting of marks

unbalanced/ difficult to assess×Potential to “cheat

the system”×Bulk (paper format)×Heavy workload

Tripartites/ 3-way meetings Useful checkpoint Opportunity to reflect

on progress and learning

Opportunity to get feedback from mentor and tutor

Enable clarification of issues

Student-centred Reliable if student and

mentor have worked closely together

× Difficult to arrange× May be challenging to

express conflicting opinions

× Likened to a “parent’s evening”

× Some students though mentor and tutor should also have private discussion

Criterion referenced assessment

Focused learning

Best if continuous assessment

Mostly valid, reliable and achievable

× Criteria not always relevant to placement

× Some criteria ambiguous/ overly complex/ unclear

× Dependent on professional judgement and experience of mentor

Key themes

People

GUIDANCE

Consistency

Becoming a

professional Doing the job

PURPOSE

PROCESS

Timing Clarity Placements

Paperwork

Application to Midwifery: A new curriculumIncorporating new NMC requirements:

• All graduate• Grading of practice• Sign-off mentor throughout programme• Ongoing Achievement Record• Case-loading and focus on normality• Essential Skills Clusters (ESC’s) for

Midwifery – theory and practice

And a new Portfolio

Portfolio work-party

Decision to develop part-paper (summative)

and part-electronic (formative/ evidence learning)

portfolio

Key changes to the portfolio• Ongoing Achievement Record (OAR) –

paper component

• Electronic portfolio

• Assessment tool: MAP (Mentor Assessment of Proficiencies) based on Essential Skills Clusters and Midwifery Proficiencies and graded annually during tripartite meeting

• Entire portfolio summative at end of programme – pass/ fail

Ongoing Achievement Record(paper component)

Comprises all documents needing signature by sign-off mentor or others:

1.Signature record

2.Mandatory sessions

3.EU Midwifery record

4.Placement record

5.Learning contract

6.MAP

MAP CRITERIA

Communication and Confidentiality *

Evidence Based Practice and Health Education/Promotion

Personal and Professional Growth

Risk Assessment and Management Skills

Initial Consultation between the Woman and the Midwife *

Ongoing Antenatal Care

Normal Labour and Birth *

Postnatal and Neonatal Care

Initiation and Continuation of Breastfeeding *

High Risk Care across the Childbirth Continuum

Emergency Procedures

Medicines Management *

Electronic PortfolioEvidence of student’s practice

learning:

Learning through SWOT analysis and reflection

How Midwifery Proficiencies, Essential Skills Clusters and learning outcomes achieved

Evidence of case-loading and other core clinical experiences (eg: non-maternity placements, skills packs) Demonstration

Pilot of Portfolio: Methodology

• 12 students and mentors (students with a range of IT abilities and learning styles)

• 7 lecturers

• 6-week trial including tripartite

• Questionnaire follow-up

Pilot of portfolio: Key findings

Guidelines: very positive

Hyperlinks: • logical system

• tricky to begin with but became easier with use

• particularly useful when making external links (eg: to national guidelines)

Feedback facility: valuable

Varied IT skills/ learning styles: coped

Summary

→

Clear understanding of

PURPOSE

Safe, competent

PRACTITIONERS

who have achieved

PERSONAL and

PROFESSIONAL

GROWTH

Clear, consistent and timely

PREPARATION

Optimise the

PROCESS

People Placement Paperwork

Copyright statement:

Copyright for this portfolio belongs to Margaret Fisher in collaboration with the University of Plymouth Midwifery,

Ceppl and Emily teams