Download - Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience
![Page 1: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/1.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Design, implementation and evaluation of a Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience.'generic' ePortfolio: the Newcastle experience.
Simon Cotterill, Tony McDonald, Paul Drummond, Geoff Hammond
School of Medical Education DevelopmentUniversity of Newcastle, UK
![Page 2: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/2.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
SST (Student Support & Tutoring)
FDTL-4 project• ePortfolios in Medicine• Evidencing outcomes• PDP
1st Newcastle-Nottingham PARs project, 1998-2000
2nd Newcastle-Nottingham PARs project, 2000-02
Internet-PARs• DfES funded • PDP focus
BackgroundBackground
![Page 3: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/3.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
• An FDTL-4 project developing ePortfolios for undergraduate Medicine (2002-2005):
• Newcastle University (lead site)
• Leeds University
• Sheffield University
• Dundee University (for consultancy)
Project funded by
BackgroundBackground
![Page 4: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/4.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
AimsAims
1. Personal Development Planning (PDP)
"a structured and supported process undertaken by an individual to reflect upon their own learning, performance and/or achievement and to plan for their personal, educational and career development”
Quality Assurance Agency for Higher Education, 2001.
![Page 5: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/5.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
a) Towards a life-long learning record
School HE CPD
AimsAims
2. Supporting Life-Long Learning (LLL)
b) Encouraging the development of LLL Skills: • Evidencing Learning Outcomes• Transition to Independent learning• Preparation for professional requirements:
- Appraisals- Assessment- Revalidation
![Page 6: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/6.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Aims (technical)Aims (technical)
3. Design a ‘generic’ ePortfolio
-transferable / useful in different contexts
4. Integration with VLEs
![Page 7: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/7.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Context-specific tools
Common tools
‘Generic’framework Common contents structure
• sharing• cross-referencing• searchable• integrated action planning
Customisation• select tools by course/year• outcomes / skills sets• nomenclature• graphics / layout• simple tool editor
• CV• Outcomes / skills log• PDP• Reflective learning diary • Meetings log• SWOT
Integrated into thecommon contents structure
ArchitectureArchitecture
Developed with Open Source products: Zope, MySQL
![Page 8: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/8.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
• create context-specific tools via simple Web forms
Student view
Selecting tools by Course and Year
Course Admin view
‘‘Generic’ framework: CustomisatonGeneric’ framework: Customisaton
![Page 9: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/9.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Configuring learning outcomes / skills sets
Course Admin view
‘‘Generic’ framework: CustomisatonGeneric’ framework: Customisaton
![Page 10: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/10.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
‘‘Generic’ framework: SharingGeneric’ framework: Sharing
![Page 11: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/11.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Piloting & Implementation
The ePortfolio developed at Newcastle has been applied in the following contexts:
2003/4 Undergraduate MedicineContract Research Staff (CPD)Generic Portfolio (project demonstrator)
2004/5 Dentistry (Undergrad + Vocational Training)Undergraduate BiosciencesSt Andrew’s University (Medicine)
![Page 12: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/12.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Generic ePortfolio:Generic ePortfolio:Public DemonstratorPublic Demonstrator http:www.eportfolios.ac.uk
![Page 13: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/13.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Medicine at Newcastle:Medicine at Newcastle:Integration with a bespoke VLEIntegration with a bespoke VLE
![Page 14: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/14.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Common Tools: Common Tools: Learning Diary (Medicine – Newcastle)Learning Diary (Medicine – Newcastle)
![Page 15: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/15.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Context-specific Tools: Context-specific Tools: PDP for Year 4 Student Selected ComponentsPDP for Year 4 Student Selected Components
![Page 16: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/16.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Common Tools: Common Tools: Record courses (CRS)Record courses (CRS)
![Page 17: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/17.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Common Tools: Common Tools: Learning Outcome / Skills (CRS)Learning Outcome / Skills (CRS)
![Page 18: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/18.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Common Tools: Common Tools: SWOT (Bioscience prototype)SWOT (Bioscience prototype)
![Page 19: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/19.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Common Tools: Common Tools: CV (St Andrew’s University)CV (St Andrew’s University)
![Page 20: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/20.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Common Tools: Common Tools: Tutor meetings (St Andrew’s University)Tutor meetings (St Andrew’s University)
![Page 21: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/21.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Context-specific Tools:Context-specific Tools:DentistryDentistry
Project involves 6 Dental schools and 2 Postgraduate Deaneries
![Page 22: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/22.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Promoting Continuity & LLL in Dentistry
Context-specific Tools: Context-specific Tools: DentistryDentistry
![Page 23: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/23.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Integration #2: Blackboard Integration #2: Blackboard (via a ‘tab’)(via a ‘tab’)
![Page 24: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/24.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Integration #2: Blackboard Integration #2: Blackboard (via (via the tool barthe tool bar))
![Page 25: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/25.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Evaluation #1: Evaluation #1: Medicine (Medicine (Years 1&2Years 1&2))
Cotterill SJ, Sarma S, McDonald AM, Bradley P
• Ethical approval / informed consent
• Focus groups + Questionnaires
• Key issues:• Generally well received but what motivation ?
• not assessed• not viewed by tutors (staff interface completed later)
• Need better clarity of purpose• Wanted a more quirky / fun design !• Learning diary - useful at first, less so over time• Good to browse the Learning Outcomes
![Page 26: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/26.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
• Ethical approval / informed consent
• Questionnaires (before and after using the ePortfolio)• Change in awareness of intended learning outcomes ?• Factors influencing use of the ePortfolio• Attitudes and perceptions of using the ePortfolio• Student perception of impact of using the ePortfolio• Evaluation of technical features and ‘usability’
• 186 students completed the portfolio (~100% compliance)
• 105 questionnaires completed (56% of cohort) at 30/06/2004
Evaluation #2: Evaluation #2: MedicineMedicine (Year(Year 4 4)) PDP for PDP for student selected componentsstudent selected components
Cotterill SJ, McDonald AM, Bradley P, Robinson R, Hammond GR
![Page 27: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/27.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Evaluation - positiveEvaluation - positive
“It encouraged me to really give thought to what I wanted to achieve during the option, which was especially useful as this was my first option. As a result of the portfolio I think I got much more out of the option than I would have otherwise.”
“It made me concentrate on creating aims at the start of the option and allowed me to plan the option with my supervisor in a defined way. Overall it made my learning for the option more organised and focused.”
![Page 28: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/28.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Evaluation - barriers Evaluation - barriers && problems: problems:
“Most of the things I learned couldn't be 'measured/quantified' so I felt it was of little value to try and invent a way in which they could.”
“Access to computer as option took place in Barnard Castle, records may have been more detailed if completed straight after meetings etc.”
![Page 29: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/29.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Building the ePortfolio was a useful learning experience
0 10 20 30 40 50
Strongly Disagree
Weakly Disagree
Agree
% responses
80% thought it wasa useful learning experience
0 10 20 30 40 50
Strongly Disagree
Disagree
Weakly Disagree
Weakly Agree
Agree
Strongly Agree
% responses
Having clearly defined intended learning outcomes influenced the way in which I approached the option
72% felt that the LOsinfluenced their approach
![Page 30: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/30.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
0 10 20 30 40 50
Strongly Disagree
Disagree
Weakly Disagree
Weakly Agree
Agree
Strongly Agree
% responses
93% reflected on their learning after their SSC
At the end of the option I spent time considering what I had learned from this option.
![Page 31: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/31.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Conclusions #1Conclusions #1
The ‘generic’ ePortfolio
• Highly configurable
• Can be readily adapted for different contexts
• Existing tools have an emphasis on PDP
-but the portfolio framework can serve other purposes
Integration in VLEs
• Fully integrated into a VLE for Medicine
• Initial work to integrate with Blackboard has been successful
![Page 32: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/32.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Evaluation (in Medicine)
• ePortfolios implemented / piloted in 2003/4
• Demonstrated: Feasibility & Acceptability
• Most students perceive these as beneficial
• However, unless assessed / mandatory the motivation to complete is reduced.
Conclusions #2Conclusions #2
![Page 33: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/33.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
A systematic map and synthesis review of the effectiveness of personal development planning for improving student learning.EPPI-Centre (2002) http://eppi.ioe.ac.uk
Educational Objectives (including PDP)
• Some evidence that PDP had a positive impact on planning and approach to learning (Year 4 SSC)
• Encouraged reflection in some students
• Good preparation (and practice) for LLL & CPD
• However, long-term impact on learning is unknown
• Further research is essential
• Some (limited) evidence that PDP improves learning:
Conclusions #3Conclusions #3
![Page 34: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/34.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Additional Developments:
• ePortfolios Extension Toolkit (ePET)
-A JISC funded project to develop a ‘Web services’
interface for the portfolio (by 04/2005)
http://www.eportfolios.ac.uk/ePET
• IAMSECT ‘Shibboleth’ Single Sign-On project
- JISC funded project
http://iamsect.ncl.ac.uk
Further information on the ePortfolio:
http://www.eportfolios.ac.uk
Simon Cotterill ([email protected])
![Page 35: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/35.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
Implementation of ePortfolios
- Humans are important too !
• Buy-in of key stakeholders:
Learners Clarity of purpose ? Sense of Ownership ?
Tutors & Admin staff Extra work ?
Senior Curriculum staff
• Integration with existing processes
• Training (all)
• Monitoring and evaluation
![Page 36: Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience](https://reader036.vdocument.in/reader036/viewer/2022070417/5681541c550346895dc219bc/html5/thumbnails/36.jpg)
A 'generic' ePortfolio: the Newcastle experience
Background
Aims
Design
Implement
Evaluation
Conclusions
www.epor
tfolio
s.ac.u
k
InstitutionalInstitutionalDataData
Portfolio for Presentation
Portfolio for Application
(job / promotion)
Portfolio for Assessment
Portfolio for Accreditation/Revalidation
PDP(shared)
PDP / Reflective(private)
Portfolio for Appraisal
Learner’sLearner’s‘‘repository’repository’
Central data:TranscriptMIS/ HR data
Programme data:Granular assessment dataOutcomes / skills sets
Selecting sub-sets of portfolio data for different purposes: