University of Saskatchewan e-Portfolio Study:Report & Discussion of FindingsforLancaster University - LEARNING TECHNOLOGY GROUP

CONFERENCE 15TH MAY 2009 - E-ASSESSMENT AND E-FEEDBACK

This project was funded by the Saskatchewan Learning TEL Initiative.

Organization of this presentation

1. Definitions

2. The UofS e-portfolio research study

3. Criticisms of e-portfolios

4. Discussion

Defining an e-portfolio

1. An e-space where students:– Store and organize visual, textual, and media artifacts during their formal HE

studies – Are provided formative and/or summative assessment on their

learning/progress– Can engage in a dialogue with tutors and peers on assessment issues– Are able to further carry on with augmenting the portfolio during life-long

learning.

• “A flexible, evidence-based tool that engages students in a process of continuous reflection and collaborative analysis of learning [which] captures the scope, richness, and relevance of students’ learning” (Zubizarreta, 2005)

• In practice, however, e-portfolio implementations vary broadly.– Purpose– Tools and functionalities– Pedagogical approach

• Variations can be grouped into types of e-portfolios

Types of e-Portfolios

1. LEARNING e-portfolios are designed to support learning and to be used for formative assessment. They tend to be structured, templated, and simple to use. This structure and simplicity comes as a trade-off for allowing room for creativity.

2. ASSESSMENT e-portfolios are designed to support formal assessment of student work need to be structured around underpinning curricula, learning goals, and criteria for assessment. They differ from the first type in that they need to be more flexible in order to allow students’ opportunities to demonstrate creativity. As a result, they are usually a bit more complex to use.

3. PROFESSIONAL, EMPLOYMENT or SHOWCASE e-portfolios need to allow users even greater flexibility and creativity in their presentation of artifacts. These portfolios display tailored collections of work for specific employers or a specific job. Therefore, students need to be able have control over the criteria by which artifacts are chosen.

4. INTEGRATIVE e-portfolios begin as learning portfolios, evolve into assessment portfolios, and become, in the end, showcase portfolios.

Types of e-Portfolios across Sampled Disciplines:

So many terms, so little time… Computer Science

Dentistry Drama Education

Learning Learning Formative Assessment

Assessment Assessment Structured Summative Assessment

Employment Employment Showcase Showcase

Integrative

Overview of Benefits & Implementation Requirements

Student Tutors Institution Benefits

Resource for academic papers and presentations; develop critical thinking and self-assessment skills; competitive advantage in acquiring employment

Additional support for enhancing student learning; an avenue to provide meaningful feedback and assessment

Documentation of student achievement; evidence for accreditation; add rigor to programs; long-term tracking of students; recruiting tool, marketing tool

Implementation Needs

Training; encouragement from tutors; appropriate structure(s) and support

Training; time to develop learning goals and assessment strategies

An e-portfolio system that can accommodate multiple structures and functions

Benefits for Students

• Resource for storing academic papers, presentations, and projects

• Develop critical thinking and self-assessment skills through:

(1) storing a record of progress and learning over time

(2) engaging in assessment discussions with tutors and peers

(3) having a resource that records & organizes drafts, critiques/feedback & revisions

• Competitive advantage in acquiring employment

Student Needs

• Training – both technical and pedagogical• Encouragement and feedback from tutors• Appropriate structure (s) and support• Recognition for the time commitment to

portfolio development

Instructor Benefits

• Additional support for enhancing student learning

• An avenue to provide meaningful feedback and assessment that is embedded in student-generated artifacts of learning

• Evidence for teaching portfolios

Instructor Needs

• Training – both technical & pedagogical• Time to develop learning goals, e-portfolio

requirements, assessment strategies, and evaluation rubrics

• Time to provide students with feedback • Recognition for time commitment

Institutional Benefits

• Documentation of student achievement• Evidence for accreditation purposes• Added rigor to programs• Avenue for long-term tracking of students• Recruiting tool / marketing tool

Institutional Needs

• An e-portfolio system that can accommodate multiple structures and functions

• Policy and procedures for e-portfolio use, access, and longevity

• Training programs for students and tutors• Policy for recognition of student and instructor

time commitments to portfolio development• Funding for needs assessment, design (or

choice of system), implementation, & maintenance

Key Requirements (1)

• Institutional endorsement • Long term institutional commitment to

recognize and support the required commitments from faculty and students.

Key Requirements (2)

• Staff endorsement:– Tutor endorsement is believed to be one of the

most influential factors in student acceptance and use.

– Tutors must be encouraged to engage early in the planning phases of e-portfolio integration to decide how e-portfolios are best used to support learners and programs.

Key Requirements (3)

• Selective Implementation

– Formative e-portfolios allow an instructor to provide continual assessment and feedback in response to student work and reflections to students throughout a course/program.

– Although a powerful model of teaching and learning, significant faculty time is required.

– Mentoring students in this manner is feasible in terms of workload and time requirements for smaller graduate and undergraduate courses, but careful thought must be given about how to manage this approach in larger undergraduate courses/programs.

Key Requirements (4)

• In the case of structured or assessment portfolios:

– Tutor time is NECESSARY to develop • the curricular framework• learning goals• activities with performance measures • Assessment & evaluation rubrics

– If inadequate time and consideration is extended to the definition of learning goals, students will become confused and frustrated from not clearly understanding what is expected.

– This can result in decreased motivation and eventual abandonment

Criticisms of e-portfolios

• “Trailing edge” technology• Examples of spectacular failures in implementation• Intellectual property and copyright concerns• Student privacy and security concerns• Subjective assessment can lead to undependability of

this form of assessment • Validity and verifiability issues• Interoperability concerns• Longevity challenges

Small-group discussions

• Given the potential benefits of e-portfolios and criticisms that have resulted from earlier implementations:– In groups of 3 or 4:– Choose the most important benefit that you can

see for adoption of e-portfolios in your teaching practice.

– Identify what you believe to be the most significant risk factor (s) you might encounter in an attempt to achieve the benefit you have identified.

• Report back your thoughts to the larger group

The full research report is available if you would like to explore this topic further.

Research Group

Special thanks for the student-researchers’ contributions to this project.

Computer Science

Education Dentistry Drama Extension ITS

Jim Greer

Gina Koehn

Chris Brooks

Zinan Guo

Gale Parchoma

Dirk Morrison

Linda Dunkley

Alan Kilistoff

Brian Phee

Moira Day

Heather Morrison

Kathy Schwarz

Kevin Lowey

Thank-you for your consideration of this presentation