The Evolution of WebPA, an The Evolution of WebPA, an online peer moderated online peer moderated

marking system, at marking system, at LoughboroughLoughborough

Melanie Bates, Melanie Bates, Learning Technology Co-Learning Technology Co-ordinator ordinator

Loughborough UniversityLoughborough University

BackgroundBackground

The value of group workThe value of group work

Group marks and the need for deriving Group marks and the need for deriving individual marks from themindividual marks from them

Peer assessment as a mechanism of Peer assessment as a mechanism of rating individual contribution in group workrating individual contribution in group work

WebPA: the evolution 1WebPA: the evolution 1

Paper

Excel

WebPA

PASS

Paper-based peer assessment

A computerised peer assessment system

PASS (Online Peer Assessment System)

WebPA (Web-based Self and Peer Assessment)

Users

DeveloperLead Academic

engCETL

WebPA

WebPA: the evolution 2WebPA: the evolution 2

Rationale for implementation of Rationale for implementation of WebPAWebPA

In terms of the Institutional policy:In terms of the Institutional policy:There are the requirements of accrediting bodies which clearly state There are the requirements of accrediting bodies which clearly state that students should have had ‘experience of working in groups or that students should have had ‘experience of working in groups or teams’.teams’.An internal report (Blease, 2006) states that group work takes place An internal report (Blease, 2006) states that group work takes place in over 500 modules in all departments at Loughborough. in over 500 modules in all departments at Loughborough. This practice clearly suggests the need for a mechanism of deriving This practice clearly suggests the need for a mechanism of deriving individual marks in group work.individual marks in group work.

In terms of online peer assessment and its benefits:In terms of online peer assessment and its benefits:Widely recognised benefits of peer assessment Widely recognised benefits of peer assessment The ability to deal with large data and automate data collection and The ability to deal with large data and automate data collection and analysisanalysisConfidentiality and flexibilityConfidentiality and flexibilityMarking not prone to human errors Marking not prone to human errors

The evaluation of WebPA at The evaluation of WebPA at LoughboroughLoughborough

The following are reported as strengths of the system in internal investigations (Pond et al., The following are reported as strengths of the system in internal investigations (Pond et al., 2006; Robinson, 2006; Willmot & Crawford, 2004; 2005):2006; Robinson, 2006; Willmot & Crawford, 2004; 2005):Students perceive the process to be fairStudents perceive the process to be fairFlexible and easy to useFlexible and easy to useConfidentiality Confidentiality The system makes a complicated process very simple to implementThe system makes a complicated process very simple to implementSetting up forms and groups takes only a few minutes Setting up forms and groups takes only a few minutes The results of the assessment are instantly available regardless of the group sizeThe results of the assessment are instantly available regardless of the group sizeA variety of reports can be easily created based on various mark sheets according to parameters. A variety of reports can be easily created based on various mark sheets according to parameters. Ease of tracking or monitoringEase of tracking or monitoringData can be easily downloaded into ExcelData can be easily downloaded into ExcelStudent satisfactionStudent satisfaction

The following are reported as issues with peer assessment (Pond et al., 2006; Robinson, 2006; The following are reported as issues with peer assessment (Pond et al., 2006; Robinson, 2006; Willmot & Crawford, 2004; 2005):Willmot & Crawford, 2004; 2005):Possible group collusionPossible group collusionNon-participationNon-participationFull marks in self-assessmentFull marks in self-assessmentExtreme subjectivityExtreme subjectivity

The WebPA projectThe WebPA project

The WebPA project was launched in October, The WebPA project was launched in October, 2006.2006.JISC funded, 2 year, research and development JISC funded, 2 year, research and development project.project.The project partnersThe project partners

- engCETL (Engineering Centre for Excellence - engCETL (Engineering Centre for Excellence in Teaching and Learning), in Teaching and Learning), Loughborough UniversityLoughborough University - The University of Hull- The University of Hull - Higher Education Academy Subject Centres in - Higher Education Academy Subject Centres in Engineering and Physical SciencesEngineering and Physical Sciences

ChallengesChallenges

Within the Institution:Within the Institution:Increasing the number of usersIncreasing the number of usersTraining issues (students and academics)Training issues (students and academics)Produce pedagogic modelsProduce pedagogic models

Outside the institution:Outside the institution:Making the system institution-independentMaking the system institution-independentIdentifying and persuading other institutions to implement WebPAIdentifying and persuading other institutions to implement WebPABuilding up the community of usersBuilding up the community of usersSupport issues - technical support Support issues - technical support

- support for embedding the system in teaching - support for embedding the system in teaching Training issues (students and academics)Training issues (students and academics)

Future and further software Future and further software developmentdevelopment

An improved version of WebPA will become open-source. An improved version of WebPA will become open-source. There is a need for:There is a need for:

- building up a community of users - building up a community of users - an effective communication system between users and the project team - an effective communication system between users and the project team - a system using which the users feed into further system development.- a system using which the users feed into further system development.

There is a need for sustainability strategies for the software to stay alive There is a need for sustainability strategies for the software to stay alive after the life of the project.after the life of the project.

The features WebPA could integrate include the following:The features WebPA could integrate include the following:Finding a way of making the system institution-independentFinding a way of making the system institution-independenta demo version for potential usersa demo version for potential usersFacility for students to justify why they give the ratings they give to their Facility for students to justify why they give the ratings they give to their peerspeersProvide more examples of assessment criteria, scoring ranges etc.Provide more examples of assessment criteria, scoring ranges etc.

ReferencesReferencesBlease, D. (2006). Blease, D. (2006). The organisation and assessment of group work at Loughborough The organisation and assessment of group work at Loughborough

universityuniversity. Unpublished internal report. Loughborough: Loughborough University.. Unpublished internal report. Loughborough: Loughborough University. Falchikov, N. (1991). Group process analysis: Self and peer assessment of working together in Falchikov, N. (1991). Group process analysis: Self and peer assessment of working together in

a group. a group. Self and Peer Assessment, , 63Self and Peer Assessment, , 63 15-28. 15-28. Pond, K., Coates, D., & Palermo, O. A. (2006). Pond, K., Coates, D., & Palermo, O. A. (2006). Student perceptions of peer review marking of Student perceptions of peer review marking of

team projectsteam projectsRobinson, C. L. (2006). Self and peer assessment in group work. TRobinson, C. L. (2006). Self and peer assessment in group work. The SEFI MWG Seminar, he SEFI MWG Seminar,

Kongsberg.Kongsberg. Sommervell, H. (1993). Issues in assessment, enterprise and higher education: The case for Sommervell, H. (1993). Issues in assessment, enterprise and higher education: The case for

self-, peer and collaborative assessment.self-, peer and collaborative assessment. Assessment and Evaluation in Higher Education, Assessment and Evaluation in Higher Education, 1818(3), 221-233.(3), 221-233.

Willmott, P., & Crawford, A. (2004). Online peer assessed marking of team projects. Willmott, P., & Crawford, A. (2004). Online peer assessed marking of team projects. International Conference on Engineering Education, International Conference on Engineering Education, Gainsville, Florida. Gainsville, Florida.

Willmott, P., & Crawford, A. (2005). Validating the assessment of individuals within Willmott, P., & Crawford, A. (2005). Validating the assessment of individuals within undergraduate teams. undergraduate teams. International Conference on Engineering Education, International Conference on Engineering Education, Gliwice, Gliwice, Poland.Poland.

ContactsContacts

The WebPA project: The WebPA project: http://webpaproject/index.php?section=1http://webpaproject/index.php?section=1Mailing list: Mailing list: http://http://www.jiscmail.ac.uk/lists/webpa.htmlwww.jiscmail.ac.uk/lists/webpa.html