self and peer assessment institute of teaching and learning seminar series jan fermelis faculty of...

Self and Peer AssessmentSelf and Peer Assessment

Institute of Teaching and Learning Institute of Teaching and Learning Seminar SeriesSeminar Series

Jan FermelisJan FermelisFaculty of Business and LawFaculty of Business and Law

14 August 200814 August 2008

Seminar OutlineSeminar Outline

1.1. Team assignment challenges and strategiesTeam assignment challenges and strategies

2.2. Online SAPA demonstrationOnline SAPA demonstration

3.3. Discussion Discussion

ITL Seminar 14 Aug 08ITL Seminar 14 Aug [email protected]@deakin.edu.au

22

Team FormationTeam Formation

Unit realitiesUnit realities Size/nature of student cohortSize/nature of student cohort Number of campuses/instructorsNumber of campuses/instructors

Teaching/learning aimsTeaching/learning aims Self-selectedSelf-selected

Reduced likelihood of team wars or divorcesReduced likelihood of team wars or divorces Instructor-selectedInstructor-selected

Improved opportunities for intercultural and Improved opportunities for intercultural and teamwork skillsteamwork skills


33

Teaching/Learning StrategiesTeaching/Learning Strategies Integration of teaching/learning/assessment Integration of teaching/learning/assessment

experiencesexperiences Lecture and reading materialsLecture and reading materials

Team dynamicsTeam dynamics Interpersonal communication skills and strategiesInterpersonal communication skills and strategies

Tutorial activities:Tutorial activities: Discussion of team member attributes and role Discussion of team member attributes and role

expectationsexpectations Speed Meeting exerciseSpeed Meeting exercise Tutorial time for team meetingsTutorial time for team meetings

Student opportunities to reflect and to trial strategiesStudent opportunities to reflect and to trial strategies Instructor availability to mentor teamsInstructor availability to mentor teams


44

Assessing Team AssignmentsAssessing Team Assignments

Shared score for all team membersShared score for all team members Instructor workloadInstructor workload Student perceptions of fairnessStudent perceptions of fairness

Individualised scoresIndividualised scores Paper systemsPaper systems Online systemsOnline systems


55

Online Self and Peer AssessmentOnline Self and Peer Assessment

Pioneered by Richard Tucker, Architecture & Pioneered by Richard Tucker, Architecture & BuildingBuilding Improved quality of workImproved quality of work Student preference for individualised assessmentStudent preference for individualised assessment

STALGS grant for cross-disciplinary pilot STALGS grant for cross-disciplinary pilot within Business Communication and within Business Communication and Architecture/DesignArchitecture/Design Testing in a more complex educational contextTesting in a more complex educational context Ongoing development and refinementOngoing development and refinement


66

Research Program Phase 1Research Program Phase 1

Implementation (arch and bus comm)Implementation (arch and bus comm) Compulsory useCompulsory use

Penalty for missed entries (arch)Penalty for missed entries (arch) Reward for entries made (bus comm)Reward for entries made (bus comm)

Individualisation of assessmentIndividualisation of assessment Voluntary entry and exit questionnairesVoluntary entry and exit questionnaires Ongoing development and refinement of the Ongoing development and refinement of the

softwaresoftwareITL Seminar 14 Aug 08ITL Seminar 14 Aug [email protected]@deakin.edu.au

77


Compulsory implementation (arch)Compulsory implementation (arch) Reward for entries made Reward for entries made

Optional implementation (bus comm)Optional implementation (bus comm) Neither rewards nor penaltiesNeither rewards nor penalties

Voluntary entry and exit questionnairesVoluntary entry and exit questionnaires Ongoing development and refinement of the Ongoing development and refinement of the

softwaresoftware


88


No implementation of SAPANo implementation of SAPA No individualisation of assessmentNo individualisation of assessment Voluntary, online questionnairesVoluntary, online questionnaires


99

Software Enhancements 1Software Enhancements 1 Three types of assessment per SAPA entryThree types of assessment per SAPA entry

Relative contribution (0.5-1.5)Relative contribution (0.5-1.5) Individual performance (1-5)Individual performance (1-5) CommentsComments

Visual attractiveness of SAPA websiteVisual attractiveness of SAPA website Password protection for studentsPassword protection for students Refinement of Multiplicative Scaling FactorRefinement of Multiplicative Scaling Factor


1010

Software Enhancements 2Software Enhancements 2 Process for keying in team and member detailsProcess for keying in team and member details

Username checkUsername check Add/change team membersAdd/change team members Password resetPassword reset

Unit co-ordinator capacity to edit settingsUnit co-ordinator capacity to edit settings Easy unit co-ordinator access to results:Easy unit co-ordinator access to results:

Students not in groupsStudents not in groups Number of student entry submissionsNumber of student entry submissions Student ratings and commentsStudent ratings and comments


1111

SAPA Procedural EnhancementsSAPA Procedural Enhancements

Motivation and training of staffMotivation and training of staff Motivation and training of studentsMotivation and training of students User friendliness of entry processUser friendliness of entry process Clarity of SAPA instructionsClarity of SAPA instructions Hotlinked email remindersHotlinked email reminders


1212

Some Initial ResultsSome Initial Results

Positive student attitude to individualisationPositive student attitude to individualisation Almost no complaints about lowered marksAlmost no complaints about lowered marks Varying levels of student complianceVarying levels of student compliance

Large, complex vs small, single teacher cohortLarge, complex vs small, single teacher cohort On-campus vs. off-campus teamsOn-campus vs. off-campus teams Compulsory vs optionalCompulsory vs optional Penalty vs rewardPenalty vs reward Staff encouragement vs discouragementStaff encouragement vs discouragement


1313

For Further ResearchFor Further Research

Student self-censorshipStudent self-censorship Visibility of MSF ratings and commentsVisibility of MSF ratings and comments Continuous vs overall SAPAContinuous vs overall SAPA Potential for student collusionPotential for student collusion Inclusion vs. exclusion of SELF assessmentInclusion vs. exclusion of SELF assessment Impact of cross-cultural factors on extreme Impact of cross-cultural factors on extreme

responsesresponses


1414

self and peer assessment institute of teaching and learning seminar series jan fermelis faculty of...

Documents