Feb 5 2004Feb 5 2004 using assessment to motivate using assessment to motivate

learning. QAAlearning. QAA

Assessment Driven Assessment Driven LearningLearning

Jean CookJean CookJ.cook@gcal.ac.ukJ.cook@gcal.ac.uk

www.calmat.gcal.ac.ukwww.calmat.gcal.ac.uk

OutlineOutline

What are we doing when we assess?What are we doing when we assess? What is Assessment Driven Learning?What is Assessment Driven Learning? How do we operate?How do we operate? What are the benefits to the student?What are the benefits to the student? Are there any benefits to the tutor?Are there any benefits to the tutor? Does it improve pass rates?Does it improve pass rates? Is it cost effective?Is it cost effective?

What should we be doing when we What should we be doing when we assess?assess?

Enabling students get a measure of their Enabling students get a measure of their achievement.achievement.

Giving feedback to teachers on effectiveness. Giving feedback to teachers on effectiveness. Providing statistics for programme boards.Providing statistics for programme boards.

Motivating students to engage in the learning.Motivating students to engage in the learning. Providing feedback so that they can improve and Providing feedback so that they can improve and

remedy their deficiencies.remedy their deficiencies. Helping them consolidate their learning.Helping them consolidate their learning.

What should we be doing when we What should we be doing when we assess?assess?

Enabling students get a measure of their achievement.Enabling students get a measure of their achievement. often by means of a final exam.often by means of a final exam.

Giving feedback to teachers on effectiveness.Giving feedback to teachers on effectiveness. usually when it is too late for that group of studentsusually when it is too late for that group of students

Providing statistics for programme boards.Providing statistics for programme boards. Great! Great!

Providing feedback so that they can improve and remedy Providing feedback so that they can improve and remedy their deficiencies.their deficiencies.

this won’t happen with just a final exam!this won’t happen with just a final exam! Helping them consolidate their learning.Helping them consolidate their learning.

best done as the module progresses rather than at the end. best done as the module progresses rather than at the end.

What is Assessment Driven What is Assessment Driven Learning (ADL)?Learning (ADL)?

Reduced lecture time, more tutorials.Reduced lecture time, more tutorials. Staged assessments, with firm deadlines for Staged assessments, with firm deadlines for

completion of assessments.completion of assessments. Immediate feedback to students on their Immediate feedback to students on their

progress.progress. Assessment as part of the learning-not the final Assessment as part of the learning-not the final

verdict for the student.verdict for the student. Staff get feedback when they can use it!Staff get feedback when they can use it! --and they still get results for Programme Boardsand they still get results for Programme Boards ..

ContextContext

This describes what has been done by the This describes what has been done by the Mathematics Division of Glasgow Caledonian Mathematics Division of Glasgow Caledonian University, in response to the SHEFC University, in response to the SHEFC Progression and Retention initiative.Progression and Retention initiative.

The students are mainly first year students, The students are mainly first year students, studying a ‘Mathematics’ module.studying a ‘Mathematics’ module.

This implementation relies heavily on CAL This implementation relies heavily on CAL software to assist the delivery of the learning software to assist the delivery of the learning material and the assessment.material and the assessment.

CALMAT Mathematics coursewareCALMAT Mathematics courseware

Learning materials with integrated tutorials and assessmentsLearning materials with integrated tutorials and assessments

Separate tutorial and assessment systemSeparate tutorial and assessment system

Management system Management system

Records Controller: both student and tutor use this to view Records Controller: both student and tutor use this to view

progressprogress..

Designer for creating html pages with links to CALMAT Designer for creating html pages with links to CALMAT

componentscomponents

Player for delivery of the Designer’s html pages Player for delivery of the Designer’s html pages

Diagnostic testing facilityDiagnostic testing facility

For more details visit www.calmat.gcal.ac.ukFor more details visit www.calmat.gcal.ac.uk

How we operateHow we operate

Two lectures and three tutorials each week.Two lectures and three tutorials each week.

The tutorials are help in computer labs.The tutorials are help in computer labs.

Tutor groups are limited to 20 students.Tutor groups are limited to 20 students.

Timetable is carefully structured.Timetable is carefully structured.

Restricted number of tutors.Restricted number of tutors.

10 computer based assessments in the last tutorial in 10 computer based assessments in the last tutorial in

weeks 2-6 and 8-12.weeks 2-6 and 8-12.

Specified CAL lessons to complete.Specified CAL lessons to complete.

Exemption given if an average of 70% is achieved.Exemption given if an average of 70% is achieved.

Why did we switch to ADL?Why did we switch to ADL?

We are now teaching a different student bodyWe are now teaching a different student body Increase in student numbers from 6% to almost 50%.Increase in student numbers from 6% to almost 50%. Most seem to need encouraging to study.Most seem to need encouraging to study. More time in PT work than spent studying.More time in PT work than spent studying. Lectures now satisfy a handful of students and Lectures now satisfy a handful of students and

possibly the lecturer.possibly the lecturer.

We are under pressure to improve pass rates We are under pressure to improve pass rates and have already pared down syllabuses to and have already pared down syllabuses to essentials already.essentials already. Does this sound familiar?Does this sound familiar?

More (mainly maths) reasons to More (mainly maths) reasons to switch to ADL?switch to ADL?

Most students need to be encouraged to enjoy their Most students need to be encouraged to enjoy their first year maths module! first year maths module!

We really ought to be helping students study the We really ought to be helping students study the whole syllabus, particularly in first year mathematics.whole syllabus, particularly in first year mathematics.

We were concerned how little some of our students We were concerned how little some of our students had to know to pass a module!had to know to pass a module! If you set 8 questions on 100% of syllabus and they have If you set 8 questions on 100% of syllabus and they have

to answer 5(63%)to answer 5(63%) Need 40% to pass which implies a student may know only 25% of the Need 40% to pass which implies a student may know only 25% of the

syllabus.syllabus. If you set 5 questions on 100% of syllabus and they have If you set 5 questions on 100% of syllabus and they have

to answer 3(60%)to answer 3(60%)• Need 40% to pass which implies a student may know only 24% of Need 40% to pass which implies a student may know only 24% of

the syllabus.the syllabus.

Benefits to the studentBenefits to the student

Quality feedback and help from week 1.Quality feedback and help from week 1. The exemption provides motivation.The exemption provides motivation. The software is widely available, and very supportive.The software is widely available, and very supportive.

What the students say:What the students say: Prefer weekly tests, keeps momentum and requirement for Prefer weekly tests, keeps momentum and requirement for

continuous work going.continuous work going. Did not feel they needed the lectures: Did not feel they needed the lectures: but staff not willing to but staff not willing to

give them up!give them up! Would have liked complete coverage of content with CAL Would have liked complete coverage of content with CAL

lessons.lessons. General satisfaction with module.General satisfaction with module.

Benefits to the tutorBenefits to the tutor

Weekly information on student progress.Weekly information on student progress. Ability to target help.Ability to target help. Reduced marking. Reduced marking. Satisfaction when students do well.Satisfaction when students do well.

What the staff say:What the staff say: If the students had worked as hard in previous years, they would If the students had worked as hard in previous years, they would

have passed!have passed! It was hard work, despite the use of computers to deliver and It was hard work, despite the use of computers to deliver and

mark tests.mark tests. I am sure the students have learnt more than they did in I am sure the students have learnt more than they did in

previous years.previous years.

Progression StatisticsProgression Statistics

  start revised Exempt Pass Pass % % %

    roll   diet1 diet2 exempt diet1 diet2

99/00 50 47   37 4   79 87

00/01 40 37   29 4   78 89

01/02 59 57 42 11 0 74 93 93

02/03 38 34 24 5 0 71 85 85

99/00 107 96   66 5   69 74

00/01 148 125   74 14   59 71

01/02 133 117 97 5 0 83 87 87

02/03 143 128 109 0 4 85 85 88

99/00   79   48 3   61 65

00/01   78   54 10   69 82

01/02 151 141 95 21 3 67 82 84

02/03 108 100 91 2 1 91 93 94

Summary of resultsSummary of results

In general, 2nd diet pass rates were In general, 2nd diet pass rates were achieved at the first diet.achieved at the first diet.

The effect of not having to resit a The effect of not having to resit a mathematics module would, no doubt, mathematics module would, no doubt, contribute to improving the progression contribute to improving the progression rates for the programmes as a whole.rates for the programmes as a whole.

Is ADL cost effective?Is ADL cost effective?

Savings are due toSavings are due to More students progressing to year 2.More students progressing to year 2. Staff time saved in marking exam papers.Staff time saved in marking exam papers.

In the first year of operation, the extra fee In the first year of operation, the extra fee income and staff time saved in marking income and staff time saved in marking examination papers is estimated at over examination papers is estimated at over £60K. This assumes 2001-2 intakes and £60K. This assumes 2001-2 intakes and 2000-1 pass rates, and a lecture replaced 2000-1 pass rates, and a lecture replaced by several small tutorial groups.by several small tutorial groups.

To summariseTo summarise

Assessment Driven Learning makes Assessment Driven Learning makes assessment a part of the learning process.assessment a part of the learning process.

Students are motivated by the exemption.Students are motivated by the exemption. Second diet pass rates are achieved at the Second diet pass rates are achieved at the

first diet.first diet. Tutors gain satisfaction from seeing Tutors gain satisfaction from seeing

students do well.students do well. It is cost effective.It is cost effective.

““Assessment methods and requirementsAssessment methods and requirements

probably have a greater influence on how probably have a greater influence on how

and what students learn than any other and what students learn than any other

single factor. This influence may well be of single factor. This influence may well be of

greater importance than the impact of the greater importance than the impact of the

teaching materials.” (Boud 1988)teaching materials.” (Boud 1988)