alt-c 2004 evaluating the e-learning experience prof tony toole university of glamorgan september...

ALT-C 2004ALT-C 2004

Evaluating the e-Learning Evaluating the e-Learning ExperienceExperience

Prof Tony TooleProf Tony Toole

University of GlamorganUniversity of Glamorgan

September 2004September 2004


SummarySummary

Based on a JISC project evaluating the effectiveness Based on a JISC project evaluating the effectiveness of the e-College Wales MLE {part of the of the e-College Wales MLE {part of the MLEs for MLEs for Lifelong Learning Lifelong Learning programme}programme}

Compared the learning journey taken by the Compared the learning journey taken by the students with that intended by the learning students with that intended by the learning designersdesigners

Found that a ‘one size fits all’ model does not work Found that a ‘one size fits all’ model does not work for the lifelong learnerfor the lifelong learner

Led to a new learning design model based on Led to a new learning design model based on student choice to be developed and piloted by the student choice to be developed and piloted by the Wales e-Training NetworksWales e-Training Networks


Evaluation MethodologyEvaluation Methodology

Focus group meetings and semi-structured Focus group meetings and semi-structured interviews with all key stakeholdersinterviews with all key stakeholders

Instructional designers, content authors, Instructional designers, content authors, multimedia specialists, web programmersmultimedia specialists, web programmers

Programme managers, e-moderators, customer Programme managers, e-moderators, customer support teamsupport team

Students at all institutions in delivery network, Students at all institutions in delivery network, including those at level 1, 2 & 3 of the degree including those at level 1, 2 & 3 of the degree programme and a substantial number of those programme and a substantial number of those who had withdrawnwho had withdrawn


The Learning JourneyThe Learning Journey

Structured materials

Navigation

Structured tasks

Discussion Forum

Assessment


The Learning JourneyThe Learning Journey

The learning was intended to be linear, the The learning was intended to be linear, the students working through the materials students working through the materials topic by topictopic by topic

The students were expected to complete The students were expected to complete the tasks associated with each topic and the tasks associated with each topic and contribute to the discussion about itcontribute to the discussion about it

The students would then complete the The students would then complete the module assessment and progressmodule assessment and progress


But ….But …. Our lifelong learners turned out to be Our lifelong learners turned out to be

transactional students with different levels transactional students with different levels of experience, expertise, learning styles of experience, expertise, learning styles and reasons for studyingand reasons for studying

As a result many didn’t take the route we As a result many didn’t take the route we had designed for them & chose a path that had designed for them & chose a path that matched their way of doing thingsmatched their way of doing things

The route they took depended on their The route they took depended on their motivation for taking the course and what motivation for taking the course and what personal benefits they wanted it to deliverpersonal benefits they wanted it to deliver


Our typical experienced student with Our typical experienced student with an eye on the qualification would …an eye on the qualification would …

Go straight to the assessment to find out what Go straight to the assessment to find out what was requiredwas required

Scan the materials to identify the scope and Scan the materials to identify the scope and depth of the moduledepth of the module

Examine the tasks to get a better feel for what Examine the tasks to get a better feel for what the assessment outcomes were expected to bethe assessment outcomes were expected to be

Participate in the discussion forums for the Participate in the discussion forums for the purpose of confirming the conclusions drawnpurpose of confirming the conclusions drawn

Target the topics where skills and knowledge Target the topics where skills and knowledge reinforcement was requiredreinforcement was required

Complete the assessment when the skills and Complete the assessment when the skills and knowledge necessary for the desired grade were knowledge necessary for the desired grade were judged to have been attainedjudged to have been attained


Our typical experienced student with the Our typical experienced student with the intention of developing their skills would intention of developing their skills would

……

Scan the materials to identify the areas of useful Scan the materials to identify the areas of useful newnew knowledge knowledge

Read thoroughly through the materials they had Read thoroughly through the materials they had chosen, including additional resources, and carry chosen, including additional resources, and carry out relevant tasksout relevant tasks

Participate in the discussion forums where this Participate in the discussion forums where this contributed to their learning goalscontributed to their learning goals

Complete the assessment when the deadline Complete the assessment when the deadline approached. The main objective here was to approached. The main objective here was to progress to the next module with the minimum progress to the next module with the minimum effort rather than achieve a good gradeeffort rather than achieve a good grade


Our typical inexperienced student with Our typical inexperienced student with the intention of developing their skills the intention of developing their skills

would …would …

Largely follow the linear structure of the course Largely follow the linear structure of the course and read through all materialsand read through all materials

Complete the tasks and participate in the Complete the tasks and participate in the discussion forums up to the point where no-one discussion forums up to the point where no-one else was contributingelse was contributing

Attempt the assessment when the deadline was Attempt the assessment when the deadline was approachingapproaching

Welcome f2f tutorials/informal student meetings Welcome f2f tutorials/informal student meetings to compare notes and reinforce level of to compare notes and reinforce level of attainmentattainment


Lessons we have learned ….Lessons we have learned ….

We need to provide multiple routes through the We need to provide multiple routes through the materials and allow students to make their own materials and allow students to make their own choiceschoices

We need to communicate proactively with the We need to communicate proactively with the students, not reactivelystudents, not reactively

We need to increase the number of assessment We need to increase the number of assessment points and provide formative feedbackpoints and provide formative feedback

We need to exploit the skills of experienced We need to exploit the skills of experienced students for the benefit of those with no experiencestudents for the benefit of those with no experience

We need to provide plenty of opportunities to We need to provide plenty of opportunities to communicate for those who are most comfortable communicate for those who are most comfortable in a community of learners, whilst allowing those in a community of learners, whilst allowing those who wish to plough a lone furrow to do sowho wish to plough a lone furrow to do so


A New Learning Design A New Learning Design ModelModel Based on constructivist, situated learning Based on constructivist, situated learning

approach employing discovery learning approach employing discovery learning activities in communities of practiceactivities in communities of practice

Modular with clusters of credit rated Modular with clusters of credit rated learning outcomes. Three levels of learning outcomes. Three levels of learning support resourceslearning support resources

Student chooses own mix of learning Student chooses own mix of learning outcomes to attain, level of support outcomes to attain, level of support required and where & when the learning required and where & when the learning will take placewill take place

Negotiated learning agreement with Negotiated learning agreement with module tutormodule tutor


Module Descriptor{This will explain the learning objectives of the module and the number of activities to be carried out}

Learning Activities{Learning Activity 1. Description, learning outcomes, student choices, examples}

.{Learning Activity N. Description, learning outcomes, student choices, examples}

Choose Learning Activity{choose personal application, agree with tutor}

Need Learning Materials?

Level 1 Materials{Bullet point explanations, assumes knowledge of software, menu structures and all basic functionality}

{bullet point 1: explanation}.

{bullet point N: explanation}

Level 2 Materials{Further explanation of topic, example, exercise)

Need Further Materials?

Level 3 Materials{Full topic coverage, examples, exercises, tutorial)

Need Further Materials?

Y

Y

Y

N

NN

Complete Learning Activity{discuss drafts with tutor}{agree final version}{submit}

Learning Activities Complete?

Y

N

End Module{discuss with tutor}

Start Module{discuss with tutor}


The Wales On-line Foundation The Wales On-line Foundation DegreeDegree

Collaboration of all post-16 Collaboration of all post-16 institutions in Walesinstitutions in Wales

Shared development of FD modules Shared development of FD modules Common learning design modelCommon learning design model Shared digital repositoryShared digital repository Validated at NVQ level 4Validated at NVQ level 4 All institutions able to exploit All institutions able to exploit

materials in whatever way they wishmaterials in whatever way they wish


For further information on:For further information on:

E-College WalesE-College Wales

The Welsh e-Training NetworksThe Welsh e-Training Networks

Contact: Contact: [email protected]@virtualcollege.ac.uk

alt-c 2004 evaluating the e-learning experience prof tony toole university of glamorgan september...

Documents

learning journeythe

learning goalscomplete

learning designersfound

lifelong learning programme

module assessment

new learning design

assessment outcomes

discussion forums