E-xcellence: quality in the changing e-learning landscapeKaren Kear, Jon Rosewell, Angeles Sánchez-Elvira, Miguel Santamaria, Keith Williams, Ebba Ossiannilsson, Covadonga Rodrigo, Harvey Mellar

Quality Assurance for e-learning• Context:

– Increasing use of e-learning in Higher Education – QA procedures were designed for face-to-face

education– They don’t necessarily fit e-learning and blended

learning• E-xcellence approach:

– Provide resources and processes for QA of e-learning

Quality Assurance spectrumCheck compliance with quality standards

Support institutions in quality enhancement

E-xcellenceFlexibility to:

v set relevant goals v demonstrate quality of performancev plan for improvement

E-xcellence resources• E-xcellence manual (in PDF format)

– 35 benchmark statements– Performance indicators– Supporting text

• Quickscan tool• Assessors’ notes• Framework for action roadmap

See: e-xcellencelabel.eadtu.eu/

E-xcellence manual and framework

1. Strategic Management2. Curriculum Design3. Course Design4. Course Delivery5. Student Support6. Staff Support

See: e-xcellencelabel.eadtu.eu/

Quickscan• Quick self-assessment of the HE institution’s e-learning • Uses the 35 E-xcellence benchmarks• Helps institution to identifies strengths and weaknesses• Online version provides guidance and feedback• Best used collaboratively by an interdisciplinary team

The Quickscan tool

E-xcellence review

• Onsite visit (1-2 days) or online meetings• Engage with the HE institution in constructive

discussions about the quality of their e-learning, and how it might be improved

• Provide an independent external review, with advice for improvement

• Collect feedback on the E-xcellence resources, in order to improve them

Review format• Preparation

– Decide programme(s) to be reviewed– Form interdisciplinary team of the HE institution’s managers, course

designers, educators, students, technical staff– Team meets to complete QuickScan self-evaluation

• Visit or online meetings– HE team meets with E-xcellence reviewers (2 - 4 e-learning experts)– Discuss institution’s e-learning offerings and the QuickScan self-evaluation– Reviewers give initial feedback and suggestions

• Reports– Summary report from E-xcellence reviewers– Reviewers consider and agree Roadmap for improvement from the

institution

E-xcellent manual update• Reflects recent trends in e-learning

– rapid rise of MOOCs– surge of interest in learning analytics– increasing use of learning design in course development

• Includes new topics– increased focus on personalization– flipped approaches to teaching– virtual and remote laboratories– digital badges and e-portfolios.

In this workshop we …

• Explore the new E-xcellence benchmarks and indicators• Consider which indicators are helpful for assessing

quality in the rapidly changing e-learning landscape

e-xcellencelabel.eadtu.eu

Strategic management

• Example benchmark– The institution has an e-learning strategy that is widely

understood and integrated into the overall strategies for institutional development and quality improvement. E-learning policies conform to legal and ethical frameworks.

Strategic management• Significant changes

– Benchmark 2 … there is an organisational framework that can foster innovation and development, and evaluation by scholarship and research.

– Benchmark 5 … development of Open Educational Resources and Massive Open Online Courses, or use of social media), the roles and responsibilities are clearly defined, communicated to those concerned, and controlled by operational agreements where appropriate.

• Text updates for additional issues– New section on Learning analytics– Paragraphs on MOOCs– New indicators for policies on: MOOCs, learning analytics, monitoring

changing technology and educational practice

Curriculum Design

• Example benchmark– Curricula that use e-learning offer personalisation and a flexible

path for the learner, while ensuring the achievement of learning outcomes.

Curriculum Design...significant changes

• Flexibility– Pre-requisites for registration and course stay to foster inclusion of life-long learners– Possibility of conforming personalized curricula – Existence of virtual mobility

• Academic community development: Evidence of online collaborative learning and team work• Professional Networks: Existence of virtual mobility for professional practices• Knowledge and Skills

– Existence of mechanisms for recognizing prior learning– Results of student’s survey regarding fulfilment of user’s expectations

• Assessment: – Clarification of Formative and Summative assessment definitions– Evidence of online collaborative learning and team work by existence of:

• Student’s peer-to-peer review• Project-based work assessment

– Information on student identification in non-proctored examining procedures: use of identity cards, fingerprints, …

Curriculum Design

• No significant changes in benchmarks• Additional indicators at excellence level:

– The curriculum includes authentic practical work which may be achieved online by virtual or remote laboratory facilities. (2.2.3)

– The institution provides digital badges that can be used to recognise generic skills and other achievements not covered by formal assessment. (2.3.1)

– There are opportunities for virtual internships or other ways of developing employability skills. (2.3.2)

Course Design

• Example benchmark– E-learning materials have sufficient interactivity (student-to-

content, student-to-student and student-to-teacher) to encourage active engagement and enable students to test their knowledge, understanding and skills.

Course Design (significant changes)

• Minor changes in benchmark wording• Additional indicator at excellence level

• Learning analytic data is used to gain insight into the success of aspects of e-learning design. This is used to improve the design of future courses. (3.5)

• Text changes– New section on MOOCs (no indicators – see OpenupEd!)– Terminology: ‘components’ à ‘elements and activities’– Other issues:

• Monitoring of student feedback• Vocational / professional competencies in outcomes• Student workload specified as part of design• Focus on user friendly & motivating user interface

Course Delivery

• Example benchmark– The systems for communication and storage of data are secure,

reliable and assure appropriate levels of privacy. Measures are in place for system recovery in the event of failure or breakdown.

Course Delivery

• No significant changes in benchmarks• Changes to indicators:

– There are secure systems for storing and analysing data on students’ learning activity and interactions with university online systems (learning analytics data). (4.1.1)

– Operating standards are implemented effectively (deleted) (4.1.2)– The VLE captures data that can be used for learning analytics approaches and there

are appropriate reporting tools such as dashboards for users of this data. (4.2.1)– The system allows for personalisation. (4.2.1)– Plagiarism detection systems are available and staff conform to policies on the use of

these systems. (4.2.5)– Learning analytics approaches use data from online assessments. (4.2.5)

• Changes to text:– Notes on mobile– Notes on learning analytics

Staff support

• Example benchmark– The institution ensures that appropriate training and support is

provided for staff and that this training is enhanced in the light of technological and educational developments.

Technical aspects Technical support Technical training

Educational aspects Educational support Educational innovation

Resources Information and media support Administrative support

Career development, incentives and recognition

6 benchmarks

• 30 indicators

• 17 indicators of excellence

Staff support• Significant changes: New indicators to address e-learning evolution

– Technical Support for …• Use of new types of software

– Tools for multimedia self-production • Learning analytics

– Educational Support for …• Mobile and ubiquitous learning• Merging of formal and informal learning• Pedagogical use of learning analytics• Indexing and archiving of e-learning materials to promote institutional reuse and OER

– Administrative Support• Improvement of the administrative Support provided by Study Centres

– Career Development• Formal organization of new roles and functions associated with e-learning within the

institution• Clear recognition of career progression and incentives in terms of e-learning contributions

Student support

• Example benchmark– Students have access to learning resources, including online library

access, study skills development and a study advisor, and they receive guidelines and training in using these resources.

5 benchmarks

• 67 indicators

• 22 indicators of excellence

3rd Edition

The chapter has been restructured and reorganized

Specific areas have been renamed

New areas and indicators have been added

Student supportStudent support organization

Institutional student support planningSupport needs for different learner groups

Support staffHuman resource planningSupport role definitionsAdministrative support

Technical supportOnline services availabilityProfessional management of technical supportOnline technical support system availability

Pedagogical supportAdvice and guidance on study skills developmentSupport for e-learning skills development

Support resourcesCourse choice advice and guidanceLibrary resourcesLearner communitiesRole of study centres in student support delivery

§ Significant changes

A systemic institutional perspective

Over to you…

• Work in groups of three• Choose one chapter that is of interest to you• Think of a specific new e-learning development

– Can you find an appropriate indicator / benchmark?• this development is evidence of achievement

– No appropriate indicator / benchmark?• suggest brief paragraph for ‘Additional notes’

e-xcellencelabel.eadtu.eu/tools/manual

1. Strategic Management2. Curriculum Design3. Course Design4. Course Delivery5. Student Support6. Staff Support

Plenary discussion

Thank you!

• Please contact us to continue discussions

• For more info on E-xcellence:w: e-xcellencelabel.eadtu.eu

e: secretariat@eadtu.eu