The Ecosystem of Quality Standards for Learning, Education, and Training

International trends & ISO perspectives of quality assurance in e-Learning

Prof. Dr. Jan M. Pawlowski14.02.2008

Licensing: Creative Commons

You are free:to Share — to copy, distribute and transmit the work

to Remix — to adapt the work

Under the following conditions:Attribution. You must attribute the work in the manner

specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).

Noncommercial. You may not use this work for commercial purposes.

Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one.

http://creativecommons.org/licenses/by-nc-sa/3.0/

Where I am from...

My background

E-Learning Internationalization / Culture

Sensitive Information Systems

Integration of KM and E-Learning

Reference Models

Quality Management and Assurance for E-Learning

Mobile & Ambient Learning

Standardization

Learning Object and Learning Design Repositories

Knowledge Management Integration Scenarios

Knowledge Management Support

E-Business

Projects COSMOS: Exchange of Scientific

Content

ASPECT: Open Content and standards for schools

iCOPER: New standards for educational technologies

Korean German Institute of Knowledge and Educational Excellence

Quality Initiative E-Learning in Germany

Virtual Education in Business Information Systems

The European Quality Observatory

European Foundation for Quality in E-Learning

Establishing Sharing Cultures in Organizations (ESCIO)

Contents

Quality Standards: Where we are…

ISO/IEC 19796-x: Quality Standards for Transparency

The Future of Quality Standardization: Trends and Directions

The Quality Ecosystem

• What is the Quality of an Ecosystem?• Diversity?• Stability?• Profitability?

• How to find your way through it?

Perspectives on the Ecosystem

Protecting the user…

(1) Quality Standards provide agreed, consensual frameworks to support stakeholders in the field of quality for learning, education, and training.

(2) Standards enable guidance, reusability, interoperability, and flexibility.

(3) Standards shall be open, transparent, adaptable, and extensible to meet the needs of their user groups.

Individual Quality through Standardization

Return on Investment

Learner Satisfaction

Strategic Impact

ProductivityPersonality

Development

Process Optimization

Customer Satisfaction

Organization‘sSuccess

Market Share

Media QualityMethod MixIndividualization/

Personalization

Interoperability Ergonomics

Learning Culture

Availability

Flexibility

Accessibility

Individual Quality Profile

Harmonisation and Standardisation

A Quality Standard can only be an open framework – An open framework is the foundation for harmonization – Harmonization does not mean unification or limitations

but living diversity

Standardization in quality means to find... – a common language– common reference frameworks– common description formats

...which are able to harmonize the diversity of approaches

…to promote tools to develop “Individual Quality”

Quality Standards Map

CEN/ISSS WS LTEFQUEL

SFS (Finnish Standards Body)

KoreaELC / Japan

Quality Activities Worldwide

ISO/IEC JTC1 SC36

Reference Framework for the Description of Quality Approaches: ISO/IEC 19796-1

NANeeds Analysis

FAFramework

Analysis

CDConception/

Design DPDevelopment/

Production

IMImplementation

LPLearning Process /

Realization

EOEvaluation/ Optimization

Reference Framework for the Description of Quality Approaches: ISO/IEC 19796-1

ID Category Process Name

Description Relations

3.2 Conception / Design

Concept of the con-tents

Concept of learning and teaching contents

1.1 Demand analysis

2.2 Qualifications

Sub proc-ess(es)

Content selection

Content Design

Objective

1. Learner Demand: The goal is to provide contents adapted to the needs and demand of the learner.

2. Adaptation: Each course shall provide different content presentation formats and entry points based on the user experience.

Method

1. A prototype of the content shall be provided to a group of learners’ representa-tives. In a consensus process, the contents shall be prioritized and agreed on.

2. For each course, classify groups of learners according to their learning type. Adapt presentation format and methods according to these learning types.

Result

1. Documentation of planned and agreed contents

2. Periodically, evaluate learning performance of different learners (test groups).

Actors Curriculum designer, didactic experts, institution accreditation authority, teacher, learn-ers’ representatives

Metrics / Criteria

The content are measured based on their relevance, importance, exemplaricity, …

Standards Higher Education Standards

Annotation / Example

Guidance

RFDQ as your guideline…

Discuss quality for all parts of the E-Learning Lifecycle

Identify relevant…– Aspects

– Processes

– Actors

– Solutions

Develop your individual profile using a standardized description

Selection

Find your approach…

Compare Quality Approaches

Compare Contexts

Decision for an approach

Combining Approaches

Assemble your individual approach…

Choose candidates– Process guidelines from ISO 9000 or EFQM or

LQW?– Material guidelines from XYZ?– Competency definitions from ABC?

Choose parts of candidates to develop your own approach

Adapting Approaches

Individualize it…

Identify needs and requirements

Improve the approach to fit your organization‘s needs

Choose relevant methods & metrics & criteria

Start the quality debate

Benchmark your solution to others

Further Parts of ISO/IEC 19796-x

ID

Category

Process Name

Description

Relations

Sub-processes / sub-aspects

Objective

Method

Result

Actors

Metrics / Criteria

Standards

Annotation / Example

Metric Description Scheme

Method Description Scheme

Reference Metric Collection

Reference Method Collection

Good Practice Guide

Harmonized Quality Model

ISO/IEC 19796-2: Harmonized Quality Model

Quality for Organizations– Policy and strategy– Management– Processes– Staff management– …– Innovation

Quality for Products, Services & Solutions– Information on learning offers– Learning objectives / target group– Structure– Content– Didactics– Media– Communication / Collaboration– …

ISO/IEC 19796-3: Reference Methods and Metrics

Reference “Collection” to support implementation and adoption

Common language to describe, exchange, and re-use methods and metrics

Reference Methods– … to achieve quality objectives– E.g., interviews, fishbone method, questionnaire

Reference Metrics– …to measure the achievements– E.g., function metrics, learner evaluation, statistical

data

ISO/IEC 19796-4: Good Practice Guide

Implementation and Adoption Support

Quality Adaptation Model

Based on European Work – European Good Practice Guide– http://www.qualityfoundation.org/ww/en/pub/

efquel/elearning/downloads/specials/cen_isss_qd.htm

Sample Usage: Quality Support System

Infopool

„Core“ Quality Integration Tool Pers. Workspace

Search

User Interface

Repository

QualityPerspectives

Business Processes

QM-Methods& Instruments

Educational Processes

User Perspectives

Summary Common description format quality in the E-Learning lifecycle

Common quality language

Reference processes

Reference criteria

Selection & Comparison

Combination

Adaptation

Basis for a global quality framework

But…there is still some way to go…

Inclusion & Participation

Join our discussion

References

Pawlowski, J.M.: The Quality Adaptation Model: Adaptation and Adoption of the Quality Standard ISO/IEC 19796-1 for Learning, Education, and Training, Educational Technology & Society, 10 (2), 2007. http://www.ifets.info/journals/10_2/2.pdf

Pawlowski, J.M.: The Quality Mark E-Learning: Developing Process- and Product-oriented Quality for Learning, Education, and Training, Int. J. Learning Technology, 3 (1), 2007.

ISO/IEC JTC1 SC36: ISO/IEC 19796-1:2005 Information technology - Learning, education and training - Quality management, assurance and metrics - Part 1: General approach, 2005

CEN/ISSS Workshop on Learning Technologies: CWA 15660 Providing Good Practice for E-Learning Quality Approaches, Brussels, 2007, ftp://ftp.cenorm.be/PUBLIC/CWAs/e-Europe/WS-LT/CWA15660-00-2007-Feb.pdf

Ehlers, U.D., Pawlowski, J.M. (Eds.): European Handbook of Quality and Standardisation in E-Learning. Springer, Berlin Heidelberg 2006.

Contact Information ITRI

Prof. Dr. Jan M. Pawlowskijan.pawlowski@titu.jyu.fi

Skype: jan_m_pawlowski

Office:

Room 525.3

Telephone +358 14 260 2596

Fax +358 14 260 2544

www.titu.jyu.fi