1|Date 07-02-2012

E-learning Quality Assurance Benchmarking in Higher Education

Fatimah Alsaif Johann Bernoulli Institute of Mathematics & Computer ScienceUniversity of Groningen NetherlandsF.A.S.Alsaif@rug.nl Arockisamy ClementkingCollege of Computer Science King Khalid UniversitySaudi Arabiacksami@kku.edu.sa

|Date 07-02-2012 2

Overview› Scope

› Introduction

› Background of the Study

› Existing Quality Assessment Approaches

› Requirements of Benchmarking

› Problem Statement and Possible Solutions

› Existing Quality Frameworks Analysis

› Discussion on Existing Quality Frameworks

› Identification of Quality Indicators

› Conclusion and Future Work

|Date 07-02-2012 3

Identification of high priority quality indicators for smart

learning systems through the analysis of the existing learning

systems quality frameworks. Identification Analysis Determination Conversion and representation Conclusion

Scope

|Date 07-02-2012 4

Identification of exiting quality frameworks.

Analysis of the existing technology-based learning systems

for the identification of its major indicators.

Determination of indicators.

Conversion and representation of the indicators and its

models.

Conclusion of major and minor influencing learning system

factors.

Scope

|Date 07-02-2012 5

› The changing landscapes and its responsibility in all domains

› The challenge in the re-assessing of the methods and processes

utilized to assure quality and gear towards excellence or smart

models

› Standard processes for quality assurance as providing measures for

system-improvement

› Rapid growth of education programs

› Appreciation of smart system in the community

Background of the Study

|Date 07-02-2012 6

› Designing a model for quality smart system in the learning domain

› Identification of the measurable indicators required to form a smart

model

› Review of the literature and comparisons between existing

recommendations and practices to assist to view expected smart

model

Background of the Study

|Date 07-02-2012 7

› The outcome of this study Identify the indicators for different quality frameworks. Prove that quality score-carded model possesses most of the

relevant features for enabling decisions. Assess and develop a model for the overall quality of a smart

education system.

Background of the Study

|Date 07-02-2012 8

› The educational– learning system domain was selected as per

researcher previous work and interest.

› Based on the literature, numerous points of view are available for

assessing the quality of education.

› Recommendations and different approaches discovered in the

literature suggest guidelines to assess programs and their quality

components for educational system.

Identification of Existing frame works

|Date 07-02-2012 9

The quality process assessed through: Benchmarking Specification of standards

Benchmarking is the process of comparing the performance and

outcomes against what was achieved by selected other

programmes operating in a similar field and comparative

practices.

Identification of Existing frame works

|Date 07-02-2012 10

Much literature is reviewed and major four benchmarking are considered towards the analysis in addition to the 13 frame-works.

Year Author Research Work Observations2001 Buss Review of

benchmarking for higher education

A process that uses a permanent reference point against which levels can be compared and measured.

2004 SCIENTER SEEL bench-marking system starter pack

A process of identifying, learning, adapting, and measuring outstanding practices and processes from any organization/ public entity to improve performance.

2001 Jackson Benchmarking in UK higher-education: an overview

A process of self-evaluation and self-improvement, of improving ourselves by learning from others, and as a way to learn how to adapt and improve as conditions change.

2006 Higher education academy, UK

E-Learning benchmarking exercise

A process through which practices are analyzed to provide a standard measurement 'benchmark' of effective performance within an organization (e.g., a university).

Analysis

|Date 07-02-2012 11

Analysis

The reviewed frameworks to demonstrate the processes available to

specify and assess quality are as follows:

Framework Number Framework Name Publication Title Author/s (year)

1The 24 Benchmarks for Success in Internet-Based Distance Education

Quality on the Line Benchmarks for success in Internet-Based Distance Education

Institute for Higher Education Policy (IHEP) (2000)

2 ACTIONS Model of Quality Managing Technological Change: Strategies for College and University Leader Bates (2000)

3Best Practices for Electronically Offered Degree and Certificate Programs

Best Practices for Electronically Offered Degree and Certificate Programs

Western Cooperative for Educational Telecommuni-cations (WCET) (2001)

4 Eight Dimensions of E-learning Framework A Framework for Web- Based Learning Khan (2001)

5 Quality Standards in E-learning Quality Standards in E-Learning: A Matrix of Analysis Frydenberg (2002)

6 Five Pillars of Quality The Sloan Consortium Report to the Nation: Five Pillars of Quality Online Education

Sloan Consortium (Lorenzo & Moore, 2002)

|Date 07-02-2012 12

Analysis

Framework Number Framework Name Publication Title Author/s (year)

7 Quality- Assurance Strategies Using Quality Assurance Strategies for Online Programs Lee & Dziuban (2002)

8 Assessment Model An Assessment Model and Methods for Evaluating Distance Education Programs

Lockhart & Lacy (2002)

9 Accreditation and Assuring QualityAccreditation and Assuring Quality in Distance Learning

Council for Higher Education Accreditation (CHEA) (2002)

10 Concentric Support ModelThe Concentric Support Model: A Model for the Planning and Evaluation of Distance Learning Programs

Osika (2004)

11 Assessment Recommendations Distance Education: A System View Moore & Karsley (2005)

12 Six-Factor Solution Dimensions of Program Quality in Web-Based Adult Education

Haroff & Valentine (2006)

13Quality Scorecard for the Administration of Online Education Programs

A Quality Scorecard for the Administration of Online Education Programs a Delphi

Sheton (2010)

|Date 07-02-2012 13

In reality the process of benchmarking is not easy to apply in

most educational domain. Mckinnon et al. (2000)

University life learning and teaching is the most difficult area to

benchmark, since it is common at universities that the approach

to teaching and the courses are not standard.

Courses, even professional and specialized courses leading to

registration, are rarely directly comparable.

Analysis

|Date 07-02-2012 14

Benchmarking became an increasingly and widely-utilized

method to implement quality- assurance and promotion.

Benchmarking allows the choice of changes that help to improve

of quality the identification of application of areas for

improvement.

Moriarty (2011) illustrates this method as an example-driven

technological process that works within an organization with the

targets of purposely changing an existing state of affairs into an

improved state of affairs.

Analysis

|Date 07-02-2012 15

Moriarty and Smallman (2009) have further illustrated it as follows:

The position of benchmarking lies between the existing states of

affairs and the states of affairs sought after and participates to the

transformation process that achieves these enhancements.

The European Centre for Strategic Management of Universities

(ESMU) defines it as follows: Benchmarking is an internal

organizational process which seeks to enhance the performance of the

organization by learning about potential enhancements of its main

and/or backing processes through looking at these processes in other,

preferable-performing organizations.

Analysis

|Date 07-02-2012 16

Discussion on Analysis

› In the literature, the major issue in assessing the quality of any

information system is to determine the standards by which the quality

is defined and measured.

› Challenge is to transfer from traditional education systems to those

which include or are entirely based on e-learning approaches.

› Seddon and Yip (1992) proposed that a variety of measures of

effectiveness are required for miscellaneous systems.

|Date 07-02-2012 17

Discussion on Analysis

› In 2006, Levy argued that there were no comprehensive studies

exploring the actual effectiveness of e-learning systems.

› Barr and Tagg (1995) argued that whenever web technology is utilized

in educational environments, it is essential to think about its influence

on students, faculty members, courses and institutions.

|Date 07-02-2012 18

Selected Frameworks

S. No Framework Name Proposed by 1 The 24 Benchmarks for Success in

Internet-Based Distance EducationInstitute for Higher Education Policy (IHEP) (2000)

2 ACTIONS Model of Quality Bates (2000)

3 Best Practices for Electronically Offered Degree and Certificate Programs

Western Cooperative for Educational Telecommuni-cations (WCET) (2001)

4 Eight Dimensions of E-learning Frame-work Khan (2001)

5 Quality Standards in E-learning Frydenberg (2002)6 Five Pillars of Quality Sloan Consortium (Lorenzo & Moore, 2002)7 Quality Assurance Strategies Lee & Dziuban (2002)8 Assessment Model Lockhart & Lacy (2002)

9 Accreditation and Assuring Quality Council for Higher Education Accreditation (CHEA) (2002)

10 Concentric Support Model Osika (2004)11 Assessment Recommendations Moore & Karsley (2005)12 Six-Factor Solution Haroff & Valentine (2006)

13 Quality Scorecard for the Administration of Online Education Programs Sheton (2010)

|Date 07-02-2012 19

Determination of Indicators

S. No Quality Indicator

1 The Institutional Commitment, Support, and Leadership

2 Teaching and Learning

3 Evaluation and Assessment

4 The Course Development

5 Faculty Support

6 Student Support

7 Technology Support

8 Financial Considerations

9 The Course Structure

10 User Friendliness

11 Advising

12 Government and Regulatory Guidelines

|Date 07-02-2012 20

Conversion and Representation of Indicators

› There are many similarities between the thirteen frameworks and studies demonstrated in this review of quality assessment for online education programs.

S. No

Framework Number Quality Indicator

1 2 3 4 5 6 7 8 9 10 11 12 13

1 The Institutional Commitment, Support, and Leadership

√ √ √ √ √ x √ √ √ √ √ √ √

2 Teaching and Learning √ √ √ √ √ √ √ √ √ √ x X √3 Evaluation and Assessment √ x √ √ √ x √ √ √ √ √ √ √4 The Course Development √ √ √ x √ x √ √ √ √ √ √ √5 Faculty Support √ x √ √ x √ √ √ √ √ √ X √6 Student Support √ x √ x √ √ √ √ √ √ √ X √7 Technology Support x x x √ √ x x x x √ x √ √8 Financial Considerations x √ x x √ √ x x √ x x X x9 The Course Structure √ x x x x x x x x x x X √

10 User Friendliness x √ x √ x x x x x x x X x11 Advising x x x x x x x x x x x √ x12 Government and Regulatory

Guidelinesx x x x x x x x x x x √ x

|Date 07-02-2012 21

Main indicators to take in consideration: The support of institution The improvement of the course The procedure of teaching and learning The structure of the course The support of student The support of faculty and the assessment Evaluation in assuming the quality of online learning

Conversion and Representation of Indicators

|Date 07-02-2012 22

Conclusion

› The evaluation of quality education is more important.

› Programs keep on growing and students everywhere in the world

search for quality in their degree programs.

› Quality education truly matters as the eventual influence is to our

students.

› Higher education is in need of a new way to classify and evaluate

quality within education programs.

› Education online is inherently different from traditional education,

therefore it requires specific benchmarks and benchmarking processes.

|Date 07-02-2012 23

Conclusion

› Business and industry have made use of quality assurance processes

for numerous years to identify and quantify quality enhancement and

develop strategic planning and decision-making.

› Quality assessment processes are being utilized in higher education.

› After looking through the literature, it is clear that the Quality Score-

carded model succeeds to obtain all of the relevant features present in

formerly-suggested frameworks.

|Date 07-02-2012 24

Future Direction

In our next steps, we plan to develop a new framework based on the

strengths of all surveyed ones, and in particular, the Quality Score-

carded model.

The previous study indicates that the e-learning systems could be

measured using the existing frameworks. For future work, the sub-

indicators from these frameworks will be identified to create a

Smart Learning Model.

25|Date 07-02-2012