1|Date 07-02-2012
E-learning Quality Assurance Benchmarking in Higher Education
Fatimah Alsaif Johann Bernoulli Institute of Mathematics & Computer ScienceUniversity of Groningen [email protected] Arockisamy ClementkingCollege of Computer Science King Khalid UniversitySaudi [email protected]
|Date 07-02-2012 2
Overview› Scope
› Introduction
› Background of the Study
› Existing Quality Assessment Approaches
› Requirements of Benchmarking
› Problem Statement and Possible Solutions
› Existing Quality Frameworks Analysis
› Discussion on Existing Quality Frameworks
› Identification of Quality Indicators
› Conclusion and Future Work
|Date 07-02-2012 3
Identification of high priority quality indicators for smart
learning systems through the analysis of the existing learning
systems quality frameworks. Identification Analysis Determination Conversion and representation Conclusion
Scope
|Date 07-02-2012 4
Identification of exiting quality frameworks.
Analysis of the existing technology-based learning systems
for the identification of its major indicators.
Determination of indicators.
Conversion and representation of the indicators and its
models.
Conclusion of major and minor influencing learning system
factors.
Scope
|Date 07-02-2012 5
› The changing landscapes and its responsibility in all domains
› The challenge in the re-assessing of the methods and processes
utilized to assure quality and gear towards excellence or smart
models
› Standard processes for quality assurance as providing measures for
system-improvement
› Rapid growth of education programs
› Appreciation of smart system in the community
Background of the Study
|Date 07-02-2012 6
› Designing a model for quality smart system in the learning domain
› Identification of the measurable indicators required to form a smart
model
› Review of the literature and comparisons between existing
recommendations and practices to assist to view expected smart
model
Background of the Study
|Date 07-02-2012 7
› The outcome of this study Identify the indicators for different quality frameworks. Prove that quality score-carded model possesses most of the
relevant features for enabling decisions. Assess and develop a model for the overall quality of a smart
education system.
Background of the Study
|Date 07-02-2012 8
› The educational– learning system domain was selected as per
researcher previous work and interest.
› Based on the literature, numerous points of view are available for
assessing the quality of education.
› Recommendations and different approaches discovered in the
literature suggest guidelines to assess programs and their quality
components for educational system.
Identification of Existing frame works
|Date 07-02-2012 9
The quality process assessed through: Benchmarking Specification of standards
Benchmarking is the process of comparing the performance and
outcomes against what was achieved by selected other
programmes operating in a similar field and comparative
practices.
Identification of Existing frame works
|Date 07-02-2012 10
Much literature is reviewed and major four benchmarking are considered towards the analysis in addition to the 13 frame-works.
Year Author Research Work Observations2001 Buss Review of
benchmarking for higher education
A process that uses a permanent reference point against which levels can be compared and measured.
2004 SCIENTER SEEL bench-marking system starter pack
A process of identifying, learning, adapting, and measuring outstanding practices and processes from any organization/ public entity to improve performance.
2001 Jackson Benchmarking in UK higher-education: an overview
A process of self-evaluation and self-improvement, of improving ourselves by learning from others, and as a way to learn how to adapt and improve as conditions change.
2006 Higher education academy, UK
E-Learning benchmarking exercise
A process through which practices are analyzed to provide a standard measurement 'benchmark' of effective performance within an organization (e.g., a university).
Analysis
|Date 07-02-2012 11
Analysis
The reviewed frameworks to demonstrate the processes available to
specify and assess quality are as follows:
Framework Number Framework Name Publication Title Author/s (year)
1The 24 Benchmarks for Success in Internet-Based Distance Education
Quality on the Line Benchmarks for success in Internet-Based Distance Education
Institute for Higher Education Policy (IHEP) (2000)
2 ACTIONS Model of Quality Managing Technological Change: Strategies for College and University Leader Bates (2000)
3Best Practices for Electronically Offered Degree and Certificate Programs
Best Practices for Electronically Offered Degree and Certificate Programs
Western Cooperative for Educational Telecommuni-cations (WCET) (2001)
4 Eight Dimensions of E-learning Framework A Framework for Web- Based Learning Khan (2001)
5 Quality Standards in E-learning Quality Standards in E-Learning: A Matrix of Analysis Frydenberg (2002)
6 Five Pillars of Quality The Sloan Consortium Report to the Nation: Five Pillars of Quality Online Education
Sloan Consortium (Lorenzo & Moore, 2002)
|Date 07-02-2012 12
Analysis
Framework Number Framework Name Publication Title Author/s (year)
7 Quality- Assurance Strategies Using Quality Assurance Strategies for Online Programs Lee & Dziuban (2002)
8 Assessment Model An Assessment Model and Methods for Evaluating Distance Education Programs
Lockhart & Lacy (2002)
9 Accreditation and Assuring QualityAccreditation and Assuring Quality in Distance Learning
Council for Higher Education Accreditation (CHEA) (2002)
10 Concentric Support ModelThe Concentric Support Model: A Model for the Planning and Evaluation of Distance Learning Programs
Osika (2004)
11 Assessment Recommendations Distance Education: A System View Moore & Karsley (2005)
12 Six-Factor Solution Dimensions of Program Quality in Web-Based Adult Education
Haroff & Valentine (2006)
13Quality Scorecard for the Administration of Online Education Programs
A Quality Scorecard for the Administration of Online Education Programs a Delphi
Sheton (2010)
|Date 07-02-2012 13
In reality the process of benchmarking is not easy to apply in
most educational domain. Mckinnon et al. (2000)
University life learning and teaching is the most difficult area to
benchmark, since it is common at universities that the approach
to teaching and the courses are not standard.
Courses, even professional and specialized courses leading to
registration, are rarely directly comparable.
Analysis
|Date 07-02-2012 14
Benchmarking became an increasingly and widely-utilized
method to implement quality- assurance and promotion.
Benchmarking allows the choice of changes that help to improve
of quality the identification of application of areas for
improvement.
Moriarty (2011) illustrates this method as an example-driven
technological process that works within an organization with the
targets of purposely changing an existing state of affairs into an
improved state of affairs.
Analysis
|Date 07-02-2012 15
Moriarty and Smallman (2009) have further illustrated it as follows:
The position of benchmarking lies between the existing states of
affairs and the states of affairs sought after and participates to the
transformation process that achieves these enhancements.
The European Centre for Strategic Management of Universities
(ESMU) defines it as follows: Benchmarking is an internal
organizational process which seeks to enhance the performance of the
organization by learning about potential enhancements of its main
and/or backing processes through looking at these processes in other,
preferable-performing organizations.
Analysis
|Date 07-02-2012 16
Discussion on Analysis
› In the literature, the major issue in assessing the quality of any
information system is to determine the standards by which the quality
is defined and measured.
› Challenge is to transfer from traditional education systems to those
which include or are entirely based on e-learning approaches.
› Seddon and Yip (1992) proposed that a variety of measures of
effectiveness are required for miscellaneous systems.
|Date 07-02-2012 17
Discussion on Analysis
› In 2006, Levy argued that there were no comprehensive studies
exploring the actual effectiveness of e-learning systems.
› Barr and Tagg (1995) argued that whenever web technology is utilized
in educational environments, it is essential to think about its influence
on students, faculty members, courses and institutions.
|Date 07-02-2012 18
Selected Frameworks
S. No Framework Name Proposed by 1 The 24 Benchmarks for Success in
Internet-Based Distance EducationInstitute for Higher Education Policy (IHEP) (2000)
2 ACTIONS Model of Quality Bates (2000)
3 Best Practices for Electronically Offered Degree and Certificate Programs
Western Cooperative for Educational Telecommuni-cations (WCET) (2001)
4 Eight Dimensions of E-learning Frame-work Khan (2001)
5 Quality Standards in E-learning Frydenberg (2002)6 Five Pillars of Quality Sloan Consortium (Lorenzo & Moore, 2002)7 Quality Assurance Strategies Lee & Dziuban (2002)8 Assessment Model Lockhart & Lacy (2002)
9 Accreditation and Assuring Quality Council for Higher Education Accreditation (CHEA) (2002)
10 Concentric Support Model Osika (2004)11 Assessment Recommendations Moore & Karsley (2005)12 Six-Factor Solution Haroff & Valentine (2006)
13 Quality Scorecard for the Administration of Online Education Programs Sheton (2010)
|Date 07-02-2012 19
Determination of Indicators
S. No Quality Indicator
1 The Institutional Commitment, Support, and Leadership
2 Teaching and Learning
3 Evaluation and Assessment
4 The Course Development
5 Faculty Support
6 Student Support
7 Technology Support
8 Financial Considerations
9 The Course Structure
10 User Friendliness
11 Advising
12 Government and Regulatory Guidelines
|Date 07-02-2012 20
Conversion and Representation of Indicators
› There are many similarities between the thirteen frameworks and studies demonstrated in this review of quality assessment for online education programs.
S. No
Framework Number Quality Indicator
1 2 3 4 5 6 7 8 9 10 11 12 13
1 The Institutional Commitment, Support, and Leadership
√ √ √ √ √ x √ √ √ √ √ √ √
2 Teaching and Learning √ √ √ √ √ √ √ √ √ √ x X √3 Evaluation and Assessment √ x √ √ √ x √ √ √ √ √ √ √4 The Course Development √ √ √ x √ x √ √ √ √ √ √ √5 Faculty Support √ x √ √ x √ √ √ √ √ √ X √6 Student Support √ x √ x √ √ √ √ √ √ √ X √7 Technology Support x x x √ √ x x x x √ x √ √8 Financial Considerations x √ x x √ √ x x √ x x X x9 The Course Structure √ x x x x x x x x x x X √
10 User Friendliness x √ x √ x x x x x x x X x11 Advising x x x x x x x x x x x √ x12 Government and Regulatory
Guidelinesx x x x x x x x x x x √ x
|Date 07-02-2012 21
Main indicators to take in consideration: The support of institution The improvement of the course The procedure of teaching and learning The structure of the course The support of student The support of faculty and the assessment Evaluation in assuming the quality of online learning
Conversion and Representation of Indicators
|Date 07-02-2012 22
Conclusion
› The evaluation of quality education is more important.
› Programs keep on growing and students everywhere in the world
search for quality in their degree programs.
› Quality education truly matters as the eventual influence is to our
students.
› Higher education is in need of a new way to classify and evaluate
quality within education programs.
› Education online is inherently different from traditional education,
therefore it requires specific benchmarks and benchmarking processes.
|Date 07-02-2012 23
Conclusion
› Business and industry have made use of quality assurance processes
for numerous years to identify and quantify quality enhancement and
develop strategic planning and decision-making.
› Quality assessment processes are being utilized in higher education.
› After looking through the literature, it is clear that the Quality Score-
carded model succeeds to obtain all of the relevant features present in
formerly-suggested frameworks.
|Date 07-02-2012 24
Future Direction
In our next steps, we plan to develop a new framework based on the
strengths of all surveyed ones, and in particular, the Quality Score-
carded model.
The previous study indicates that the e-learning systems could be
measured using the existing frameworks. For future work, the sub-
indicators from these frameworks will be identified to create a
Smart Learning Model.
25|Date 07-02-2012