how effective is icdl training for omani teachers
Post on 16-Nov-2014
118 Views
Preview:
DESCRIPTION
TRANSCRIPT
HOW EFFECTIVE IS ICDL TRAINING FOR OMANI TEACHERS?
A dissertation submitted to the University of Manchester for the degree of Master of Arts in " Digital Technologies,
Communication and Education" in the Faculty of Humanities
YEAR OF SUBMISSION: 2009
FAHAD KHALIFA HUMAID AL HATMI
SCHOOL OF EDUCATION
UNIVERSITY OF MANCHESTER
List of ContentsList of Tables and Figures. 4 Abstract 5 Declaration 6 Copyright Statement 6 Acknowledgment 7 Chapter 1: Introduction 8 1.1. Background of the Topic 8
1.2. Statement of the Problem 10 1.3. Purpose of the Study 11
1.4. Significance 121.5. Research Questions 12 1.6. Theoretical Model 13 1.7. Scope and Limitations of the Study 15 1.8. Outline of the Study 16
Chapter 2: Literature Review 17 2.1. Introduction 172.2. ICT 17
2.2.1. Defining ICT Literacy 17 2.2.2. ICT in Teaching 19 2.2.3. Teacher Readiness for ICT Instruction 20 2.2.4. Approaches to ICT Education 21 2.2.5. Effectiveness of ICT Training 24
2.2.6. ICT Training Methods 25 2.2.7. Assessing ICT Literacy 26
2.3. ECDL/ICDL 272.3.1. Background 272.3.2. Standardization and Vendor-neutral Approach 272.3.3. Teaching Methods for ECDL/ICDL 282.3.4. Effectiveness of ECDL/ICDL 29
2.3.4.1. Effectiveness in Assessing ICT Literacy 292.3.4.2. Effectiveness for ICT Training 31
2.3.5. Gaps in Existing ECDL/ICDL Literature 332.4. ICT and ECDL/ICDL in Sultanate of Oman 33
2.4.1. ICT in the Sultanate of Oman 332.4.2. ECDL/ICDL in the Sultanate of Oman 34
2.5 Conclusion 35Chapter 3: Methodology 36
3.1. Introduction 363.2. Justification of the Methodology 363.3. Merging Quantitative and Qualitative Data 393.4. Quantitative Phase of the Study 39
3.4.1. Survey Questionnaire Methodology 403.4.2. Quantitative Sampling 403.4.3. Quantitative Instruments 41
2
3.4.4. Data Collection Procedures 423.4.5. Data Analysis 423.4.6. Reliability and Validity 44
3.5. Qualitative Phase of the Study 443.5.1. Sampling 453.5.2. Instruments 463.5.3. Data Collection Procedures 473.5.4. Data Analysis 483.5.5. Trustworthiness 49
3.6. Limitations of the Methodology 503.7. Summary 51
Chapter 4: Findings 524.1. Quantitative Findings 52
4.1.1. Survey Questionnaire 524.1.2Pre- and Post- Test of Access Class under Observation 59
4.2. Qualitative Findings 604.2.1. Descriptive Statistics: Motivation, Expectations, 60
and Skills Gained4.2.2. Descriptive Statistics: Deficiencies in the Training 624.2.3. Descriptive Statistics: Suitability of Training for 62 Teachers, And Recommended Improvements4.2.4. Observation of Access Class 634.2.5. Interviews with Trainers 674.2.6. Interviews with Students of Access Class 68
Chapter 5: Discussion 71Chapter 6: Conclusion 78Chapter 7: Recommendations 81
7.1. Recommendations for Improvement 817.1.1. Make Training Relevant to Educators 817.1.2. Training Length 817.1.3. Training Time 827.1.4. Training Materials 827.1.5. Trainer’s Training 827.1.6. Phased Approach to ICT Training 82
7.2. Recommendations for Future Research 837.2.1. ICT Competency Differences for Teachers in 83 Different Subject Areas7.2.2. The Effect of Lack of and Poorly Maintained 83 Equipment
Chapter 8: References 85Appendix A: Survey Questionnaire 92Appendix B: Structured Observation Form 97Appendix C: Teacher Interviews 100Appendix D: Trainer Interviews 101Appendix E. Participant Affirmation Letter 102
3
List of Tables
Table 1. Research Overview 38Table 2. Quantitative and Qualitative Processes. 38Table 3. Mean and Standard Deviation, Part 2 of the Survey 53 QuestionnaireTable 4 Means and F statistic for Gender 56Table 5 Statistically Significant Differences in Responses in Subject 57 Taught Data Grouping Table 6 Means of the Subject Taught Groups with Statistically 58 Significant DifferencesTable 7 Skills Expected to Improve form ICDL Training as Ranked1 60 by RespondentsTable 8 Skill gained in ICDL Training most Likely to Transfer 61 to the Classroom SettingsTable 9. Pre-Test Post-Test Scores 64
Word Count: 19, 535
4
Abstract
The increasing importance of information and communication technology
(ICT) in business and research challenges the education system to produce graduates
with strong ICT skills (Randall & Zirkle, 2005). Instruction in ICT is particularly
significant for developing nations to ensure that the workforce can compete in the
global environment. One approach to ensure that teachers have the necessary
fundamental skills to support ICT instruction in primary and secondary schools is to
require International Computer Driving License (ICDL) training and certification.
This approach, however, does not appear to provide the full range of ICT training
required to help teachers understand how to use ICT in the classroom to improve
learning outcomes. This concurrent mixed methods study examines the effectiveness
of ICDL training among Omani teachers. Results suggest that while ICDL is effective
at teaching basic skills in computer and application use, the training is not effective in
training educators on critical assessment of technology or in how to use technology in
the classroom. Based on the results of survey questionnaires, interviews, and
structured and unstructured observations, recommendations are made for how ICDL
training might be improved and for how ICT training in general can better meet the
needs of the Omani teachers.
5
DECLARATION I declare that no portion of the work referred to in the dissertation has been submitted in support of an application for another degree or qualification of this or any other university or other institute of learning.
Signed ……………………………………………………………………………………… (Candidate)
Date ………………………………………………………………………………………
COPYRIGHT STATEMENT
1. Copyright in text of this dissertation rests with the author. Copies (by any process) either in full, or of extracts, may be made only in accordance with instructions given by the author. Details may be obtained from the appropriate Graduate Office. This page must form part of any such copies made. Further copies (by any process) of copies made in accordance with such instructions may not be made without the permission (in writing) of the author.
2. The ownership of any intellectual property rights which may be described in this dissertation is vested in the University of Manchester, subject to any prior agreement to the contrary, and may not be made available for use by third parties without the written permission of the University, which will prescribe the terms and conditions of any such agreement.
3. Further information on the conditions under which disclosures and exploitation may take place is available from the Head of the School of Education.
Signed ……………………………………………………………………………………… (Candidate)
Date ………………………………………………………………………………………
6
Acknowledgment
I offer my most sincere gratitude to all the individuals who supported me in
bringing this study to a successful conclusion. First, I wish to acknowledge the
immense help of Dr. Andrew Whitworth whose supervision was always there when
needed. Second, the success of this study would not have been possible without my
parents’ prayers which gave me spiritual support. Finally, my thanks are extended to
those who helped me in collecting my data.
7
Chapter 1: Introduction
1.1. Background of the Topic
The increasing importance of information and communication technology
(ICT) in business and research challenges the education system to produce graduates
with strong ICT skills (Randall & Zirkle, 2005). Instruction in ICT is particularly
significant for developing nations to ensure that the workforce can compete in the
global environment. The education system in many nations, however, has been slow
to incorporate ICT training into the classroom curriculum (Mioduser, et al., 2003).
Several factors account for insufficient ICT educational opportunities. Many
classrooms are without sufficient physical resources to support ICT instruction.
Teachers may not have sufficient knowledge of ICT if they have not received training
(Gordon, 2007). Teachers may have sufficient ICT training and skills, but may not
have the knowledge necessary to incorporate ICT into classroom teaching at the
primary and secondary level (Law & Sim, 2008). Teachers must also have sufficient
confidence in their ICT skills to use them in the classroom environment, which may
require repeated training or sufficient time to practise skills outside the classroom
(Gordon, 2007). When teachers receive in-service training in a subjects related to
ICT, they assume the role of the student, and their attitudes as learners influence the
effectiveness of the training. (Preston, Danby, & Wegrif, 2005).
One approach to ensure that teachers have the necessary fundamental skills to
support ICT instruction in primary and secondary schools is to require International
Computer Driving License (ICDL) training and certification. The ICDL, which is
known as the European Computer Driving License (ECDL) within the European
8
Union, is a certification attesting to basic proficiency in the use of certain types of
software or computer systems (Csapo, 2002). The ICDL/ECDL is the world’s largest
computer certification programme, with more than 9 million candidates for
certification in 148 countries (EDDL Foundation, 2009). The ICDL/ECDL
Foundations responsible for oversight of the programme recommend that individuals
take specific training prior to certification testing. Developed by private organizations
and delivered through courseware that is often combined with tutorial services, the
training consists of seven modules. These modules include: 1) basic concepts of
information technology; 2) using the computer and managing files; 3) word
processing; 4) spreadsheets; 5) database; 6) presentation; and 7) information and
communication. While the courseware conforms to the ICDL/ECDL specifications
and is generally considered standardised, there remains some variability in the content
and in the manner of presentation of the material. In some learning environments, the
courseware is presented and a tutor is available, while other learning environments
provide students only with the courseware (Calzarossa, et al., 2005). In addition, the
ICDL/ECDL instructional modules are intended to be vendor neutral, offering general
information about ICT that can be used with different types of software (Randall &
Zirkle, 2005).
In the Sultanate of Oman, the Ministry of Education urges all teachers to
receive ICDL training and to apply for certification regardless of the subject they
teach (Ministry of Education, n.d.). The Ministry of Education recognises the need to
ensure teachers have fundamental ICT skills in order to facilitate integrating ICT into
classroom teaching. The ministry of Education also considers the ICDL training an
9
effective in-service training approach that allows teachers to obtain fundamental ICT
skills. The ICDL programme in Oman is considered a form of in-service training for
teachers. The ICDL programme in Oman is administered by the ICDL GCC
Foundation, which has been appointed by the ICDL Foundation as the sole operator
of the ICDL in the Gulf Cooperation Council nations, which include the Sultanate of
Oman. The ICDL GCC Foundation operates four training and testing centres in the
Sultanate.
1.2. Statement of the Problem
The specific problem investigated in this study is that Omani teachers do not
effectively use ICT skills in the classroom to enhance and improve student learning.
A prerequisite for using ICT in classroom teaching is sufficient proficiency with the
technical skills related to the operation of computers and software applications
(Rottman, 2004). In addition, teachers must have sufficient confidence in their ICT
skills and an understanding of appropriate methods to integrate ICT skills into
classroom teaching (Jung, 2005). Previous research has not fully determined that the
ICDL programme is effective for teaching the basic skills necessary for operating the
specific computer systems and software applications necessary to obtain certification
(Perez & Murray, 2007). The outcome of the ICDL instruction programme may
depend on factors such as design of the training modules, the specific methods used
to provide instruction, and the previous ICT knowledge of the individual (Dillon &
Tearle, 1999). Previous research is also inconclusive in findings concerning the
effectiveness of the ICDL programme for teaching the higher order or critical
thinking skills necessary to use ICT in applied academic or business situations
10
(Whitworth, 2005). This previous research suggests that the knowledge of
fundamental ICT skills differs from the knowledge necessary to apply the skills in
professional applications. The specific ICT needs of the education profession,
requires both fundamental ICT operating skills and the ability to integrate the skills
into practical classroom applications. .Previous research has not examined generic
ICDL training courseware effectiveness and effectiveness of teaching programmes
for meeting these needs. Because of the inconclusive findings of previous researchers,
uncertainty exists about the effectiveness of ICDL training for teaching fundamental
ICT skills to teachers necessary for supporting their ability to use ICT in classroom
teaching.
1.3. Purpose of the Study
The purpose of this concurrent mixed methods study was to examine the
effectiveness of ICDL training among Omani teachers. The quantitative phase of the
study examined the relationship between the independent variable of ICDL training
and the dependent variables related to improved ICT skills and the attitude toward the
use of ICT for classroom teaching, Data was collected from a population of Omani
teachers that have completed the ICDL in-service training. The qualitative phase of
the study examined the effectiveness of ICDL training by collecting data using a pre-
and-post test, interviews, and observations. The population for the qualitative phase
was Omani teachers and trainers in a single classroom teaching the ICDL database
module using Access. For the quantitative phase of the study, statistical data is
provided from a larger sampling of teachers. The quantitative data was combined
11
with the qualitative data to assess the effectiveness of ICDL training among Omani
teachers.
1.4. Significance of the Study
The findings of the study may have significance for teachers taking the ICDL
course as in-service training programme by identifying the effectiveness of the
training for meeting individual ICT objectives. The findings of the study may also
have significance for the education profession by identifying the extent that ICDL
training produces the desired outcomes of improvement in basic ICT skills and the
critical thinking skills necessary to use ICT in practical educational situations. As a
result, the findings may also have direct significance for the Ministry of Education in
Oman by assessing the effectiveness of the ICDL programme in the Sultanate. The
findings may also have importance for the organisations involved with developing
courseware and providing training by identifying the effectiveness of current ICDL
training for meeting the needs of the education profession.
1.5. Research Questions
There is a primary research question and four sub-questions that the research
attempts to answer.
Primary Question: Does ICDL provide the best approach for Oman with respect to
training its teachers to use technology effectively? The research question examines
whether the ICDL training is effective for providing teachers with the ICT skills
necessary in the academic and educational environment (Young, 2004). For the
purposes of the study, effectiveness is defined as a positive change in ICT proficiency
and skills in the specific subject matter areas addressed by the ICDL modules as well
12
as a positive change in the critical thinking skills necessary to apply the knowledge in
a professional teaching environment.
The sub-questions were derived from the main research question. They
investigate aspects of the ICDL training programme necessary to answer the main
research question of the study. The sub-questions are as follows:
1. Which ICT competencies were improved and which were not improved
through the ICDL training?
2. Does ICDL training influence the attitudes of teachers towards incorporating
ICT into classroom teaching methods?
3. Are there differences in ICT competencies gained through ICDL training that
are based on the mediating factors of gender, number of years teaching, and
class being taught?
4. Are there other approaches that could fill the gap in ICDL training to provide
the full range of ICT competencies required for Omani teachers to be more
effective in their classrooms?
1.6. Theoretical Model
The theoretical framework for the research is based on the Kirkpatrick
evaluation model developed to assess ICT training among teachers as described by
Wu, et al. (2004). This model defines four levels of outcome evaluation: 1)
evaluation-reaction, 2) evaluation-learning, 3) evaluation-behaviour, and 4)
evaluation-results. The first level examines participant reaction to the training using a
closed-end questionnaire. The second level measures what content was learned from
the training and pre and post testing is one method of taking this measurement. The
13
goal of level three measurement is to determine if the training results in behaviour
changes and looks at whether the knowledge gained is transferred to the workplace.
Kirkpatrick suggests interviews as an effective means of making this measurement.
Level four examines the bottom line results of the training (e.g., how did the training
affect ROI, learning outcomes, or any other bottom-line desired result). Level 4 is
difficult to measure because there are many other factors influencing the bottom line
result.
From the perspective of Kirkpatrick’s model as described by Wu, et al.
(2004), ICT training is effective if it provides both the basic skills necessary to use
ICT systems and the critical thinking skills necessary to apply the knowledge to
practical or academic situations. An underlying assumption in the evaluation model is
that ICT training not only should include technical skills necessary to operate
software and hardware, but also knowledge about the appropriate uses of the ICT
system to obtain and manipulate information or to apply ICT in professional
situations. The evaluation model is concerned with technical proficiency, which is a
necessary pre-requisite for the higher order skills necessary to obtain and manipulate
information. It also examines other factors such as the effect of the learning on
changing attitudes towards ICT and the willingness and ability of the learner to apply
ICT in a wide range of professional or academic tasks. In the context of this study, the
model examines the extent that the ICDL standardised training provides teachers with
fundamental ICT skills. It also evaluates the effect of the training on the attitude of
teachers towards the use of ICT in general and the use of ICT in classroom
instruction. Based on the theoretical model, the ICT or ICDL training would not be
14
considered effective unless it meets the criteria of providing both basic ICT skills as
well as critical thinking skills necessary to apply the knowledge in a professional
teaching environment.
1.7. Scope and Limitations of the Study
The scope of the study is restricted to an investigation of ICDL training
among Omani teachers. The scope of the quantitative phase of the study is limited to
Omani teachers that have completed ICDL training. The scope of the qualitative
phase of the study is limited to Omani teachers in the process of taking the ICDL
training course in the database module using Microsoft Access. As a result, the study
examines only the ICDL training courses as provided by the ICDL GCC Foundation
in the training centres in the Sultanate of Oman. The scope of the study is further
limited to an investigation of the ICDL training programme and does not include
direct assessment or observation of the way in which teachers completing the
programme use or apply the knowledge in classroom teaching. The limitations on the
scope of the study are necessary to reduce the effect of confounding variables that
could influence the findings from the use of courseware and teaching methods offered
by different training organisations.
The scope of the study creates a limitation for the study in the ability to
generalise the findings concerning the effectiveness of ICDL training beyond the
population of Omani teachers. While the quantitative component increases the ability
to generalise the findings, the data is based on the use of a specific type of courseware
and instructional approach used by the ICDL GCC Foundation. Another limitation of
the study is the possibility of researcher bias influencing the findings. In the
15
qualitative phase of the study, the researcher interacts with the data collection process
and the weight placed on the data during analysis. The use of the mixed method
approach with a quantitative component, however, reduces the influence of researcher
bias on the findings.
1.8. Outline of the Study
The second chapter of the study contains a review of literature related to the
topics of ICT and ICDL training and effectiveness, and issues related to teacher ICT
readiness. Following the literature review is a methodology chapter, which details the
specific research design and procedures used in the quantitative and qualitative
phases of the study. The fourth chapter presents the findings from the statistical
analysis of the quantitative data and the analysis of the qualitative data, and includes a
discussion of the findings. It combines the data to form a case study of the
effectiveness of ICDL training among Omani teachers. The final chapters contain a
summary of the research, conclusions linking the findings to the literature, and
recommendations for future research.
16
Chapter 2: Literature Review
2.1. Introduction
ICT in education and the effectiveness of EDCL/ICDL are examined in the
literature through both quantitative and qualitative studies. The literature generally
assumes the theoretical perspective that instructors must have both fundamental and
critical thinking competencies with ICT to incorporate technology into classroom
instruction. The literature review is divided into sections consisting of: a) ICT skills
education, b) ECDL/ICD education, and c) ECDL/ICDL effectiveness. Literature
focusing on ICT and the ICDL programme in the Sultanate of Oman is also included.
2.2. ICT
2.2.1. Defining ICT Literacy
No standard] definition of ICT literacy exists with institutions establishing
local objectives for ICT (Kennewell, 2002). Gordon (2007) noted that ICT represent
core skills and transferable skills necessary to use information technology. The skills
are core because they require knowledge about the fundamental processes necessary
to input, manipulate, and extract information from an information technology system.
The skills are transferable because they allow an individual to use the core skills on
any type of information technology system. ICT in education plays a critical role in
developing human capital and in increasing the standard of living (ECDL/ICDL,
2008). Kennewell (2002) suggests that ICT competency can be defined in terms of
specific skills such as the ability to recognise when using an ICT system is required,
the ability to use the functions of an ICT system, and the ability to understand the full
range of applications available to solve specific problems. Both Kennewell’s and
17
Gordon’s definitions suggest that ICT literacy encompasses both computer literacy
and information literacy. Computer literacy is defined as an understanding of the
basic function of a computer and understanding the use of some software applications
Information literacy is an ability to know when information is needed as well as the
ability to locate, analyze, and effectively apply the information to a specific
circumstance (Easton, Easton & Addo, 2006). Rockman (2004) indicated that ICT
literacy involves the ability to use technology to collect and manipulate information,
which is a specialised application of information literacy theory. Whitworth (2009),
however, suggests that people may not be “mentally or cognitively free” (p. 304) to
define their information needs and might not understand the conflicts between what
information they need and what limitations exist in getting the needed information.
Whitworth also introduces media literacy and states that a critical understanding of
the medium might be what actually defines literacy, not just the ability to use and
operate the medium. Garnett (2008) defines ICT literacy a little differently. He offers
that ICT literacy is the ability to use a computer to access resources on the World
Wide Web and to use email as a communication tool. Garnett lists six literacies that
he feels are interrelated: 1) computer literacy, 2) ICT literacy, 3) information literacy,
4) system literacy, 5) e-learning literacy, and 6) e-government literacy. ICT literacy
includes not only having basic computer skills, but also having the ability to
recognise the type of information necessary to solve a problem, having the knowledge
about methods to access the information, understanding limits to accessing the
information, and having the capacity to apply critical analysis to the information.
Garnett’s six literacies suggest that the definition of ICT literacy might be further
18
expanded to include system literacy and Internet/Web literacy, and Whitworth might
add media literacy. Clearly, the definition of ICT literacy is still being developed and
the lack of a standard definition contributes to the difficulty in establishing objectives
for ICT education.
2.2.2. ICT in Teaching
A review of research examining the use of ICT in teaching identified two
categories of barriers to the use of ICT in the classroom consisting of teacher
knowledge and behavioural issues and school-related issues (Becta, 2004). Barriers
related to teachers include insufficient knowledge and confidence to use ICT in the
classroom, the belief that ICT does not enhance learning, and difficulties with
classroom management when using computers. Barriers related to the school include
insufficient or obsolete ICT equipment, lack of technical or administrative support,
and the failure to differentiate training for teachers with varying ICT skills. An
investigation of the factors influencing the use of ICT in classrooms among primary
teachers in five European nations determined that the teacher-related factors were
more significant than school-related or structural factors. If teachers did not believe
they had sufficient ICT knowledge or skills, they were reluctant to incorporate ICT
technology into classroom teaching methods. A qualitative study by Zakopoulos
(2005) also identified similar behavioural and structural barriers to incorporating ICT
into classroom teaching among primary teachers in the United Kingdom. The
behavioural barriers included lack of time to prepare computer activities related to the
curriculum and insufficient training and knowledge in ICT. The structural barriers
19
included insufficient technical support and restrictions contained in the curriculum
related to the type of learning activities used in the classroom.
Lau and Sim’s (2008) investigation of the factors influencing adoption of ICT
by secondary school teachers in Malaysia found that prior training and knowledge of
ICT was critical for the development of a positive attitude towards ICT and increased
use in the classroom. While most teachers participating in the study had a generally
positive attitude towards ICT, they often did not incorporate ICT instruction in their
teaching. Significant reasons for not incorporating ICT into teaching were limited
knowledge of methods to integrate ICT into teaching and limited knowledge about
how to make full use of ICT. These findings suggest that ICT training for teachers
should extend beyond instruction in fundamental technical skills and include
instruction in the use of ICT in the classroom. An additional finding of the study was
that lack of technical support was a reason for teachers not using ICT in the
classroom, because teachers lacked the competencies to make technical adjustments
or repairs to IT equipment. This finding is similar to a structural barrier to ICT
training of insufficient technical support identified by Becta (2004). The findings of
Smeets and Mooij (2001) also indicated that teacher confidence in their knowledge
and use of ICT influences the way in which ICT is integrated into classroom teaching.
2.2.3. Teacher Readiness for ICT Instruction
Teacher readiness for ICT instruction is a critical issue for ICT literacy, with
numerous studies indicating that primary and secondary teachers often lack sufficient
competencies to incorporate ICT instruction in the classroom (Law & Sim, 2008).
Gordon (2007) indicated that official programmes to encourage teachers to
20
incorporate ICT into teaching had little effect unless the teachers had the necessary
skills and incentives to use ICT. Teachers differ from other ICT users because they
are expected to maintain professional status with students and colleagues and must
learn new computer skills and often use the skills before they have had adequate time
to consolidate the learning (Preston, Danby, & Wegrif, 2005). Teachers should have
sufficient competencies in the technical skills to focus on using ICT for more
advanced academic applications rather than focusing on fundamental software
operations. In addition, ICT competencies for teachers should extend well beyond the
fundamental skills necessary to use IT systems or software applications to include
sufficient information literacy to instruct students in critical evaluation of information
sources and uses of information to solve problems (Rockman, 2004). The readiness of
teachers to teach ICT is an issue separate from structural factors such as the
sufficiency of computers in the classroom to support ICT instruction (Zakopoulos,
2005).
2.2.4. Approaches to ICT Education
Reffell and Whitworth (2002) noted that the approach to ICT teaching in
universities is to focus on the skills necessary to operate the equipment rather than the
skills necessary to evaluate the effectiveness and relevance of ICT outputs. In
addition, the skills necessary for performing different ICT tasks are highly varied and
do not involve understanding the methods of providing inputs to automated software
programmes. In general, ICT education does not focus on providing students with the
full range of skills necessary to perform ICT functions. The lack of integration in ICT
21
education results in gaps in student knowledge. Student often develop strong skills in
some areas of ICT knowledge while lacking skills in other areas.
A subsequent assessment by Whitworth (2005) indicates that universities in
the United Kingdom traditionally approach ICT instruction as a service subject where
the expectation is that students will attain proficiency in operational skills necessary
to use the technology. In the context of higher education, service subjects are
accorded low status. Although ICT education is critical for success in both academic
and post-academic endeavours, it nonetheless suffers from low prestige as a service
subject. Expertise among teachers varies for instructing students in ICT. This leads to
poor outcomes such as students rejecting the use of technology in the classroom. The
failure to provide appropriate ICT training does not permit university students to
build on their ICT knowledge and to transfer the knowledge to uses in different
academic courses (Rockman, 2004).
Tanner (2002) indicated that ICT instruction in schools at the secondary level
should have multiple subjective and objective goals. The subjective goals include
instilling confidence in ICT users and a sufficient sense of self-efficacy to allow
students to explore ICT without tutorial guidance. The objective goals include the
mastery of the basic skills necessary to use ICT systems, which are foundations for
the subsequent development of higher order skills necessary for full ICT capabilities.
Tanner (2002: 7) further defines higher order ICT skills as recognizing when the use
of ICT might be appropriate, planning how to approach a problem using ICT, making
and testing hypotheses, monitoring progress in a task and evaluating the result,
reflecting on the effect of using ICT in a particular situation. The higher order skills
22
are necessary to use ICT effectively for teaching academic subjects and for
maximising the value of the ICT systems in business applications. Rockman (2004)
suggested that the objective of ICT instruction in secondary schools should be to
provide students with sufficient information literacy in addition to the fundamental
skills necessary to use ICT systems. Kennewell (2002) indicated that the specific
curriculum used in the secondary schools for ICT education should incorporate
teaching to achieve specific objectives such as recognition of the right time to use the
ICT system to solve a problem. Although Tanner’s (2002), Kennewell’s (2002) and
Rockman’s (2004) discussion of ICT in the secondary school identifies the general
theoretical principles underlying the development of curriculum, it does not provide
specific information to teachers about instructional content and the type of
fundamental tasks students must learn to support subsequent development of higher
order skills.
Jung (2005) developed a theoretical model of teacher training for ICT
intended to address the issues of skills knowledge, confidence, and understanding of
the methods to use ICT in the classroom. The model has two learning continuums.
The first continuum is bounded at one extreme by learning how to use ICT, which
progresses to the other boundary of learning by using ICT. Learning how to use ICT
involves the fundamental skills necessary to obtain and extract information from ICT
systems. Learning by using ICT involves using ICT to solve problems and to acquire
knowledge useful in general academic endeavours. The second continuum is bounded
at one extreme by core technologies and at the other extreme by complementary
technologies. At the core technology extreme, the teacher must obtain skills with
23
basic ICT hardware and software. The eventual progression is towards the ability to
engage in a learning curve necessary to use all types of ICT. While this framework is
an ideal learning pattern for teachers responsible for integrating ICT into the
classroom, the theory has not been tested in practical situations involving instruction
for teachers.
2.2.5. Effectiveness of ICT Training
The effectiveness of ICT training depends on variables such as the methods
used in instruction, the content of the instructional materials, and the attitudes of the
students towards technology (Preston, Danby, & Wegrif, 2005). Ideally, ICT training
should take different types of learning styles into consideration and compensate for
personal characteristics such as low self-efficacy with the use of computers and
technology systems. Poulter and McMenemy (2003) conducted a survey to evaluate
the effectiveness of advanced ICT training modules at the university level. The
findings of the survey indicated that ICT training is most effective when students
have basic computer skills and can focus on learning higher order skills such as
troubleshooting. The findings also indicated that the ability of tutors to communicate
with students in the learning process was an important factor for positive learning
outcomes.
In qualitative research examining the effectiveness of ICT training using CD-
ROMs with a modular learning approach to provide instruction to users outside the
educational environment, Dillon and Tearle (1999) found that outcomes depended on
the content of the training modules, the methods of presentation, and the degree of
prior computer knowledge of the learner. Maximum effectiveness in terms of
24
knowledge acquisition occurred when the content of the training was related to a
specific objective the learner perceived as useful, such as a task to which the
knowledge could be applied. This finding conforms to the general theories of adult
learning in which the learner is motivated to obtain knowledge for specific purposes.
The findings also indicated that the learners had variable motivations depending on
the content of the modules. These findings imply that the most effective ICT training
provides motivation to learners by demonstrating the utility of the ICT knowledge
rather than treating the learning solely as an academic exercise.
Stanley and Tanner (2002) recommended a series of assessments to
determine the effectiveness of ICT training. An initial baseline assessment determines
the existing skill levels and knowledge of students. Teachers should also make
continuous formative assessments to determine whether the approach for learning is
having the desired effect of imparting the desired level of knowledge to students.
Teachers should also assess the outcome the ICT training in terms of both functional
skills and the ability to apply the knowledge in other academic areas. This structured
approach emphasises assessment theory, but does not provide information about the
specific tools that should be used to perform the assessment. As a result, considerable
variability could exist in the methods used to assess outcomes and the effectiveness of
2.2.6. ICT training methods.
According to Smeets and Mooij (2001), ICT instruction in secondary schools
is most effective when the teacher adopts a pupil-centered approach in which the
curriculum and rate of learning is designed to meet the specific needs of the student.
In theory, ICT instruction allows curriculum design to have sufficient variation to
25
meet the needs of students with different learning styles by creating a multimedia and
multisensory learning environment. In addition, ICT approaches allow students to
progress at their own rate, which may be slower or swifter than the median. A survey
conducted by Smeets and Mooij (2001), however, found that teachers most often use
ICT as an adjunct to traditional teaching methods. As a result, the ICT learning tool is
linked to an existing teaching approach rather than designing the curriculum around
the ICT capabilities. The findings of Costa and Peralta (2006), however, determined
that an additional factor influencing the effective use of ICT in primary schools was
the orientation of the educational philosophy of the teacher. ICT use in the classroom
was more common and perceived as more effective when the teacher had a
constructivist rather than a traditional pedagogic orientation toward teaching.
Teachers using a constructivist paradigm felt more comfortable incorporating
emerging technologies into classroom teaching because of the perception that they
made learning more relevant from the perspective of the students.
2.2.7. Assessing ICT Literacy
Some evidence exists indicating that the assessment of ICT literacy does not
depend on the type of instrument. Robbins and Zhou (2007) used regression analysis
to compare outcomes from administering the TAIT assessment and the CSP, which is
a component of the ICDL. The study population was university students in a single
institution that had taken a basic computer skills course modelled on ICDL
curriculum. The findings showed no statistically significant differences in outcomes
using the two assessment tools. The research, however, did not control for
demographic variables such as traditional or non-traditional students, age, and
26
previous computer experience. In addition, instruments measuring the perceptions of
students and employers on their ability to use ICT to solve practical or education-
related problems (Dixie & Wesson, 2001) can assess outcomes in ICT education.
2.3. ECDL/ICDL
2.3.1. Background
The ECDL Foundation (2005) provides certification for courseware
developers to ensure that the contents of the training for ECDL are standardised and
prepare students for the certification process. The ECDL Foundation recommends
courseware developers map the required skills to the training components, ensuring
that students are exposed to multiple methods to accomplish the required tasks. The
use of ECDL approved courseware ensures that students learn the tasks necessary for
certification.
2.3.2. Standardization and Vendor-neutral approach
Most of the literature discussing the ECDL/ICDL system adopts a qualitative
and non-empirical approach describing the theoretical benefits of standardization of
content and certification to demonstrate basic computer literacy for employers. Csapo
(2002) indicated that the standardisation of the core curriculum for the ECDL/ICDL
creates a vendor-neutral approach to content prompting software developers to create
course material based on the ECDL/ICDL standards. The vendor-neutral approach
provides a range of basic or introductory computer skills not specific to any
organization or technology platform (Randall & Zirkle, 2005). The ECDL/ICDL
content may also meet the objectives for teaching the basic skills associated with ICT
in secondary schools. According to Kennewell (2002), the objective of ICT in
27
secondary school should be to provide the knowledge and skills necessary to use
common computer tools found in business and learning other subjects such as word
processing and spreadsheets. The literature implies that the primary benefit from
ECDL/ICDL is the standardization of basic computer literacy teaching objectives
(Randall & Zirkle, 2005).
2.3.3. Teaching Methods for ECDL/ICDL
Research examining the use of ECDL/ICD has determined that it forms the
core curriculum for developing ICT literacy in many secondary schools and
universities. Calzarossa, et al., (2005) used a survey questionnaire to collect data
about the methods used by 46 Italian universities to provide instruction to students in
programs leading to ECDL certification. The findings indicated that 40 universities
used classroom teaching or blended classroom and self-learning as the primary mode
of instruction, with courses offered in all seven of the ECDL modules at 28 of the
institutions. The remaining six universities used the self-learning instructional
approach, with a tutor assigned to each student to provide assistance as necessary.
The research identified practices among universities and did not examine the
effectiveness of different approaches based on ECDL certification outcomes.
In Ireland, the universities offer a mixed teaching approach in which students
can receive training by combining self-instruction courseware with occasional tutor
support or can opt for full self-instruction through automated courseware (Fallon,
n.d.). The variability in the approaches to ICT training at the university level
corresponds to the findings of Kiridis, Drossos, and Tsakiridou (2006) who found
substantial differences in the approaches used for ICT education in Greek secondary
28
schools. The variability was because of the ambivalence of the faculty towards the
need to provide students with formal computer literacy training. These studies,
however, did not evaluate cultural factors in ICT and ECDL training. Using
ECLD/ICDL as the core curriculum for ICT education does not meet the full range of
objectives for ICT education as identified by Tanner (2002) and Reffell and
Whitworth (2002) because it focuses only on basic skills to operate software
programmes.
2.3.4. Effectiveness of ECDL/ICDL
2.3.4.1. Effectiveness in Assessing ICT Literacy
Some of the qualitative literature is critical of the ECDL/ICDL approach to
assessing computer literacy. Young (2004) suggested that the ECDL/ICDL assess
only the knowledge and ability to use specific types of software such as word
processing or spreadsheet programs. It does not assess critical thinking skills or the
ability to use the software to solve research problems. As a result, some organizations
such as the Educational Testing Services in the United States have developed
alternative computer literacy assessments. Whitworth (2005) also noted that the
ECDL focuses primarily on skills valued by business and does not provide students
with ICT skills necessary for the academic environment. At the same time, the
standardised assessment of fundamental ICT skills provided by ECDL overcomes the
problems of developing appropriate measurements to assess outcomes of ICT
instruction (Stanley & Tanner, 2002). These somewhat negative opinions regarding
the effectiveness of the ECDL/ICDL approach to computer literacy may be because
the approach focuses primarily on fundamental skills, which are in the early part of
29
the Jung (2005) learning continuum that involves learning how to use ICT.
Individuals with the fundamental skills may not be sufficiently challenged by the
ECDL/ICDL skills development modules.
Perez and Murray (2006) adopted the position that the ECDL/ICDL
certification should be viewed as an assessment tool to determine if individuals have
the basic skills to operate rudimentary computer programs, but not as an assessment
of computer literacy. From this perspective, computer literacy includes theoretical
knowledge about why processes take place as well as analytic skills to manipulate
processes to achieve specific objectives. The ECDL/ICDL certification process also
addresses only one level of learning in both the four-level learning assessment model
described by Wu, et al. (2004) and the three-level learning assessment model
proposed by Hamtini (2008). The certification does not assess factors such as the
ability to apply knowledge in practical situations, the perceived degree of ease in
interacting with the ICT environment, and the confidence of the learner in using the
knowledge.
Randall and Zirkle (2005) suggested that additional vendor-specific training
might be required after obtaining ECDL/ICDL certification because the certification
does not encompass many task-specific skills. The general implication of the
literature is that the EDCL/ICDL learning modules and certifications address only the
most rudimentary computer skills and does not provide a clear indication of computer
literacy. The effectiveness of the core curriculum in the educational environment may
also depend on factors such as the ICT training of teachers and the ability of teachers
to adapt the teaching to local student needs (Mioduser, 2003). To some degree, these
30
assessments of ECDL/ICDL support the argument of Whitworth (2005) that ICT
training is regarded as a service subject that focuses primarily on teaching operational
skills and not on critical thinking for using skills in applied situations. In addition,
they support the premise advanced by Dillon and Tearle (1999) that the effectiveness
of ICT training in terms of outcomes depends on the content of the ICT training
modules as well as the degree of previous knowledge and experience of the learner.
2.3.4.2. Effectiveness for ICT Training
Dixie and Wesson (2001) conducted a survey among faculty and alumni of
universities in South Africa to assess the perceived effectiveness of the ICDL
program modules used as the foundation for classroom instruction. The findings led
to the conclusion that the ICDL modules cannot be used for complete IT proficiency
education because of the failure to incorporate critical thinking skills and theory into
the instruction. The research, however, did not examine the possibility of outcome
variation in terms of passing the ICDL certification test with the use of different
instructional approaches or techniques. These findings support the conclusions of
Randall and Zirkle (2005) from a review of previous research, which indicated that
vendor-neutral methods of instruction in computer proficiency are appropriate only
for providing basic operational skills. Additional education is necessary to provide
students with theoretical foundations in ICT and more advanced computer skills to
solve complex problems. To some degree, these findings conform to the argument of
Reffell and Whitworth (2002) suggesting that ICT education is multi-faceted, with the
ICDL programme addressing only a single type of knowledge necessary for ICT
proficiency. In addition, standardised ECDL/ICDL training modules to prepare
31
teachers in secondary schools for certification may fail to address the issue of lack of
customised training for teachers that takes existing ICT skills into consideration,
which was identified in the Bectra (2004) survey.
Research conducted by Preston, Danby and Wegrif (2005) found that the
ECDL was highly effective for instructing professional educators in the fundamental
skills necessary to use basic computer programmes. The study used a pre-test and
post-test methodology in which an initial assessment was made of teachers in an
ECDL preparation programme was made using a survey questionnaire. The same
questionnaire was administered to the teachers after the completion of the
programme. The programme was structured to provide both courseware and tutors to
answer questions. The findings indicated that the teachers completing the programme
to prepare for the ECDL believed that the modular approach provided them with the
necessary skills for certification and as a result, use basic ICT in the classroom. The
findings also indicated that success in completing the preparation programme is
linked to support for the teachers from their schools. The findings of this study should
be treated with caution, however, because the research was commissioned by a firm
producing courseware to prepare teachers for the ECDL.
Kriksciuniene, Sarkiunaite, and Sakalauskas (2006) conducted one of the few
empirical evaluations of the ECDL using a return on investment approach. The
methodology examined the time associated with providing training for students to
support their ability to obtain ECDL certification compared to the number of errors
made by the students before and after the training process in routine computer
processes. The findings indicated that the error rate decreased substantially after
32
training, which the authors argued was a proxy measure for higher productivity. The
findings imply that schools, firms or other institutions training personnel to obtain the
ECDL can expect a return on investment in the form of higher productivity that
offsets the cost of the training.
2.3.5. Gaps in Existing ECDL/ICDL Literature
A gap exists in the literature from the lack of prior research investigating the
effectiveness of the ECDL/ICDL in meeting the objectives of providing sufficient
computer skills necessary for business and educational purposes as suggested by
Kennewell (2002). The existing research has investigated methods used to prepare
students for ECDL/ICDL testing (Calzarossa, et al., 2005) and perceptions of
effectiveness (Dixie & Wesson, 2001) in different cultural environments. Fallon (n.d.)
also indicated that there has been insufficient investigation with regard to the
following: 1) the relevance of the ECDL/ICDL instructional models, 2) the cost
effectiveness compared to alternative ICT training approaches, and 3) the
effectiveness in providing students and professionals with the desired skills for
accomplishing necessary tasks. There have been no cross-cultural comparisons of the
ECDL/ICDL and no analysis of the effect of different educational methods on
outcomes.
2.4. ICT and ECDL/ICDL in Sultanate of Oman
2.4.1. ICT in the Sultanate of Oman
In 2006, the Sultanate of Oman established the ICT Authority to promote
growth and development in the nation in the ICT sector, which is intended to remedy
the deficiency of relatively low levels of ICT training (Oxford Business Group,
33
2009). The ICT Authority works in conjunction with the Ministry of Education to
develop educational programmes to ensure that students in Omani schools receive
sufficient basic ICT training that can be applied to academic and business tasks. The
ICT education programme in the Sultanate of Oman is based on an integrated strategy
developed by the Ministry of Education to ensure that students obtain both core skills
and critical thinking abilities when using technology (Ministry of Education, n.d.).
The core skills include the ability to use ICT to obtain information and to perform
routine processes. The critical thinking skills are more complex and involve the
ability to synthesise ideas and construct new knowledge using ICT and the ability to
identify problems in processes and outputs. A particularly important critical thinking
skill is the ability to adapt to change and independently acquire new ICT knowledge.
The educational programme is also intended to increase the general computer literacy
level of the Omani population by offering short skills-development courses open to
the general public (Oxford Business Group, 2009).
2.4. 2. ECDL/ICDL in the Sultanate of Oman
The Sultanate of Oman introduced the ICDL programme to general education
students in grade 11 in 2004 as a mandatory part of the curriculum (Ministry of
Education, n.d.). The purpose of the programme was to ensure students had basic
competencies in skills such as word processing and spreadsheet operations necessary
for the job market. The programme was also intended to make ICDL accreditation
available to all teachers, who were particularly urged to obtain ICDL certification to
ensure they have fundamental ICT skills (Ministry of Education, 2004). To
implement the ICDL programme, 360 schools were designated as ICDL training
34
centres and 12 schools were designated as ICDL testing centres. In 2007, however,
the Sultanate of Oman eliminated the ICDL programme as mandatory for grade 11
replacing it with optional ICT courses for general education students. The ICDL
training remains available for students and teachers at the designated schools and
certification can be obtained through the testing centres. The current approach to ICT
education in the Sultanate of Oman could lead to insufficient range of knowledge,
which is a potential outcome from the lack of integrated ICT education as noted by
Reffell and Whitworth (2002). In addition, there have been no assessments of the
effectiveness of ECDL/ICDL in Oman using approaches such as the four-level ICT
training assessment model recommended by Wu, et al., (2004).
2.5. Conclusion
ICT, literacy, described in terms of both core skills and transferable skills, is
an important element of education from the standpoint of both students and teachers.
ICT education needs to include core skills training, critical thinking skills applied to
ICT selection and use, and the ability to evaluate the outcomes related to the use of
ICT. In general, and specifically in the Sultanate of Oman, ECDL/ICDL programs are
not achieving all of these goals.
35
Chapter 3: Methodology
3.1. Introduction
This study used a concurrent mixed quantitative and qualitative method to
investigate the effectiveness of the ICDL in-service training program for professional
development and ICT competency among teachers in Oman. The quantitative phase
of the study uses a survey questionnaire to collect data about the effectiveness of the
ICDL training programme among teachers in Oman who completed the ICDL
programme. The qualitative phase of the study uses teachers' interviews, classroom
observations, and pre-and post tests to collect data from teachers about the
effectiveness of ICDL training. The use of these mixed methods supported the
development of a case study of ICDL training in a Microsoft Access module.
3.2. Justification of the Methodology
The mixed method approach to research uses a pragmatic research paradigm that
combines the positivist paradigms associated with quantitative and empirical research
with the phenomenological research paradigm associated with quantitative research
(Gliner & Morgan, 2000). The positivist approach to research uses deductive
reasoning to draw conclusions about the subject under investigation from data
collected with objective methods using boundaries established by the researcher. The
quantitative phase of the research is based on the positivist paradigm and has the
advantages of the ability to collect numerical data capable of statistical analysis to
determine the influence of independent variables on the dependent variables.
Quantitative research findings are also capable of generalisation to a larger population
36
if the data was gathered using appropriate sampling procedures. The disadvantage to
the quantitative approach to research is the need for the research to remain confined
within artificial boundaries that may overlook critical factors influencing variables.
The phenomenological approach to research uses inductive reasoning to draw
conclusions about the subject under investigation from data collected with subjective
methods with the subjects of the research establishing the boundaries for the
investigation (Creswell, 2003). The qualitative approach to research allows the
subjects of the research to establish the boundaries for the data collection, with the
researcher interacting with the data during collection and analysis. The advantage of
the qualitative approach is its ability to identify a wider range of factors influencing
the topic under investigation. The disadvantage of the qualitative approach is the
possibility that researcher bias will influence the findings. When the two methods are
used in combination in a mixed approach to research, the advantages of each method
overcomes the disadvantages of the other method to provide a more accurate and
complete presentation of reality (Gorard & Taylor, 2004). The mixed method
approach and methodological triangulation with different approaches to data
collection and analyses provides greater rigour for the findings because of (Tashakori
& Teddie, 2008).
37
Table 1. Research Overview
Table 2. Quantitative and Qualitative Processes
Quantitative Study Process Qualitative Study Process
Invite and send surveys to Omani primary and secondary teachers from Al Dhahera Educational Region who have completed ICDL training to participate in responding to survey
Structured and unstructured observation of 12 student class in Microsoft Access ICDL Training
Completed surveys are returned by participants
Interview 12 students after they complete Microsoft Access ICDL Training (semi-structured interview)
Administer Pre-test to 12 students in Microsoft Access ICDL Training
Interview 2 ICDL trainers during the ICDL training (semi-structured interview)
Administer Post-test to 12 students after completion of Microsoft Access ICDL Training
38
3.3. Merging Quantitative and Qualitative Data
Driscoll, et al. (2007) suggests a method for merging quantitative and
qualitative data. Qualitative data analysis software like NVivo is able to generate
quantitized frequencies of emergent themes in narrative data and can suggest
influential codes. Qualitative data can also be enumerated by the frequency of themes
within the relevant sample or by the percent of themes or categories related to a given
respondent category (e.g., gender). This quantitized qualitative data can then be
effectively compared to the quantitative data captured in the study. The disadvantage
to this method is that the rich content and depth of understanding gained through
qualitative methods can be lost through the quantitization process. One challenge to
merging the two types of data is that qualitative data is usually obtained with smaller
sample sizes than the quantitative data. A sample size that is too small might not
allow quantitization of the qualitative data.
The process for merging qualitative and quantitative data recommended by
Driscoll, et al., (2007) includes the following steps: 1) code open-ended in-depth
interview content with content analysis software, 2) integrate the data using a data
management package (e.g., Microsoft Access), 3) assess combined data, explaining
discrepancies in survey responses by providing contextual data on those survey
responses.
3.4. Quantitative Phase of the Study
Because of the mixed method research design, the procedures used in the
quantitative phase of the study differed from the procedures used in the qualitative
phase. The quantitative phase used a survey questionnaire disseminated to teachers
39
who had completed the in-service ICDL training to obtain data concerning the
independent and dependent variables of the study. The quantitative procedures
required establishing a sampling protocol, developing test instruments and data
collection procedures, and establishing the reliability and validity of the instruments.
3.4.1. Survey Questionnaires Methodology
Survey methodology is an important measurement in social research. Surveys
include any measurement method that is centered on asking participants questions in
a structured format. Surveys are used to reduce bias in research and their standardized
format allows for better generalizability of quantitative results (Trochim, 2009).
Survey questionnaires depend on the participant’s ability to respond, their
understanding of the questions, and their motivation to respond honestly. Survey
questionnaires do not allow researchers to ask probing questions to understand a
participant’s honesty, motivation, or understanding.
3.4.2. Quantitative Sampling
The sampling for the quantitative phase of the study used purposive sampling
which is appropriate when the objective of the sampling is to relate the characteristics
of the respondents to certain characteristics under investigation in the research (May,
2001). Sampling required two approaches to identify the study population. A
directory of Omani primary and secondary teachers from Al Dhahera Educational
Region in Oman was used to identify potential subjects for the survey questionnaire
component of the study, which is the approach to identifying members of the
population recommended by Lodico, Spaulding, and Voegtle (2006). The criteria for
inclusion in the survey questionnaire sample were completion of the ICDL training,
40
and a current teaching position in a primary or secondary schools in Oman. Obtaining
ICDL certification was not a criterion for inclusion because some prospective
participants completing the training course may not yet have qualified for
certification. The individuals in these directories and who met the criteria were
solicited for participation in the study through email contact, with an explanation of
the purpose of the study and a qualifying question for participation as to whether they
have completed ICDL training. The sample consisted of those teachers who agreed to
participate. The sample included 203 teachers who finished the ICDL training in Al
Dhahera Region in Oman. . The procedure resulted in 82 participants that returned
completed surveys.
3.4.3. Quantitative Instruments
A survey test instrument was developed for the study to collect data from
teachers that have completed the ICDL training (see Appendix A). The survey
questionnaire consisted of three sections. The first section used closed-end questions
to obtain data from the respondents concerning motivation for taking the training,
existing ICT competency, use of ICT in the classroom, and ICDL training evaluation.
The second section consisted of a five-point Likert scale asking respondents to rate
their level of agreement with various statements concerning ICDL training ranging
from strongly agree to strongly disagree, which is a scale for assessment
recommended by Bernard (2000). The third section of the questionnaire used open-
ended questions to obtain information about recommended improvements to the
ICDL training course.
41
3.4.4. Data Collection Procedures
The data collection procedures for the survey component of the quantitative
research involved the dissemination of the survey questionnaire by email to the
teachers agreeing to participate in the study. A follow-up request was made to non-
respondents approximately one week after the initial dissemination of the survey
questionnaires. The procedure resulted in the return of 82 usable survey
questionnaires. In order to preserve the confidentiality of the respondents, the survey
questionnaire instruments did not contain information about the identity of the
respondents. While an initial coding was used for follow-up with the survey
questionnaires the coding linking the identity of the respondents with the instruments
was destroyed immediately after the completion of the study. Preserving
confidentiality ensured candour in the responses and maintained the privacy of
respondents (Schutt, 2008).
3.4.5. Data Analysis
Descriptive statistics describe the basic elements of the research data and
provide summaries about the sample and the measurements. Together with graphics
analysis, descriptive statistics form the basis of most quantitative analysis of data
Descriptive statistics allow comparisons of the same data across different groups
Descriptive statistics are helpful in understanding a phenomenon without trying to
find relationships that extend beyond the data itself (Trochim, 2009). Descriptive
statistics were used to describe the respondents in terms of gender, # of years
teaching, subject taught, and prior experience using ICT, showing frequencies and
percentages for each demographic. Additional descriptive statistics were generated
42
to describe the frequencies and percentages for each survey response with regard to
the participants’ motivation to take the ICDL course, skills that they hoped to
improve, and skills that they believe they did in fact improve, current and post-
training ICT skills, reasons for not using ICT in teaching, and new applications that
the teacher might use after training. Additionally, the frequencies and percentages
were generated for questions related to the following: 1) most useful modules, 2)
training aspects that were missing, 3) content that was missing, and 4) optimal
learning style for the participant. Each of these response categories was then
compared based on the demographics of gender, years teaching, class taught, and
years of ICT experience.
Univariate analysis examines, across cases, one variable at a time. Three
characteristics can be looked at: distribution, central tendency, and dispersion :For
this study, the distribution of the variables described above were looked at across the
four demographic factors (gender, years teaching, course taught, and previous ICT
experience) to see the frequencies and percentages of response categories that fall into
each demographic category. Central tendency in this study will look at the mean, of
each demographic and response category. Standard deviation measured dispersion.
The Standard Deviation shows the relationship of values to the mean of the sample.
ANOVA was used to analyse the data from the survey questionnaire to
determine if ICDL training significantly influences the attitude of teachers toward
incorporating ICT into classroom teaching methods. ANOVA was also used to
determine the existence of statistically significant differences in ICT competencies
gained through ICDL training based on the mediating factors of gender, number of
43
years teaching, and class being taught. ANOVA is suitable for examining differences
among groups following a treatment such as ICDL training (Pedhazur & Schmelkin,
1991). For the ANOVA, the alpha level was set at .05, which is suitable for research
involving variables that are difficult to measure with precision (Berry & Sanders,
2000). The survey instrument also generated descriptive statistics.
3.4.6. Reliability and Validity
A pilot test with ten respondents meeting the inclusion criteria for the study
was conducted to establish the reliability of the instrument using a test-and-retest
procedure. The pilot test followed the recommendations of Aaker, Kumar, and Day
(1995) for ensuring that the questions were not ambiguous and that the instrument
produces the same results when used in repeated testing. The test and retest had a
correlation of .83, which is sufficient to establish the reliability of the instrument. The
instrument used for the pre-and-post test was the standardised ICT competency test
used for the ICDL modules. Courseware providers assessed this ICT competency test
for reliability. The external validity of the instruments was established by the use of
random selection methods for participants, which increases the ability to generalise
the findings (Black, 1999). The internal validity of the instruments was established by
a peer review of the ability of the respondents to understand the questions in the
instruments.
3.5. Qualitative Phase of the Study
The qualitative phase of the study involved the development of a case study of
ICDL training for Microsoft Access in a single class of Omani teachers. The data for
the case study consisted of observations of classrooms in which teachers were
44
receiving in-service training in an ICDL module. Additionally, semi-structured
interviews with teachers that had completed ICDL in-service training were used to
collect data for the qualitative phase of the study. . The qualitative phase also
included a pre-and-post test of competencies in Microsoft Access. Interviews are
appropriate for qualitative research because they allow the interviewees to control the
data and direction of the interview (Minichiello, et al. 1995). Interviews can help
measure learning outcomes related to Kirkpatrick’s Level 1, reaction to the learning
(Wu, et al., 2004). Observational methods are suitable to obtain data about the
methods used by teachers in the classroom (Waxman & Padron, 2004). This method
contributes to capturing learning outcomes related to Kirkpatrick’s Level Three,
behaviour changes that result from the learning (Wu, et al.). A pre-and-post test
provided information about the outcomes of the training in the Microsoft Access
class. Use of the pre- and post-test contributes to measuring Kirkpatrick’s Level 2
learning outcome related to content learned (Wu, et al.). By combining the three
methods, the research obtains data about teaching methods and student response to
the methods.
3.5.1. Sampling
Purposive sampling was used for both components of the qualitative phase of
the study, which is appropriate when the objective of the sampling is to relate the
characteristics of the respondents to certain characteristics under investigation in the
research (May 2001). The sampling for the observational and interview component of
the qualitative phase limited the observation to three classes teaching the Microsoft
Access module of the ICDL training. This class was selected for observation because
45
it involves a software application with which many of the teachers may not have had
previous experience. In addition, confining the observations to a single ICDL topic
reduced the possibility that different instructional content could function as a
confounding variable. A specific criterion for the selection of a class to observe was
that it included a minimum of ten students to ensure a sufficient cross-sectional
representation in the class. Because the research design for the qualitative phase
called for interviewing the students of the observed class following its conclusion, the
agreement of the students to participate in the interviews was also a criterion for the
selection of a class to observe. The sampling process resulted in 12 interviews of the
students in the observed Microsoft Access class.
3.5.2. Instruments
The data collection instrument used for the observation component of the
study is a modified form of the classroom observational schedule (COS)
recommended by Waxman and Padron (2004) (See Appendix B). The COS
establishes a structure for observations based on: a) interactions of students with
teachers and peers; b) purpose of interactions; c) settings or situations in which
interactions occur; d) type of material on which students are working; e) specific
activities on which students are working; f) manner or approach used by teachers and
students; g) language used. The approach focuses on the content of the interactions.
The modifications are based on the assumption that in-service teachers are motivated
adult learners. The COS approach focuses on the students for observation rather than
the teacher. This provides greater breadth to the data for determining the effect of the
instruction methods on the behaviours of students in the learning environment. The
46
interviews used with the teachers in the ICDL class under observation employed a
semi-structured conversational format with open-ended questions (see Appendix C).
The interview questions followed the recommendations of Wengraf (2001) for using
questions that employ the language common among the. Both telephone and face-to-
face interviews were also conducted with ICDL trainers.
The qualitative component of the methodology also involved the use of a pre-
and-post test instrument for teachers taking ICDL training in the Microsoft Access
class under observation. The instrument used for the pre-and-post test component of
the quantitative research was the assessment of Access competencies and was a
standardised instrument with the participants certifying they had never previously
taken the Access test.
3.5.3. Data Collection Procedure
In the COS data collection procedure, each student is observed multiple times
during the course of the observational period (Waxman & Padron, 2004). The process
repeats, producing multiple observations of each student. The data collection
procedure also collects information about the teacher and teaching methods through
the observation of interactions with the students. Observational data was collected
about the behaviours of the students and the interactions of the students with the
teacher and peers for each student during each observation period, resulting in
multiple observations per student per class observations. Several observations were
made of the same class to ensure that the data obtained from the observation was
typical, and not anomalous data caused by a temporary external factor such as student
47
weariness. The first observation was unstructured, the second structured using COS,
and the final observation was unstructured.
The interviews were conducted using Skybe and an add-in called HotRecorder
to record the interviews, with each interview lasting approximately 40 minutes. The
interviews were conducted using a conversational style, with the researcher acting as
interviewer in this process. The interviewer could ask follow-up questions to obtain
additional information or clarification of issues raised by the interviewee. The
interviewer also avoided commenting or demonstrating non-verbal reactions to the
responses of the interviewees as recommended by Taylor and Bogdan (1998) to
reduce the possibility of influencing subsequent responses. The interviews were
recorded with the knowledge of the interviewees to facilitate transcription of the data
for analysis. The interviewees were not identified by name during the interviews or in
the interview transcripts to preserve their confidentiality.
The data collection procedure for the pre-and-post test component of the
quantitative phase of the study involved administering the standardised test
instrument to the subjects of the study prior to beginning ICDL in-service training in
Microsoft Access and after the completion of the in-service training on the final day
of the training. The procedure excluded any participants that did not complete the
training. The procedure resulted in 12 completed pre-and-post tests.
3.5.4. Data Analysis
The observational data was analysed by assessing the type and frequency of
behaviours and interactions occurring with each student under observation to identify
critical events. A critical event is a behaviour or interaction that could influence
48
learning activities or outcomes (Wragg, 1999). The specific critical events examined
in the analysis were the differences in learning styles among the students and the way
in which the ICDL courseware and the interactions with the teachers addressed these
differences.
Content analysis was used to analyse the data from the interviews as
recommended by Anastas (1999). This approach examines the data to identify the
general constructs noted by the interviewees, and is referred to as open coding. The
data it then examined several additional times to determine the patterns related to the
general constructs, which is referred to as axial coding. The review of the data
continues with sufficient iterations to ensure that all patterns within the data have
been identified. The outcome of the analysis is presented as an integrated descriptive
narrative of the findings organised around the constructs and patterns identified
within the data. The analysis for the pre-and-post test data used ANOVA, which is
appropriate when the purpose of the analysis is to determine the existence of
statistically significant differences in data means following a treatment such as ICDL
training.
3.5.5. Trustworthiness
The methodology has an inherent limitation from the possibility that
researcher bias influenced the research design, data collection procedures, and
conclusions drawn from the findings of the study. Trustworthiness is the approach
used to establish the reliability and validity of qualitative research, which reduces the
effect of researcher bias. Trustworthiness involves the four elements of credibility,
dependability, applicability, and ability to transfer the data and conclusions (Lincoln
49
& Guba, 1985). Credibility in the qualitative research was established by the
triangulation of observation data, interview data, and pre-and-post test data and by
prolonged engagement with the data collected over an extended period. The use of a
structured and consistent method of collecting the data for the qualitative phase of the
study established dependability by reducing the possibility of researcher bias
influencing data collection techniques. The purposive sampling approach established
the applicability of the data and findings because the data collection occurred using
subjects exposed to the same type of learning environment for the same purpose of
obtaining the ICDL training. The element of transferability is the weakest of the
trustworthiness factors in this methodology. It is similar to external validity in a
quantitative approach and refers to the ability to generalise the findings (Potter,
1996). Because the qualitative component of the research involved a single class with
a limited number of teachers, the ability to transfer the findings to other ICDL
instructional contexts may be limited.
3.6. Limitations of the Methodology
The research design did not include observation of the teachers’ use of
technology in their teaching practices in the classroom. As a result, the data
concerning the teachers’ use of ICT in the classroom following ICDL training is
based on self-reports, which have not been objectively verified. In addition, the
research design did not include pre-ICDL training surveys for teachers to determine if
the training resulted in a change in self-reported skill levels. The Likert survey does
ask the teachers to evaluate their ICT competency, as they perceived it prior to the
50
training and after the training. However, the self-reports are not objectively verified
with observation.
3.7. Summary
The goal of this mixed method research was to add to the body of knowledge
about the effectiveness of the ICDL in-service training program for improving ICT
competency among teachers in Oman. As the literature review reveals, ICDL training
often provides value in teaching core ICT competencies, but the training can be
lacking in providing insight into how teachers might better utilize ICT in the
classroom. This study provides improved understanding of the value of ICDL training
in Oman and provides insight into how the training might be improved. The findings
in this study can be generalised since the same educational policies and training are
found everywhere in Oman because the policies come from the Capital of Oman
(Muscat) to all the educational regions.
51
Chapter 4: Findings
Because the mixed method research design used concurrent quantitative and
qualitative components, the findings are divided into separate sections for each
component. The findings for the quantitative component based on the survey
questionnaire are presented as descriptive statistics from the questions in part 2 of the
survey questionnaire. The information contained in part 3 of the survey questionnaire
is qualitative in nature. The analysis of this data is presented with the findings of the
qualitative component of the study. The qualitative findings also include the analysis
of the observations of the ICDL Access class, the pre-and-post test for the class, and
the interviews with the students and trainers from the class. The results of the pre- and
post-test are reported under the quantitative section below.
4. 1. Quantitative Findings
The quantitative findings of the study are based on the survey questionnaires
from 82 teachers that have completed the ICDL training and the pre-and-post tests of
the Access class under observation.
4.1.1. Survey Questionnaire
The quantitative findings present the analysis of the data from part 2 of the
survey questionnaire. Table 3 reports the mean and standard deviation for each
question in part two of the survey questionnaire.
52
Table 3
Mean and Standard Deviation, Part 2 of the Survey Questionnaire_____________________________________________________________________Question Mean Std. Dev_____________________________________________________________________1. ICDL Training is effective for improving myICT competencies 3.13 1.082. The ICDL training provided me with skillsI will transfer to the classroom 3.08 1.073. The ICLD training provided adequate time tothoroughly learn the information about the software 2.87 .974. The ICDL training taught me to use various ICTsoftware applications to solve future education or research problems 3.01 1.145. The ICDL training increased my understanding of theimportance of ICT in the educational system 3.38 1.156. The ICDL training increased my understanding of theway in which ICT can be integrated into classroominstruction 3.14 1.197. After completing the ICDL training and certification process, I feel more confident integrating ICT into myclassroom teaching 3.17 1.158. The ICDL certification is necessary for careerdevelopment. 3.85 1.079. The ICDL training taught me the full range of capabilities of the software applications covered bythe ICDL certification. 2.64 1.0410. The ICDL training provided all the books, handouts or online materials necessary for learning to use the applications required for certification 1.90 .9811. The ICDL trainer adapted to my needs and addressedany obstacles I experienced during the class. 3.09 .9912. The ICDL trainer provided adequate guidance andassistance to learn the applications required forcertification 3.26 1.0213. The ICDL trainer used teaching methods that werehelpful to me. 3.25 1.0514. In general, the ICDL training was useful for me as a teacher 2.93 1.0415. The ICDL training provides a strong foundation for future ICT education and learning 3.36 1.14____________________________________________________________________
53
The analysis of the means and standard deviation for part 2 of the survey
questionnaire provides an indication of the central tendency of the data, with a mean
of 3.0 suggesting the respondents were neutral towards a proposition. The findings
show that the respondents were neutral toward the proposition in question 1 that the
ICDL training was effective for improving ICT competencies, which had a mean of
3.13. In addition, the standard deviation for this question was relatively high at 1.10,
suggesting substantial variability in the responses to this question. The data also
indicated that the respondents were neutral towards the proposition in question 2
concerning whether the ICDL training provided skills that can be transferred to the
classroom. The findings from questions 4 through 7 are particularly significant for
the attitude of the respondents towards incorporating ICT use into classroom
teaching. While the respondents were neutral towards the statement in question 4 that
the ICDL training taught them to use software applications to solve future educational
problems, the findings for questions 5 and 6 suggested that the training increased
awareness of the importance of ICT in the classroom and slightly improved
understanding of the way that ICT can be integrated into classroom teaching. The
responses to question seven suggest that the ICDL training slightly increased the
confidence of the respondents when using ICT in the classroom.
The findings also showed that the respondents had substantial agreement with
the statement in question 8 concerning the need for ICDL certification in career
development. Another notable finding was the substantial disagreement among the
respondents with question 10 concerning the adequacy of the materials in the ICDL
course, which had a mean of 1.90. The data from question 14 indicates that there was
54
slight disagreement with the proposition that ICDL training was useful for teachers
but there was general agreement with the proposition in question 15 that it provides a
foundation for future ICT learning.
The data from part 2 of the survey questionnaire was used to examine whether
differences exist in ICT competencies gained through ICDL training that are based
on the mediating factors of gender, number of years teaching and class being taught.
The first approach to analysis tested whether a relationship existed between the
number of years teaching and questions 1 and 2 of the survey questionnaire. Question
1 was related to the self-reported improvement in ICT competencies while Question 2
was related to the effectiveness of the ICDL training in providing skills transferrable
to the classroom. Because the data about number of years teaching was sequential, a
regression analysis was performed between the data and the responses to questions 1
and 2. The findings showed a coefficient of correlation of .028 for question 1
and .067 for question 2, with coefficients of determination of .0008 and .004 for the
two questions respectively. This evidence indicates that no significant relationship
exists between the number of years teaching and the effectiveness of ICDL training.
55
Table 4
Means and F statistic for Gender (Questions 1 and 2)_____________________________________________________________________ Mean –Male Mean – Female F-statistic F-critical_____________________________________________________________________
Gender1. ICDL training is effective for 2.97 3.16 .574 3.96improving my ICDL competencies
2. The ICDL training provided me 3.45 3.26 5.44 3.96with skills that I will transfer to the classroom_____________________________________________________________________
The second factor examined was whether gender functions as a moderator for
the findings, with 48.8% of the respondents being male and 51.2% of the respondents
being female. The ANOVA analysis of the data is shown in Table 4 for Questions 1
and 2 of the survey. The results indicate that no statistically significant differences
existed in the means between the male and the female group for questions 1 and 2.
The mean for the male group in question 1 was 2.97 and 3.16 for the female group
while the mean for the male group for question 2 was 3.45 and 3.26 for the female
group. The analysis indicates that the differences in the means were due to normal
statistical variation in the sampling. In addition, no statistically significant differences
were found by gender for the answers to the other survey questions.
56
Table 5
Statistically Significant Differences in Responses in Subject Taught Data Grouping_____________________________________________________________________Question F statistic F Critical_____________________________________________________________________1. ICDL training is effective for 2.96 2.13Improving my ICDL competencies5. The ICDL training increased my understanding of the 4.90 2.13importance of ICT in the educational system7. After completing the ICDL training and certification 4.33 2.13process, I feel more confident integrating ICT into myclassroom teaching10. The ICDL training provided all the books, handouts 3.24 2.13or online materials necessary for learning to use the applications required for certification13. The ICDL trainer used teaching methods that were 2.40 2.13helpful to me.15. The ICDL training provides a strong foundation for 2.85 2.13future ICT education and learning_____________________________________________________________________
The third factor examined was whether the subject matter taught by the
respondents moderated the findings. These results are shown in Table 5. The
respondents indicated they taught the following subjects: English, Social Science,
Arabic, ICT, Physical Education, Mathematics, Science, and Islamic Studies. The
ANOVA analysis of the data from the second part of the survey questionnaire
indicated that statistically significant differences among the groups existed for
questions 1, 5, 7, 10, 13, and 15 of part 2 of the survey questionnaire.
57
Table 6
Means of the Subject Taught Groups with Statistically Significant Differences
_____________________________________________________________________
Question English Social Science Arabic ICT Physical Math Islamic Studies Education Studies___________________________________________________________________________1. 3.06 3.63 2.86 3.41 4.57 2.50 2.25 3.60
5 3.40 4.18 2.86 3.75 4.14 3.50 2.07 4.00
7. 2.93 3.63 2.60 3.58 4.14 3.25 2.25 4.00
10. 1.86 2.18 2.00 1.91 3.14 2.00 1.16 1.60
13. 3.20 3.72 3.20 3.66 3.57 3.50 2.33 3.80
15. 3.53 3.63 3.06 3.75 4.28 3.50 2.41 3.20
An analysis of the means of the responses of the teachers in the eight subject
matter groups provides an indication of the direction of the differences between the
groups and the results are shown in Table 6. The analysis of the responses for
question 1 indicate that the physical education and mathematics teachers did not agree
with the statement that ICDL training is effective for improving my ICT
competencies (mean = 2.50 physical education; mean =2.25 mathematics). The
mathematics and science teachers also had significant disagreement with the
statement in question 5 that the ICDL training increased my understanding of the
importance of ICT in the education system (mean = 2.86 science; mean = 2.07
mathematics). Similarly, the science and mathematics teachers disagreed with the
statement in questions 7 that the ICDL training increased their confidence for
integrating ICT into classroom teaching (mean = 2.60 science; mean =2.25
mathematics). These findings with respect to the lack of effectiveness of ICDL
58
training among science and mathematics teachers was not anticipated because of the
assumption that science and mathematics teachers are conversant with ICT and would
be more likely to appreciate the importance of ICT in education. The pattern of
disagreement or lower agreement with the statements by the science and mathematics
teachers continued in the other questions assessing the effectiveness of the training
for certification and the overall evaluation of the training.
4.1.2 Pre-and-Post Tests of Access Class under Observation
During the qualitative phase of the study, the 12 members of the Access class
under observation took a test prior to the beginning of the class to establish a baseline
for Access skills and another test at the conclusion of the class to determine the effect
of the class for improving Access skills. These results are quantitative and the
analysis method is discussed here in the quantitative phase, even though this data was
part of the qualitative phase of the study. The mean for the group in the pre-test was
40.8 out of a possible 100, with a standard deviation of 12.02. The mean for the group
in the post-test was 70.33 with a standard deviation of 14.80. When the means of the
group were analysed with ANOVA, it produced an F statistic of 28.53 and an F
critical of 4.30. This data indicates that attending the class produced a statistically
significant difference in the mean scores of the study participants. The higher mean of
the post-test indicates that the class was effective for improving the Access skills of
the learners. The increase in standard deviation, however, suggests that the class
produced variable outcomes, with some students benefiting more from the instruction
than others do.
59
4.2 Qualitative Findings
4.2.1 Descriptive Statistics: Motivation, Expectations, and Skills Gained
The first section of Part 1 of the survey questionnaire obtained data on the
motivation and expectations of the respondents for taking the ICDL training.
The findings indicated that 30.4% the respondents were motivated to take the ICDL
training for professional development, while 50% of the respondents took the training
because it was an employment requirement. The remaining 19.5% were motivated by
personal interest. The data also provided information about the ranking of skills that
the respondents hoped to improve through the ICDL training. This section also
identified the priority for the skills the respondents hoped to improve through the
ICDL training, which is presented in Table 7.
Table 7
Skills Expected to Improve form ICDL Training as Ranked by Respondents
_________________________________________________________________1. Improved Computer Skills2. Microsoft Application Skills3. Word Skills4. Access Skills5. Improved Internet Skills6. No Need_____________________________________________________________________
Among the respondents, 41.4% reported no change in their self-rated levels if
ICT competency before and after taking the ICDL training. The group reporting no
change included individuals who rated their pre-training competency at both the high
and low end of a five-point scale, suggesting that the level of pre-existing ICT
knowledge was not a factor influencing the outcome of ICDL training.
60
The data obtained from the first section of the survey questionnaire also asked
respondents to indicate some of the reasons they have not used technology in the
classroom. The data indicates that the two reasons most often cited by the respondents
are personal lack of technology skills and lack of sufficient ICT equipment in the
classroom. Some of the respondents also noted that technology did not enhance
learning. The respondents also prioritized the skills gained in the ICDL training that
they were most likely to transfer to the classroom setting, which would occur despite
the barriers to use of ICT in the classroom identified by the respondents. Table 8
presents the skill gained in the ICDL training that the respondents reported as most
likely to transfer to classroom teaching in order of priority. The priority in Table 8
also reflects the evaluation of the usefulness of the ICDL modules.
Table 8
Skill gained in ICDL Training most Likely to Transfer to the Classroom Settings_____________________________________________________________________1. Presentation (Power Point)2. Spreadsheets3. Basic Concepts of IT4. Information and Communication5. Using the Computer and Managing Files6. Information and Communication_____________________________________________________________________
61
4.2.2 Descriptive Statistics: Deficiencies in the Training
The respondents also indicated that the lack of one-on-one time with the
trainer was a significant deficiency in the ICDL training, which was followed by
insufficient homework. The type of materials most often identified as missing form
the course was paper handouts followed by web tutorials. The preferred leaning style
among the respondents was direct instruction from the trainer. The data suggests that
the respondents would have preferred greater direct instruction from the trainer on a
one-on-one basis, with additional explanatory materials distributed during the ICDL
training.
4.2.3 Descriptive Statistics: Suitability of Training for Teachers, Recommended
Improvements
Part 3 of the survey questionnaire contained open-ended questions obtaining
information about suitability of the ICDL training for teachers and suggestions for
improving the training. The data from this section of the survey questionnaire
indicated that 60% of the respondents believed that ICDL training was important for
teachers, although there was disagreement about the actual effectiveness of the
training experienced by the respondents. Among the respondents who did not believe
that ICDL training was suitable or important for teachers, the reasons cited included
the lack of training concerning methods to incorporate ICDL into classroom teaching
and the lack of training about using the software in general applications not related to
the ICDL test. A common criticism among the group that did not believe the training
was relevant to teachers was the unwillingness or inability of the trainer to discuss
ICT application uses in the classroom.
62
The respondents provided numerous recommendations for improvements to
the ICDL training. Respondents felt that the amount of time for the training in each
module should be increased. They also suggested that an initial assessment be made
of pre-existing ICT knowledge and skills, with the classes divided based on ability.
Respondents felt that the trainers should be knowledgeable in the full range of
applications of the software to the educational environment because all students in the
classes are teachers interested in professional use of ICT skills. The training should
also provide books, handouts, and other materials to supplement the CD testing
software. An additional area of improvement would be adding applications such as
Photoshop or Flash that are outside the ICDL testing, but which represent ICT
applications that teachers might use.
4.2.4 Observation of Access Class
Descriptive statistical data from the pre-and-post tests is included as part of
the observational findings for the Access class.
63
Table 9
Among the individuals taking the class, 8 or 66% passed the ICDL test for the Access
module. The highest rate of failure in the ICDL test was among teachers of Arabic
and Islamic Studies. Three out of four of the Arabic teacher participants failed the
ICDL test and one out of two Islamic Studies teacher participants failed.
The observation of the Access class consisted of unstructured and structured
observations. The unstructured observation occurred with the initial class during
which the learners presumably having no prior knowledge of Access and employed a
COS method. The initial class lasted for 67 minutes. In the first ten minutes, students
listened to direct instruction from the trainer in the basic use of Access. The trainer
used an overhead projector to demonstrate. The students took notes during the direct
instruction period. For the next fifteen minutes, students perform rudimentary tasks
64
with the software such as creating a file name and a table as instructed by the trainer.
The instructor frequently asks general instructional questions with no students able to
provide the answers. When the instructor asked if the learners had any questions, the
same group of learners responded with questions. Forty minutes after the class began;
the instructor asked if any learners required assistance with the assigned task of
creating fields, with three learners asking for assistance. The instructor provided the
one-on-one assistance. At the same time, the other learners were working in pairs
with peers. One hour after the beginning of the class, the trainer moved to the next
task of customising tables, and sent a file to the learners’ computers to allow them to
practice sorting records.
The criteria for the structured observations were based on the data obtained
from the questionnaires and interviews. The observation indicated that 100% of the
students were beginners in Access, with 75% attending the class to obtain
certification while 25 attended the class for professional development. During the
class, the only materials provided consisted of CDs with software applications. No
supplemental materials such as paper handouts, videos, or tutorials were provided to
the students. The central training material was the test software. The instructor used a
projector to provide visual demonstrations to accompany direct instruction. Although
there were more than a sufficient number of computers for students, the test software
did not operate in many of the computers and some were infected with viruses. As a
result, only 11 computers were available for 12 students, forcing two students to share
the same computer. The fact that two students had to share a computer created a risk
to independent learning for these two students because one student was interacting
65
with the software while the other student was merely watching. During the class, the
students did not request additional materials from the instructor.
During the class, the trainer thoroughly covered the topic of certification
requirements, but did not cover topics related to use of ICT in the classroom or using
ICT tools to improve productivity. The trainer provided minimal information about
critical assessment of ICT tools. Although the trainer tried to promote critical
thinking by asking questions, the learners were not able to provide answers to the
questions. As a result, the instructor rather than the learners provided the answer.
During the class, the instructor allowed time for practise of basic tasks with the
instructor and peers providing assistance. In addition, the instructor welcomed
questions, and adapted to the needs and obstacles identified by the students. The
instructor sometimes attempted to engage the learners in discussions about the topic,
engage in subject-related tasks, and managed technical issues that arose. The
instructor did not inform the students of future learning opportunities in ICT and the
students did not inquire about such opportunities. In addition, the materials provided
for instruction did not provide information about future learning opportunities in ICT.
Several unexpected observations were noted. The learners were tired and
distracted during the class, which may have been because of a relatively short amount
of time between finishing work teaching in schools and the ICDL instructional
session. The instructor did not provide any supplementary assignments for the class,
relying solely on the test software for Access. As a result, the students had no
opportunity to extend their knowledge of ICT beyond the information necessary for
ICDL certification. The trainer was also ineffective in managing the class, with at
66
least ten minutes of instructional time lost through side discussions with individual
students. The trainer failed to discuss some of the issues raised by individual learners
with the entire class, confining the discussion to the individual raising the issue.
4.2.5 Interviews with Trainers
The open coding of the data from the two interviews with the ICDL trainers
identified the themes or dimensions of specific training to instruct ICDL courses,
methods of instruction, supplemental materials, and assessment. In the theme of
specific training for ICDL instruction, the interviewees noted they had received no
specialised training for the modules or the subject matter. One trainer had a
Bachelor’s Degree in ICT education while the other trainer had Bachelor’s Degree in
Education with certification in physics and mathematics. Both trainers used direct
instructional methods. There was a difference, however, between the two trainers in
the approach to one-on-one instruction. One trainer considered ICDL students as
adult learners with the motivation and responsibility for acquiring knowledge from
the instructional course. This trainer stated: ‘If the learner does not develop such
skills by practicing and using his mind I as a trainer can do nothing for him.’ The
other trainer viewed the students as adult learners, but was willing to provide one-on-
one assistance. Both learners believed the purpose of the training was specifically to
prepare learners to pass the ICDL certification test. The trainers indicated that
supplemental materials such as paper handouts were distributed at the beginning of
the course and believed the amount of materials were adequate to support learning.
The assessments performed by the trainers are aimed at ensuring the students are
prepared for the ICDL testing. At the same time, the trainers admit that passing the
67
ICDL test does not mean that the skills learned are the same for all students or that
the student will be able to apply the knowledge gained from training in a work related
situation. The trainers also suggest that the learner’s primary concern during class is
passing ICDL certification. One stated, “The learners do not care about them (higher
level skills). Their only concern is to pass the test”
4.2. 6 Interviews with Students of Access Class
The open coding of the interviews with the students in the Access class
identified the themes of insufficient time in the ICDL training, the focus of the ICDL
training on passing the certification test, structural deficiencies in the ICDL
instructional methods, change in attitude towards ICT following the ICDL course,
and increase in confidence for use of ICT in the classroom.
All the interviewees noted a shortcoming in the ICDL modules of insufficient
instructional time to learn the applications. The expectation of many of the
interviewees when taking the course was to have sufficient time during the training
sessions to learn the applications. The interviewees noted that the expectation was
generally not met. This was a particular difficulty for individuals who did not have
sufficient self-efficacy to continue studying the subject on their own. One interviewee
commented that: ‘we did not have enough time for discussions with him [the tutor].’
An issue related to time was the standardised number of hours of instruction for each
module course, although the skills required for some software such as Excel or
Access were more complex than for other types of software such as word processing.
An additional criticism of the interviewees was the strong focus of the training
on the requirements to pass the ICDL certification testing. The training was limited to
68
the specific skills necessary to pass the test. Other applications or uses of the software
not directly related to the certification testing were not considered because of the
limited time allowed for instruction. Possible additional uses of applications included
possible uses of the software for classroom instruction. Several of the interviewees
indicated that they focused on the testing software to a greater extent than the CD
provided as a supplement. One outcome of insufficient time was the need for students
to practice the tests together.
The time of day when the courses were offered presented a structural
deficiency in the training. Courses were offered at the end of the normal workday
when the students were tired. The student’s physical exhaustion seemed to reduce the
rate of learning. Another structural problem suggested by the students was the
mandatory attendance. This seemed to result in students not being motivated to learn
the material. For some students with existing ICT competencies, the material was too
fundamental, while it is too difficult for other students without sufficient basic ICT
skills. Another structural deficiency noted by the interviewees was a lack of
instructional support materials. While the students could access the instructional CD
software, they felt a need for additional books, handouts, and web-tutorials to allow
them to obtain a more comprehensive knowledge of the material. Not all the
interviewees, however, had sufficient ICT skills for independent study. These
findings suggest that the course did not address the needs arising from the variability
of the learners’ existing ICT knowledge. The majority of the interviewees, however,
were satisfied with the direct instructional methods used by the ICDL tutor. Several
interviewees also considered the poor maintenance of computers in the training
69
classroom as a structural deficiency, with the time spent in attempting to operate the
computers detracting from learning time.
Another central theme among the interviewees was a positive change in their
attitudes towards ICT in general because of the ICDL training. The positive change
appeared to be related to increased confidence in the ability to use various software
applications. All the interviewees noted improvement in confidence and skills in
some ICT areas, although most also noted lack of confidence with software such as
Access or Excel that they found particularly difficult. The interviewees further noted
the limitations of the ICDL training for understanding the full range of application of
the software. While the course taught the fundamental skills necessary to use the
software, many of the interviewees remained uncertain about appropriate applications
of the software.
The interviewees indicated that they would be using some of the software
covered in the ICDL training in classroom applications. Power point for presentations
was the most commonly cited classroom use of the knowledge obtained during the
course. One interviewee noted, ‘I am no longer afraid of using PowerPoint
presentations in the classroom.’ Several of the interviewees also indicated that they
would be using spreadsheets to maintain grades and other office applications for
designing lessons. These uses of ICT, however, involved presentation and
administrative tasks, and did not involve fully incorporating ICT into the tasks or
activities required of students. The interviewees noted that insufficient ICT
equipment in the classroom was a structural barrier to using ICT applications in
teaching.
70
Chapter 5: Discussion
ICDL training was evaluated in this study based on Kirkpatrick’s Model of
Training Evaluation, as described in Wu et al. (2004). The following four factors from
the model were used in the evaluation: 1)training outcome with regard to participants’
reaction to the training, 2) participants’ content learning, 3) participants’ behaviour
changes after training, and 4) as best as could be determined how the ICDL training
might impact overall teaching effectiveness related to improved use of ICT in the
classroom.
The quantitative component of the study indicated that the respondents to the
survey questionnaire had only very slight agreement with the statement that ICDL
training is effective for improving ICT competencies. In addition, 4.1.4percentage
reported no change in their ICT competencies because of the ICDL training. The data
obtained from the interviews with the students in the Access class appears to contradict
the findings from the survey questionnaire by indicating that the interviewees believed
that the ICDL training had improved their ICT skills. The quantitative evidence from the
pre-and-post test of the Access class also indicates that the participants experienced
substantial improvement in their ICT skills at least with respect to Access. Even the
individuals failing to pass the test for ICDL certification in Access had significantly
higher test scores after the ICDL training, which is an objective measure of improvement
in ICT competency. The discrepancy between the findings from the survey questionnaire
and the findings from the examination of the Access class may be because of the small
size of the sample for the Access class. The particular individuals included in this
component of the study may have experienced a benefit from ICDL training for
71
improving ICT competencies greater than the general experiences of a larger population.
These results provide a contradictory answer to the research question that asks which ICT
competencies were improved and which were not improved through the ICDL training,
Empirical evidence suggests that the application competency is improved because of the
training. However, the participants’ reaction to the training reflects an overall view that
they did not improve in ICT competencies with the exception of a specific mention of
improved use of PowerPoint. This might reflect a contradiction in how participants were
interpreting ICT competency. . If we refer to Easton, Easton, and Addo (2006), computer
literacy reflects an understanding of the basic function of a computer as well as an
understanding of the use of some software applications. On the other hand, information
literacy reflects the ability to know when information is needed and to have the ability to
locate, analyze, and effectively apply the information to a specific circumstance. The
range of participant interpretation of ICT competency could vary widely across the
spectrum between computer literacy and information literacy.
Similarly, the quantitative component of the study indicated that the respondents
to the survey questionnaire were neutral towards the statement that the ICDL training
provided them with skills they would transfer to the classroom. In the interviews in the
qualitative component of the study, however, the teachers (participants) in the Access
class indicated that they would use some skills in the classroom, and particularly
PowerPoint. The respondents to the survey questionnaires also indicated that PowerPoint
and other presentation software was the element of ICDL training that they would most
likely use in the classroom. . The interviewees in the qualitative component of the
findings noted that the ICDL training improved their confidence, although they remained
72
somewhat uncertain about the specific uses of ICT in the classroom beyond assistance
with presentation and administrative tasks. Although these findings are contradictory,
they nonetheless suggest that in answering the research question regarding how ICDL
training may have influenced the attitude of teachers towards incorporating ICT into
classroom teaching, within the limited scope of presentation and administrative tasks,
attitudes were improved.
The quantitative findings of the study provide evidence that the number of years
teaching and the gender of the teacher do not moderate the effectiveness of the ICDL
training. The subject taught, however, does have some influence on the effectiveness of
the ICDL training. Teachers of science, physical education, and mathematics had
substantially lower levels of agreement with various areas examined in part 2 of the
survey questionnaire. The outcome of the ICDL training for the Access class under
observation also had the highest certification failure rate among teachers of Arabic,
followed by teachers of Islamic studies. These findings suggest that teachers of certain
non-technical subjects may have a lower ICT base knowledge when taking an ICDL
course, which may contribute to a slower pace of learning the material.
Additionally, the differences in the responses to the various questions based on
subject taught may be because of the standardisation of the ICDL training and the failure
to segregate teachers taking the course based on abilities. The teachers of certain subjects
such as physical education may not have had previous training in ICT and enter the ICDL
course as a beginner. These students would require more materials, support, and attention
during the ICDL training. At the other extreme are teachers in subjects such as science
and mathematics that are likely to have substantial ICT knowledge prior to taking the
73
ICDL course. For these individuals, a standardised course may be too fundamental. .In
answering the research question of whether differences in gender, years teaching, and
subject taught would moderate ICDL effectiveness in teaching ICT competency the
answer is yes, but only in regard to subject taught. The issue of differences in the
effectiveness of the ICDL training among teachers of different subjects could be
addressed by segregating teachers into different ICDL classes based on their abilities.
The findings also identified many shortcomings in the ICDL training that might
be improved to increase effectiveness of training for teachers. The suggested
improvements might be considered as alternatives to the current form of ICDL training.
The trainers do not receive specialised instruction concerning the methods for ICDL
training, which might contribute to substantial variability in the manner of ICDL
classroom instruction. At the same time, however, the trainers indicated that they use
direct instruction methods, which conformed to the preferred learning style of the
respondents to the survey questionnaire. The variability in the instructional methods and
approaches among the trainers could account for the finding in the survey questionnaire
that one-on-one instruction and sufficient handouts were lacking form the ICDL training.
The observation of the Access class indicated that the trainer became involved in side
discussions with individual students reducing the amount of time the trainer was able to
devote to instruction or answering general questions applicable to all students.
The duration and the content of the ICDL training were also identified as
significant problems. According to participant responses, the time allotted to each ICDL
training course was insufficient to cover the material, and particularly affected the
modules with more complex content such as Excel or Access. In addition, holding
74
training in the evening created difficulties for some teachers because they were tired at
the end of the workday. Supporting Reffell and Whitworth (2002) trainers indicate that
this ICT teaching focuses on the skills necessary to operate the applications rather than
the skills necessary to evaluate the effectiveness and relevance of ICT output. Trainers
also identified the fact that the ICDL training used standardised content focused solely on
the requirements to pass the ICDL test, and they identified this as an obstacle to
understanding the full range of application use. The teacher participants were often
unable to identify practical classroom applications for much of the ICDL training; At
most, many of the teachers participating in the study indicated they would use
PowerPoint for presentations. The lack of applicability of training to the participant’s
specific educational context may have accounted for the high percentage of respondents
indicating that the ICDL training had not influenced their use of ICT in the classroom.
Smeets and Mooij (2001), show that ICT instruction is most when the curriculum and
rate of learning is designed to meet the specific needs of the student. Both quantitative
and qualitative analysis in this study provides an answer to the final research question
that alternative training methods might better meet the ICT learning needs for teachers
These findings suggest that alternative approaches to teaching ICT competencies
to teachers be considered. One alternative may be to modify the content of the modules to
increase applicability to the professional educational environment. Modifications could
include shifting the focus of the training away from passing the ICDL test and toward
classroom applications of ICT. Modifications could also include the use of trainers with
prior experience in using ICT applications in the classroom. The findings might also
75
suggest that a different training course, in addition to ICDL, could provide training
specific to use of ICT in the classroom.
Becta, (2004), supports the lack of ICT equipment and improper equipment
maintenance identified in this study. The factor of little or poorly maintained equipment
is relevant to the effectiveness of ICDL training as well as to the subsequent use of ICT
in the classroom. The observation of the Access classroom as well as the interviews with
the teachers attending the class indicated that improper equipment maintenance reduced
the ability of all students to use a separate computer. It also caused delays in teaching
while the trainer and the students jointly attempted to ensure that the software could be
used in the computers, many of which had been infected by viruses. In addition to the
specific problems in the ICDL training environment, the respondents to the
questionnaires frequently indicated that insufficient ICT equipment and maintenance was
a barrier to incorporating ICT into classroom teaching.
It appears that the general structure of the ICDL course being geared toward
passing a certification exam sets expectations for both trainers and students as to what the
purpose of the training is. Trainers perceive students as being there to learn how to pass
the test, and students see the training as being designed as a standardized process to
facilitate passing certification tests. This expectation at both ends inhibits the ability of
the training to extend outside of these boundaries.
The primary research question for this study asked whether ICDL provided the
best approach for Oman with respect to training its teachers to use technology effectively
and whether the ICDL training is effective for providing teachers with the ICT skills
necessary in the academic and educational environment. Overall, the study suggests that
76
while ICDL is effective in helping teachers receive certification, the training is not
effective in expanding teacher awareness of how to use ICT in the classroom to improve
learning. This finding supports Perez and Murray (2006) who adopted the position that
the ECDL/ICDL certification should be viewed as an assessment tool to determine if
individuals have the basic skills to operate rudimentary computer programs, but not as an
assessment of ICT literacy. With the exception of increased confidence and
understanding of how to use PowerPoint in the classroom, all aspects of the study show
that ICDL training is limited in the scope of what it teaches.
77
Chapter 6: Conclusion
The rapid growth of technology and Internet use demands that educators improve
ICT competency to improve the use of software applications and ICT technology for
presentation, administration, and enhanced teaching effectiveness. Additionally, the
advanced level of technology use by students suggests that effective teaching might need
to include expanded use of technology in the classroom in order to engage students and
hold their attention. ICDL training for ICT competency was designed as a course to help
students pass certification training. For this purpose, standardization and a narrow focus
were important in the design of the training. However, what educators need in terms of
ICT training includes the expanded ability to make critical assessments of ICT and to
understand how to effectively apply ICT to enhance the educational experience for
students.
This study shows contradictory results with regard to a participant’s belief that
they have improved ICT competency and what empirical test scores show about
improved understanding of the applications that were studied. Definitions play a role here
and the varied interpretations of how ICT competency or literacy is interpreted across
participants probably play a role in explaining the contradictions. It might be important to
educate teachers first on the differences in computer literacy, ICT literacy and
competency, and information literacy. The definitions vary even in the literature review.
There might be value in the education industry creating definitions for each term that are
specifically relevant to educators. The fact that most participants in this study were
neutral as to whether they improved ICT competency in the ICDL course, reflects a lack
78
of clarity in understanding the term. Neutral responses often emerge when there is
ambiguity.
In addition to potential differences in how ICT competency is being interpreted,
there are also variations in ICT competency and ICDL success across educators who
teach different subjects. The study suggests that those who teach in completely non-
technical areas like languages, humanities, or cultural areas have lower starting ICT
competency, and do not appear to learn the required competencies taught for certification
in the ICDL training. It is unknown what contributed to the obstacle in learning for some
students, but it does suggest that ICDL training is not effective for students starting at the
lowest end of the ICT competency spectrum and supports the need for a more
rudimentary beginner’s course that precedes the certification course. Perhaps with the
addition of this rudimentary course, an advanced course could also be developed. ICDL
training could be seen as one phase in the overall process of a more complete ICT
literacy-training program.
Training needs to have clearly stated objectives, and it is clear that the objective
of the ICDL training is to prepare students to pass a certification test in specific software
applications. Use of the ICDL for purposes beyond this is unlikely to be successful. The
question arises then, as to whether ICDL training should be altered, or whether there are
other alternative methods to train educators to apply ICT in their classrooms. The training
of critical assessment and application of ICT to a variety of scenarios requires critical
thinking and problem solving training that is not likely to be well supported in the
standardized format of the ICDL course. Additionally, to add this element to the ICDL
course, takes away from the objective of a narrow focus on passing certification tests.
79
Although students had the opportunity to suggest web-based tutorials or self-paced
training, the preference appeared to be or instructor led training and this should be
considered when moving forward to a more comprehensive training plan. Students did
indicate a willingness or interest in doing homework for the course, which reflects some
interest in self-directed work while still under the supervision of a face-to-face trainer.
Participants in the study demonstrated improved confidence for the application
they learned in the ICDL training. However, it might be that the increased confidence is
not taken advantage of because there is no further training to show educators how to
apply ICT within the classroom. Additionally, the study identified lack of equipment and
poorly maintained hardware as a factor that impedes learning during training. The
importance of hardware and software problems might be underestimated as a factor that
impedes both learning and application of the learning. Difficulty with equipment can
reduce a new learner’s confidence and create a fear of trying to apply ICT in the
classroom.
Improving ICT competency for educators appears to require training beyond the
ICDL. Better definition of ICT competency within the educational field is necessary so
that learning objectives can be defined, and from those objectives, effective training can
be developed.
80
Chapter 7: Recommendations
The study results also allow the recommendation of a number of improvements
for ICT competency training for educators. In the short term, some improvements to the
exiting ICDL training can be recommended.
7.1. Recommendations for Improvement to Training
7.1.1. Make Training Relevant to Educators
Providing the ICDL training with a context of using the application in a classroom
setting would add to the value of the training for educators while still accomplishing the
objective of training for ICDL certification. Assignments, activities, and exercises could
be designed to reflect how an application might be used in a teaching environment.
All students have the right to be taught by teachers with the same competence, for which
standards need to be set. Nowadays, competence not only means subject competence, but
also includes the other skills required for innovative education.
7.1.2. Training Length
While the students feel that the training should be longer, the addition of
homework assignments might add time to the use of the application while still preserving
the allotted time for the existing training. The homework could serve the purpose of
additional use of the application and apply relevant scenarios for educators as homework
assignments. Additionally, trainers need to optimize the classroom training time by
avoiding sidebar conversations and by bringing all questions and issues to the entire class
to expand the learning.
81
7.1.3. Training Time
Normally teachers in Oman take their professional development courses within
the working days. The ICDL training was an exception. The teachers took it after they
finish a full working day, which caused lack of concentration or carelessness. It is
recommended that it be given within the working day as any other professional
development course. The teachers can be released either for a full day or for some time
within the working day.
7.1.4. Training Materials
The students clearly state a desire for and need for additional materials in the form
of handouts and books. To reduce the expense of paper materials, online materials might
be a solution. The students could be provided links to supplemental tutorials, books, and
articles as well as suggestions or ways to continue their learning when the ICDL training
is over. Although the scope of the ICDL training does not include training on how to
apply ICT in the classroom, material in this area could be provided as Web-based content
for future reference.
7.1.5. Trainer’s Training
It is obvious from the interviews with trainers and trainees that the trainers did not
receive any adequate and specific training to teach the ICDL other than passing the ICDL
test, so it is recommended that the trainers should receive adequate training to train ICDL
to others.
7.1.6. Phased Approach to ICT Training
The ICDL should be considered one phase towards ICT literacy in Omani schools.
Teachers completing this programme should be enrolled in more general courses first or
more specific courses aiming at introducing the computer into the classroom.
82
In the long term, the results of this study suggest a phased approach to ICT competency
training with stages for the novice, the beginner, the intermediate, and the advanced
student. The end goal of this phased training would be to enable and empower teachers to
use ICT skills in the classroom to enhance and improve student learning.
7.2. Recommendations for Future Research
Several areas for future research also emerge from the results of this study.
7.2.1. ICT Competency Differences for Teachers in Different Subject Areas
The somewhat unexpected variation in ICT competency and success of learning
between educators in different subject areas suggests a need to understand the causes of
this variation. Understanding the applications of ICT in non-technical courses is
important and understanding the motivation of teachers in non-technical courses to learn
ICT would be important areas for research. Because the participants who failed the post-
test also demonstrated failing scores for the pre-test scores, it suggests a need to further
analyze the specific areas of ICT competency that are inadequate to determine what
needs to be taught in a beginner’s course.
7.2.2. The effect of lack of and poorly maintained equipment
The influence of poorly maintained equipment and lack of equipment on training
success is a significant area for study. The influence of equipment problems for the
students at the low end of ICT competency might be a relevant factor in their success in
that these students are less able to understand or adapt to these types of problems.
Additionally, studies related to the effect of outdated, inadequate hardware and software
on teacher use of ICT in the classroom could reveal interesting results. The focus of most
ICT competency research and research in how ICT is applied in the classroom focuses on
83
the teacher’s knowledge of ICT and the quality of ICT training. Understanding the
influence of hardware and software problems on the willingness of teachers to use ICT in
the classroom might be revealing.
84
Chapter 8: References
Aaker, A., Kumar, D., & Day, G. (1995). Marketing research. New York: John Wiley &
Sons.
Anastas, J.W. (1999). Research design for social work and human services. New York:
Columbia University Press.
Becta. (2004). What the research says about the barriers to the use of ICT in teaching.
Available at:
http://www.mmiweb.org.uk/publications/ict/Research_ClassroomOrg.pdf.
[Accessed 24 July 2009].
Bernard, H.R. (2000). Social research methods. Thousand Oaks CA: Sage Publications.
Berry, W. & Sanders, M. (2000). Understanding multivariate research. Boulder CO:
Westview Press.
Black, T. (1999). Doing quantitative research in the social sciences. Thousand Oaks CA:
Sage Publications.
Colosi, L. & Dunifon, R. (2006). What’s the difference? Post then Pre? & Pre then Post?
Cornell Cooperative Extension.
Calzarossa, M., Ciancarini, P., Maresca, P., Mich, L., & Scarabottolo, N. (2005). The
ECDL programme in Italian universities. Computers & Education, 49(2), 514-
529.
Costa, F. & Peralta, H. (2006). Primary teachers’ competence and confidence regarding
the use of ICT. In Pearson, E. & Bohman, P. (Eds.), Proceedings of world
conference on educational multimedia, hypermedia and telecommunications 2006
(pp. 46-52). Chesapeake, VA: AACE
85
Creswell, J. W. (2003). Research Design: Qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, CA: Sage Publications.
Csapo, N. (2002). Certification of computer literacy. T.H.E. Journal, 30(1), 46-53.
Dillon, P. & Tearle, P. (1999). Ensuring effectiveness of ICT training. Telematics Centre,
School of Education, University of Exeter.
Dixie, C.H. & Wesson, J. (2001). Introductory IT at the tertiary level—is ICDL the
answer? Proceedings of SAICSIT Annual Research Conference. Available at:
http://osprey.unisa.ac.za/saicsit2001/Electronic/paper36.PDF. [Accessed July 7,
2009].
Driscoll, D., Appiah-Yeboah, A., Salibz, P., & Rupert, D. (2007). Merging qualitative
and quantitative data in mixed methods research: How to and why not. Ecological
and Environmental Anthropology, 3(1), 19-28..
Easton, A. Easton, G., & Addo, T. (2006). But I am computer literature: I passed the test.
Journal of College and Teaching, 3(2), 39-44.
ECDL/ICDL digital literacy for education. (2008). Available at:
http://www.pliroforiki.org/joomla/index.php?
option=com_docman&task=doc_download&gid=34&Itemid=57. [Accessed 25
July 2009].
European Computer Driving License Foundation [ECDL]. (2005). Developing
courseware for ECDL/ICDL.
Fallon, H. (n.d.). The use of the European Computer Driving Licence (ECDL) as an IT
training package in Irish universities. Available at:
http://eprints.nuim.ie/11/2/sconul.pdf. [Accessed 24 July 2009].
86
Garnett, F. (2008): Six ICT literacies, working paper.
Gliner, J. & Morgan, G. (2000). Research methods in applied settings. Mahwah NJ:
Lawrence Erlbaum Associates.
Gorard, S. & Taylor, C. (2004). Combining methods in educational and social research.
Maidenhead UK: Open University Press.
Gordon, N. (2007). Experiences of embedding information technology into discipline
based teaching. Italics, 14 (January). Available at:
http://www.ics.heacademy.ac.uk/italics/Vol4-1/gordon.pdf. [Accessed 25 July
2009].
Hamtini, T. (2008). Evaluating e-learning programs. Journal of Computer Science, 4(8),
693-698.
Jung, I. (2005). ICT-pedagogy integration in teacher training: Application cases
worldwide. Education Technology & Society, 8(2), 94-101.
Kennewell, S. (2002). The nature of ICT as a subject. In Kennewell, S. (Ed.) Learning to
teach ICT in the secondary schools (pp. 18-36). London: RoutledgeFalmer.
Kiridis, A., Drossos, V., & Tsakiridou, H. (2006). Teachers facing information and
communication technology. Journal of Technology and Teacher Education,
14(1), 75-93.
Kriksciuniene, D., Sarkiunaite, I. & Sakalauskas, V. (2006). Evaluating return on
computer literacy of the enterprise with ICDL. Innovations in Information
Technology, 1-5.
Lau B. & Sim, C. (2008). Exploring the extent of ICT adoption among secondary school
87
teachers in Malaysia. International Journal of Computing and ICT Research,
2(2), 19-36.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage
Publications.
Lodico, M., Spaulding, D., & Voegtle, K. (2006). Methods in educational research. New
York: Wiley.
May, T. (2001). Social research issues, methods and processes. Philadelphia: Open
University Press.
Minichiello, V., Aroni, R., Timewell, E., & Alexander. L. (1995). In-depth interviewing:
Principles, techniques, analysis. Melbourne: Longman Cheshire.
Ministry of Education, Sultanate of Oman. (2004). Ministry of Education integrates the
ICDL programs. (September 24) Press Release.
Ministry of Education, Sultanate of Oman. (n.d.). ICT and Education in the Sultanate of
Oman. Available at:
http://www.moe.gov.om/portal/SiteBuilder/Sites/EPS/Arabic/MOE/Newest/ICT
%20and%20education.pdf. [Accessed 24 July 2009].
Mioduser, D., Nachmias, R., Tubin, D., & Forkosh-Baruch, A. (2003). Analysis schema
for the study of domains and levels of pedagogical innovation in schools using
ICT. Education and information technologies, 8(1), 23-36.
Oxford Business Group. (2009). The report Oman 2009. Oxford: Oxford Business
Group.
Patton, M. (1990). Qualitative evaluation and research methods (2nd Edition). Newbury
Park: Sage Publications.
88
Pedhazur, L. & Schmelkin, L. (1991). Measurement, design and analysis: An integrated
approach. Mahwah NJ: Lawrence Erlbaum Associates.
Perez, J. & Murray, M. (2006). Journey to the centre of the core: Computers and the
internet in the core curriculum. Issues in Informing Science and Information
Technology, 3, 489-497.
Potter, W.J. (1996). An analysis of thinking and research about qualitative methods.
Mahwah NJ: Lawrence Erlbaum Associates.
Poulter, A. & McMenamy, D. (2003). Beyond ECDL: Basic and Advanced ICT Skills for
the New Librarian. Paper presented at 4th Annual LTSN-ICS Conference, Galway,
pp. 200-202.
Preston, C., Danby, M., Wegerif, R. (2005). ECDL for Educators. BCS/Aston Swann.
Randall, M. & Zirkle, C. (2005). Information technology student-based certification in
formal education settings: Who benefits and what is needed. Journal of
Information Technology Education, 4, 287-306.
Reffell, P. & Whitworth, A. (2002). Information fluency: Critically examining IT
education. New Library World, 103(1182-1183), 427-435.
Rockman, I. (2004). Introduction: The importance of information literacy. Available at:
http://media.wiley.com/product_data/excerpt/78/07879652/0787965278.pdf.
[Accessed 24 July 2009].
Robbins, R. & Zhou, Z. (2007). A comparison of two computer literacy testing
approaches. Issues in information systems, 8(1), 185-191.
Schutt, R. (2008). Investigating the social world. Thousand Oaks CA: Pine Forge Press.
Tashakkori, A. & Teddie, C. (2008). Foundations of mixed methods research. Thousand
89
Oaks CA: Sage Publications.
Smeets, E. & Mooij, T. (2001). Pupil-centred, learning, ICT, and teacher behaviour:
observations in educational practice. British Journal of Educational Technology,
32(4), 403-417.
Stanley, N. & Tanner, H. (2002) Assessing attainment. In Kennewell, S. (Ed.)
Learning to teach ICT in the secondary schools (pp. 109-129). London:
RoutledgeFalmer.
Tanner, H. (2002). The place of ICT in secondary education. In Kennewell, S. (Ed.)
Learning to teach ICT in the secondary schools (pp. 3-17). London:
RoutledgeFalmer.
Taylor, S. & Bogdan, R. (1998). Introduction to qualitative research methods. New
York: John Wiley.
Trochim, W. (2006). Research methods knowledge base: Descriptive statistics. Retrieved
on July 1, 2009 from http://www.socialresearchmethods.net/kb/statdesc.php
Waxman H. &.Huang, S..(1999).. Classroom Observation Research and the Improvement
of Teaching. InNew Directions for Teaching Practice and Research, Waxman, H
& Walberg, H. (Eds). Berkeley, CA: McCutchan.
Wengraf, T. (2001). Qualitative research interviewing. London: Sage.
Waxman, H. & Padron, Y. (2004). The uses of classroom observation schedule to
improve classroom instruction. In Waxman, C., Tharp, R. & Hilberg, S. (Eds.)
Observational research in U.S. classrooms. (pp. 72-87). Cambridge: Cambridge
University Press.
Whitworth, A. (2005). The state of ICT skills education. Italics, 14(January). Available
90
at: http://www.ics.heacademy.ac.uk/italics/Vol4-1/ITALICS_editorial.pdf.
[Accessed 24 July 2009].
Whitworth, A. (2009). Computer and information literacy. In A. Whitworth.
Information Obesity (pp. 75-105). Oxford: Chandos Publishing.
Wragg, E. (1999). An introduction to classroom observation. London: Routledge.
Wu, C., Chen, Y., Lee, G., Ho, R. & Chiou, G. (2004). Evaluation of an inservice teacher
training program ICT Integration. In Cantoni, L. & McLoughlin, C. (Eds.),
Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2004 (pp. 3451-3455). Chesapeake, VA: AACE
Young, J. (2004). Testing service to unveil an assessment of computer and information
literacy. Chronicle of Higher Education, (November 12), 33.
Zakopoulos, V. (2005). An evaluation of the quality of ICT teaching within an ICT-rich
environment: The case of two primary schools. Education and Information
Technologies, 10(4), 323-340.
91
Appendix ASurvey Questionnaire
Gender: Subject Taught: Level Taught: Years of Experience:
Part 1:
Motivation and Expectations
1. What was your motivation for taking the ICDL course?
a) Professional developmentb) Employment requirementc) Personal interestd) Not sure
2. Rank the skills that you hoped to improve through the ICDL training (1 will be the highest need)
o _____No needso _____Word skillso _____Access skillso _____Microsoft application skillso _____Improved computer skillso _____Improved Internet skills
_Others. Please specify: ___________________________
ICT Competency
3. How would you rate your level of ICT competency before starting the ICDL training?
1 2 3 4 5 Incompetent Slightly Neither Competent Highly Competent Competent
4. How would you rate your current level of ICT competency after finishing the ICDL course?
1 2 3 4 5 Incompetent Slightly Neither Competent Highly Competent Competent
92
Use of ICT in the classroom
5. Some of the reasons that I have not used technology in the classroom include: My own lack of technology skills My students’ lack of technology skills I do not think that technology enhances learning I am not sure which applications to use for specific purposes I don’t think my students like me to use technology Others: Please Specify
6. What specific skills gained in ICDL training are you most likely to transfer to the classroom setting?
_____Basic concepts of IT _____Using the computer and managing files _____Word processing _____Spreadsheets _____Database _____Presentation (PowerPoint)_____Information and Communication
7 Did the ICDL training result in your use of new software applications in the classroom?a. yes b. no
If yes, which applications? List them below: __________________________________If no, why not?___________________________________
ICDL Training Evaluation
8. Rate the modules from 1 to 7 according to which ones were the most useful for you. Start with 1 as the most useful and 7 as the least useful. _____ Basic concepts of IT ____ Using the computer and managing files ____ Word processing ____ Spreadsheets ____ Database ____ Presentation ____ Information and Communication
93
9. What learning support from the trainer do you think is missing in the course? (select as many as are applicable)
o One-on-one timeo Discussion timeo Be able to ask questionso Homeworko Practice time in class
Others: Please specify: _____________________________
10. What learning support from the materials do you think is missing in the course? (as many as applicable)
o CD contento Paper Handoutso Online sourceso Web tutorials
Others: Please specify: _____________________________
Favorite learning style:11. What is your favorite method of learning ICT?a. direct trainer's instructionb. Learning from peersc. self studyd. Other. Please specify: ______________________
Part 2: The following section asks you to rate your level of agreement with statements about ICT and ICDL training. A rating of 1 indicates you strongly disagree with the statement while a rating of 5 indicates that you strongly agree with the statement. Please circle the number that corresponds with your level of agreement.A Strongly
DisagreeDisagree Neither
Agree nor Disagree
Agree Strongly Agree
ICT Competencies1. ICDL training is effective for improving my ICT competencies.
1 2 3 4 5
2. The ICDL training provided me with skills that I will transfer to the classroom.
1 2 3 4 5
3. The ICDL training provided adequate time to thoroughly learn the information about the software applications
1 2 3 4 5
ICT Use in Classroom4. The ICDL training taught me to use 1 2 3 4 5
94
various ICT software applications to solve future education or research problems5. The ICDL training increased my understanding of the importance of ICT in the educational system
1 2 3 4 5
6. The ICDL training increased my understanding of the way in which ICT can be integrated into classroom instruction
1 2 3 4 5
7. After completing the ICDL training and certification process, I feel more confident integrating ICT into my classroom teaching
1 2 3 4 5
ICDL effectiveness for Certification 8. The ICDL certification is necessary for career development
1 2 3 4 5
9. The ICDL training taught me the full range of capabilities of the software applications covered by the ICDL certification.
1 2 3 4 5
10. The ICDL training provided all the books, handouts or online materials necessary for learning to use the applications required for certification
1 2 3 4 5
ICDL Training Evaluation11.The ICDL trainer adapted to my needs and addressed any obstacles I experienced during the class.
1 2 3 4 5
12. The ICDL trainer provided adequate guidance and assistance to learn the applications required for certification
1 2 3 4 5
13.The ICDL trainer used teaching methods that were helpful to me.
1 2 3 4 5
14. In general, the ICDL training was useful for me as a teacher
1 2 3 4 5
Future learning opportunities15. The ICDL training provides a strong foundation for future ICT education and learning
1 2 3 4 5
95
Part 3:1. Do you think that the ICDL training course is suitable for you as a teacher? Why?
2. What do you thing the course is missing?
3. How can the course be improved to be more effective?
96
Appendix BStructured Observation Form
Structured Observation Form
ExpectationsThe trainer introduced the purpose of the training to be_______________________________The students could be categorized into the following ICT knowledge level at the beginning of the course:
Beginner Intermediate Proficient# of students # of students # of studentsFrom what you observed in the class, the student’s purpose in being in class could be categorized as:
Certification Professional Development Personal Development# of students # of students # of studentsMaterialsThe materials provided in class included:CDs with software applications
CDs with training materials
Paper Handouts
Videos Tutorials
Each training module included the following:Training Material Comment
Did the students have questions about the materials?Questions about Materials Response from Trainer
Did the students request or express a desire for different or additional materials and what were they?________________________________________________________________________
97
TrainerThe trainer’s credentials and qualifications were:__________________________________________The trainer used the following teaching methods during the course:Lecture Interactive
DiscussionDemonstration Class
ActivitiesVideo Tutorial
Comments______________________________________________________________________________________________________________________________________The trainer’s primary focus during the training appeared to be:
Topic Trainer covered topic thoroughly
Trainer covered topic minimally
Trainer did not cover this topic
Certification RequirementsUse of ICT in the classroomUse of ICT tools to improve learner’s productivityCritical assessment of ICT toolsSecurity basics related to ICT toolsOther:
Comments______________________________________________________________________________________________________________________________________During training, the teacher did the following:
Often Sometimes NeverWelcomed questions and provided adequate responsesEngaged students in discussion about the topicProvided opportunities for students to engage and talk to or work with each otherAdapted to student’s needs and addressed obstaclesEffectively managed technical issues that arose
98
Comments:______________________________________________________________________________________________________________________________________Technical IssuesThe following technical issues arose during the course:Technical Issue Resolution to the problem Impact on student
learning
Comments:______________________________________________________________________________________________________________________________________Future Learning OpportunitiesDid the instructor inform students of future learning opportunities in ICT?___________________________________________________________________________________________________________________________________________
Did the students inquire about future learning opportunities in ICT
Future learning opportunities discussed by trainer or student during the class included:Advanced ICDL courses
Web-based training
Book/paper materials
Vendor Training
Did the students offer information about specific future learning interests?___________________________________________________________________________________________________________Unexpected ObservationsIn addition to the above observations, I observed the following
Observation Comment
Appendix C
99
Teachers' Interviews Questions
1. What did you expect from the ICDL training prior to the ICDL course?
2. A. What went well in the training?
B. What went badly? 3. Which module was
A. the most useful module and why?
B. the least useful module and why?
4. How did you learn? A. How did you interact with the tutor?
B. How did you interact with other learners?
C. Which teaching or learning materials did the tutor use and which did you use?
D. Which teaching methods did the trainer use?
5. Do you think that the training helped you learn ICT independently?
6. Has the training changed your attitudes toward the use of technology and if so, how?
7. What are the areas of weakness in the training and how can they be improved?
8. Has the training changed your confidence in using ICT in the class and if so, how?
9. Has the training helped you to use the skills you learned in unpredictable situations in the future?
10. How are you going to use the skills you learned in the future (in the classroom and life)?
Appendix D
100
Trainer Interview Questions
1. Which kind of training did you obtained before teaching ICDL?
2. What about ICDL itself, did you receive specific training for teaching the ICDL?
3. Which teaching methods do you use in teaching the ICDL?
4. What teaching materials do you use in teaching the ICDL?
5. Do you think the learners use the learning materials and resources as they should be used?
6. Which kind of difficulties do learners face and how do you help them in overcoming those difficulties?
6. Do you give the learners anything to study at home?
7. Do you find it a disadvantage to depend on the software test for assessing your students and if so, why?
8. Do you think that obtaining the ICDL certificate means that the learner is skilled in ICT and if so, why?
9. Did you receive any kind of support from the Ministry of Education or the ICDL organisation to help you in teaching the course?
10. How can the ICDL training course be improved?
Appendix E
101
Affirmation Letter from Participants
I affirm that I did not practice for either the pre- or post-test for the Access ICDL course before the tests were administered by the researcher.
Learner ID: ____________________________________
Signature: ____________________________________
102
top related