evaluation strategies - nba professional de… · web viewfor example, in the word processing...

Piloting Professional Development of Teachers in Use of IT in the Classroom

Evaluation of a SCOPE/Schoolnet Pilot Project

──────────────────────────────────────

Contents──────────────────────

Introduction 1

Chapter One: Management of the Pilot Project 5

Chapter Two: The School Selection Process 9

Chapter Three: The Introductory Course 13

Chapter Four: Mentor Support 18

Chapter Five: The Learning Process 27

Chapter Six: The Course Materials 36

Chapter Seven: Recommendations 51

AppendicesAppendix One: Invitation to Attend A SchoolNet Teacher Development

Planning Workshop 55

Appendix Two: Key Players 57Appendix Three: Comments on School Selection by Gerald Roos 59Appendix Four: Introductory Course Reports 63Appendix Five: Introductory Course Schedule 75Appendix Six: Mentor Guide 77Appendix Seven: Agenda for Information Literacy Materials Workshop 83Appendix Eight: Gerald Roos Interview Analysis 85


IntroductionBACKGROUND

This document is an evaluation of the pilot of a project run jointly by SCOPE1 and Schoolnet. The purpose of the pilot was to test approaches to training teachers how to use information and communication technologies (ICTs) effectively in supporting their classroom practice. This teacher development programme adopted a distance education approach, using e-mail mentor support.

The SCOPE programme has the following three focuses: • Incorporation of Colleges of Education into Higher Education; • Use of Information and Communication Technologies (ICTs) for Enhanced Learning; • Introducing Inclusive Education.

The overall objective of the ICT enhanced learning component (the second component above) is to extend and improve learners’ competencies in using ICTs in learning. More specifically, it focuses on enabling learners, educators and managers in the Northern Cape and Mpumalanga provinces to use ICTs in appropriate ways. This component aims to achieve three results during the four years of the SCOPE Project, as follows:• Result 1: Learner’s competencies in ICT extended and improving• Result 2: Educators competent in ICT enhanced learning and teaching• Result 3: Management of ICT improved

In the first result, the focus is on constructive and collaborative approaches in ICT in education, emphasizing communication between learners, teachers, and school administrators. In result two, educators’ competence in ICT enhanced learning will be studied and supported by developing a national framework for teacher development in ICT. On the basis of that framework, national in-service and pre-service courses will be developed. The major idea on the third result is to support managers to develop an ICT plan for provinces.

Schoolnet’s vision is to seek to support educators and learners in transforming education through the application of ICTs by providing leadership, expertise and developing effective partnerships in the areas of: • Internet connectivity and appropriate technology • Human Resource Development and capacity building • Content and curriculum management and development • Advocacy and marketing Schoolnet, and its partners, aims to meet the challenge of transforming South Africa’s education system from an industrial to a knowledge-based model, contributing to South Africa’s global competitiveness.

1 The South African–Finnish Cooperation Programme in the Education Sector (or SCOPE) is a four-year programme supported by the Ministry for Foreign Affairs of Finland and run in partnership with the South African Department of Education. The overall objective of the programme is to increase the capacity and to enhance the quality of education in the education system of South Africa.

1


The pilot project being evaluated in this report will be followed by a larger project during 2001, involving over 5000 educators.

PROJECT DESCRIPTION

PILOT PROJECT DEVELOPMENT

Project development began with a meeting held on 12 July, in Cape Town (see invitation in Appendix One). This first meeting was followed by a four-day workshop involving various course developers and materials writers, including, Alixe Lowenherz, Trudi van Wyk, Jyrki Pulkkinen, Cathy Zandee, Janet Thomson, and Gerald Roos, while others attended intermittently. (The roles of these individuals in the pilot, and others, are defined in Appendix Two.) During this workshop, the nature of the virtual learning environment was discussed extensively, the outcomes were formulated collaboratively, and the materials development process commenced.

The materials development process then continued beyond the workshop with a number of individuals taking part in materials writing and editing. The materials and most of the support resources for the course are contained on a CD-ROM provided to learners. During this time, various schools were selected from Kwazulu-Natal, Western Cape, and the North West Province to participate in the pilot, involving a total of 30 teacher learners.

INTRODUCTORY COURSE

In early September, three introductory courses were held in each of the three provinces. During these one-day courses, the learners were introduced to the resource CD, the distance education process was explained to them, and learners had to select one of three modules that they wished to pursue.

THE DISTANCE EDUCATION COMPONENT

The distance education component of the pilot project was supported by e-mail and online interaction (using the Internet), which were intended to involve collaborative groups of about ten educators, who had all selected the same learning module. A mentor, who made e-mail contact with the learners, facilitated each module group.

The learners were encouraged to follow a module on the CD, completing the activities described and submitting certain activities to their mentor and the rest of the learner group. The activities on the course encouraged learners to be reflective about what they were doing and also required them to prepare and try out activities in the classroom amongst their learners. After the classroom experience, learners were asked to share their thoughts with their group by e-mail and to enter and save their reflections in an MS Word document that was referred to as a reflective journal. The reflective journal was developed throughout the distance education component and sent to mentors, and was used to assess learners’ progress through modules.

2


MODULES

There were three modules available on the pilot, as follows:

1. Word Processor - this module concentrated on making learners independent users of the word processor as an educator. It helped learners to look for contexts in their work in which the word processor would be a useful tool.

2. Questioning Skills - this module helped learners to develop skills in asking questions that make their learners think and that encourage educators to provide situations in class whereby their learners can ask questions. Such enquiry would place the Worldwide Web and its information sources in context as a valuable resource in schools.

3. Information Search and Evaluation - this module concentrated on the skills required for using search engines on the Internet. It also helped learners to identify and use criteria for evaluating the reliability and value of information that is found on the Internet.

EVALUTION PROCESS

Until the SCOPE Project decided to proceed with an external evaluation, Schoolnet had decided to limit the evaluation process to internal evaluation undertaken by Gerald Roos. The decision to incorporate an external evaluation was only executed a considerable time after the start of the pilot project. The evaluation process was conducted over a period of three months following the pilot. This severely limited what the evaluation was able to determine about the efficacy of the educational approaches used (although the internal evaluation work commissioned by Schoolnet did gather much useful data). For example, we have had to rely entirely on secondary sources and interviews in examining the introductory course, as all three had been run before the evaluation began. As this data is largely constitutes the subjective opinions of the organizers and facilitators, it has been very difficult to offer meaningful comment on this component of the evaluation. Likewise, participants in the pilot had largely completed their work before the evaluation began, thus removing opportunities for observing learner engagement with the materials themselves.

Evaluation focused on different elements of the pilot, namely management of the project, school selection, the learning process, introductory course, the mentoring process, and the course materials. Information was gathered on each of these aspects of the course from a variety of sources (the details of these sources for each of the sections are discussed at the start of each of the relevant chapters). However, overall information sources for the analysis included the following: • Schoolnet documentation, including documents on the Schoolnet Internet site and a

Terms of Reference document.• Notes made by an independent consultant during interviews with various key

stakeholders, namely, Janet Thomson, Gerald Roos, Trudi van Wyk, Sandy Zinn, Alixe Lowenherz, Stephen Marquard, and Jyrki Pulkkinen.

• Gerald Roos’s notes on comments made during his interviews with ten participating learners, and his additional commentary on his findings.

• Responses to a short questionnaire compiled and e-mailed to certain stakeholders. • Gerald Roos’s summary document describing the schools involved in each region, as

well as the participants and how they faired on the course.

3


• Feedback forms (nine) completed by learners and other feedback from learners provided in e-mails.

• Trainer reports on the introductory courses.• Detailed observer reports on the introductory courses compiled.• A telephonic interview with one of the introductory course trainers. • E-mail discussions between the mentors, and between the mentors and learners.• The resource CD. • A number of text books on the development and evaluation of instructional materials in

distance education.

The analysis was primarily qualitative, with a small amount of quantitative analysis being conducted where possible and appropriate.

4


CHAPTER ONE Management of the Pilot Project

INTRODUCTION AND INFORMATION SOURCES

This chapter explores issues around the management of the pilot project itself. This section is placed first because management issues affected every other aspect of the pilot project, and thus this section is intended to provide a context for all observations and comments that follow. The information in this chapter is based on interviews with key participants in the pilot project.

ABSENCE OF PROJECT MANAGEMENT STRUCTURES

Management problems within the pilot project have been discussed at length through various processes revolving around this evaluation. Given this reality and because several problems have already been resolved, this report will not dwell at length on management problems and their effects. However, some aspects of management require raising because they yield important lessons and because they raise the difficulty of separating problems caused by ineffective management from problems caused by design or implementation flaws.

The pilot project suffered significantly from the near-total absence of any effective project management structures. Most notable amongst these was that no formal agreement was ever reached between SCOPE and Schoolnet on the nature and purpose of the collaboration. Although this may seem to be a minor omission on the face of it, it had serious implications throughout the entire pilot project, as it appeared that the motives of the two participating organizations2 often seemed to contradict each other. This did not reflect any fundamental disagreements, but was rather the result of aggregations of small differences of opinion as to the purpose behind the pilot. Because no agreement had been reached, each organization inevitably pursued its own well-defined organizational agendas rather than a vague collaborative agreement. Establishing this agreement up front would have helped to avoid this problem.

Following on from this omission, there was also no formally constituted project management team and jointly agreed project manager (although various people were informally tasked with various management responsibilities). These omissions had implications for every aspect of the pilot, as has been acknowledged by project participants. For example:• There was little evidence in the pilot project of risk analysis of the teaching and learning

methods selected. The remainder of this evaluation report demonstrates that, while the teaching and learning methods selected for testing during the pilot have undoubted potential, they also carry serious risk of failure. More systematic risk analysis would have avoided some obvious problems that arose during the pilot project (see bullet points below), and is a simple example of a management function.

2 The evaluators are aware that – technically – SCOPE is not an organization. However, for the purposes of this report, we have used this shorthand for convenience, as the pilot project – to all intents and purposes – constituted a collaboration between two distinct parties.

5


• Selection of schools was undertaken in a random, unsystematic way (see chapter two) that undermined many aspects of the pilot. The first phase of the Schoolnet Supercentres and Thintana Projects has demonstrated that setting up management structures and tasking individuals with specific management responsibilities significantly enhances processes such as schools selection. The addition of management accountability ensures that criteria for selection are agreed and adhered to during such a process, which would have significantly enhanced the pilot project. It also creates a framework for such activities to take place within timeframes that do not compromise the remaining activities in the project.

• Implementation proceeded in a random and haphazard way. This began with absence of criteria for selecting schools, but had much wider implications. For example, participating teachers did not have the implications of their participation adequately explained to them prior to attending the introductory course. Many appear to have thought that the one-day workshop was the full extent of their involvement, which undermined testing of the e-mail mentoring system. Likewise, there were unacceptable delays between running the introductory course and implementing the distance education component of the programme in the North West and Kwazulu Natal (because the training in the Western Cape was held a week later than in the other prvinces), making it impossible for the pilot to capitalize on the momentum generated by the introductory course.

• Contributors to the project also came into the project in random ways and their roles were often not made clear, either to them or others. This appears, at least in part, to have be related to the absence of a structured working agreement between SCOPE and Schoolnet, which saw individuals making management decisions without reference to a broader management framework. For some contributors, the effect was very alienating, and it also minimized the possibility of benefiting from new inputs. For example, Alixe Lowenherz was invited to an initial course materials development workshop and asked to participate as an editor. When the materials development process overran its deadlines, new deadlines were set without reference to their implications. At least one consequence of deadlines being too tight was that Alixe was not able to perform her editorial role with most materials because there simply was not enough time.

• The absence of effective management structures was most noticeable in the materials development process, where differences of opinion became post problematic. Several personal problems arose, which would have been easily avoidable in a more tightly managed process (particularly as they did not reflect any fundamental underlying conflicts). An identified manager could also easily have mediated and resolved conflicts as they arose, instead of which they were allowed to persist for unacceptably long periods, to the detriment of the pilot project as a whole.

There were other symptoms of this management gap, particularly in the lack of any documentation on the roles of individuals within the team and the activities and deadlines for the materials development process. Where deadlines were set, they often seemed unreasonably tight (a problem throughout the entire pilot). Perhaps most significantly, though, the person tasked with ensuring the conceptual coherence of the materials and indeed the whole learning process (Gerald Roos) also became the de facto manager of the course materials development process. The benefit of hindsight indicates

6


that it would have been better to separate these functions, as often the role of providing conceptual coherence creates tensions within a team, which it is easier for a different, more objective person to mediate.

• The pilot project clearly relied strongly on the commitment and energy of individuals to run. This is not a criticism of the project, or of the people whose commitment led to the pilot actually taking place. Without this commitment and enthusiasm, nothing would have been possible. However, it does run the risk of hindering growth and development, because, for example, when an individual leaves or becomes unmanageably overloaded, the project may not retain the skills and energy contributed by that person. More importantly, it runs the risk of undermining the potential of the project to go to scale in future, as it becomes unclear whether success is a function of the models developed or has worked in spite of the models through the commitment and hard work of a few key people. A key responsibility of effective management should be to ensure that such work becomes part of a sustainable scaleable system.

CONCLUSION AND RECOMMENDATIONS

Management problems were a serious impediment to the pilot project under evaluation. Absence of clear agreements and allocation of specific management responsibilities led to problems in all aspects of the project. This is not intended to suggest that other positive and negative lessons cannot be learned from the experience (as the following chapters reflect), but it does demonstrate the importance of effective management as a prerequisite to running projects of this nature, particularly when the scale up beyond their pilot phase. More importantly, they raise concerns about the ability of the pilot to test educational strategies thoroughly. For example, when problems arose with e-mail mentoring, it is not possible to establish clearly whether or not e-mail mentoring is a problematic strategy per se because the schools selection process did not take sufficient account of the prerequisites for participation in such a learning experience. Likewise, the opportunity to test strategies for collaboration between two projects sharing similar goals was undermined by lack of management structure and accountability in the relationship. Consequently, we believe that it is important to take the following into consideration in the next phase of this project:1. Ensure that there are formal, structured agreements between participating

organizations, that clearly define the purpose and nature of collaboration, as well as roles and responsibilities of each partner.

2. Establish joint management structures that are expected to meet regularly to discuss and resolve problems as they arise, and that incorporate other informal communication strategies for solving urgent problems in the shortest possible time.

3. Allocate specific management responsibility for separate components of projects to individuals and hold them accountable for this management function. Also ensure that one person is held accountable for managing the overall project, someone who is able to balance the strategic interests of all parties and is preferably positioned to be able to mediate any conflicts as they arise.

4. Ensure that management responsibility at all levels extends to ensuring that dependence on individuals and/or occurrence of specific, uncertain events or factors is

7


systematically minimized over time. This should include ensuring that all implementation models are capable of being scaled up where appropriate.3

3 Much of this is the role of the originally envisaged educational manager of the Thintana / SuperCentres project. The role of education manager appears to have been abandoned or fragmented at this stage. The consequences of this need to be examined carefully.

8


CHAPTER TWOThe School Selection Process


This section explores issues around the selection of schools involved in the pilot, reviewing the process and criteria that were used, as well as discussing key issues that should be considered. Because information on this aspect of the pilot was limited, questions were compiled and e-mailed to certain stakeholders, but only Janet Thomson was in a position to respond. Gerald Roos provided a summary document describing the schools involved in each region, as well as the participants and how they faired on the course. This document is summarized in the table in this section, but the full version can be seen in Appendix Three. Further comments on the school selection process also emerged from various documents and from interviews with other stakeholders.

SUMMARY OF SCHOOL SELECTION PROCESS AND ISSUES

According to Janet Thomson, no specific process was used to select the schools, although a planning meeting was held during which ten options for schools were discussed (which according to Janet, was ‘far too many’). The intended selection philosophy was examined, which sought to avoid problems by rewarding ‘good’ schools and neglecting ‘bad’ schools. It was highlighted that this approach had potentially negative long-term impact on school development, since only the best schools would get good resources, rather than schools where resources were most needed. Based on this, it was felt that schools that were most receptive amongst the disadvantaged schools should be selected (‘the best of the worst’). This would incentivize problem schools to implement plans to improve. The need to generate ‘short-term wins’ was also highlighted, which would not be achieved by selecting the ‘worst of worst’ schools.

The need to involve the Department of Education (DoE) was also discussed at this planning meeting: It was noted that use of technology in schools should be integrated with the DoE’s technology planning. Technology should be used to improve the situation at schools and not as a prize or reward. In this way, technology serves as a support for development. After the planning meeting however, no progress was made with plans for the pilot for some time, and, when the schools were finally selected a month later, it was done hastily, with no reference to the discussions at the meeting.

Originally it was decided that schools in three provinces (Kwazulu Natal, the North West Province, and the Western Cape), with ten educators from each province, would be selected. It was decided that most educators should not have used computers before. However, although many of the educators involved in the pilot did not have long-term exposure to computers, very few were first-time users of computers. Originally the intention was to have ten educators at one school, but this was not adhered to in the selection process . In reality, there was only one school where a group of educators from the same school took the course (four educators at Silverstream High School in the Western Cape). The design of the

9


programme assumed regular access to computers and the schools needed to be equipped with networked and online computers.

According to Janet, no single person had overall responsibility for organizing selection of the schools; Janet eventually took on the role of organizing schools in Kwazulu Natal, Penny Bussetto of WCSN (with some help from Sandy Zinn) organized the Western Cape schools, and Ntutule Tshenye organized schools in the North West Province.

Selection of schools appeared to be most thorough in the Western Cape. For example, Kulani High School was specifically identified by Sandy Zinn, who asserted that the school management was informed and involved (and had approved the staff participation) Vista and Silverstream were recommended by the Western Cape Education Department (WCED), as they both received special IT awards and had networks installed with the support of the WCED IT Unit. The assumption in these instances was that these schools would be better functioning and have better support than the average, which clearly turned out not to be the case for Vista.

Janet believes that the hasty and almost haphazard selection of schools was a major factor in many of the problems associated with the pilot, and highlights a number of issues that should be considered. A vital issue relates to access to computers, as this was clearly a problem at some schools, with teachers having limited time to work on the computer and therefore on the materials and assignments. One of the mentors, Sandy Zinn, indicated that, at one school, the principal did not allow access to computers, while six educators on the introductory course run in Cape Town mentioned the restrictive measures that were in effect at their schools with respect to teacher use of computer rooms. Therefore, it is clear that support from the schools involved is vitally important.

A further issue concerns the technical infrastructure at the schools, including the amount of technical backup and support to make sure that the technology functions and to address issues when they arise. It was also essential that people attending course had an existing e-mail address and that the school e-mail was in working order. Alixe Lowenherz suggested that Schoolnet should establish closer ties with provincial departments to go into schools and ensure that these problems are resolved in projects of this kind. Stephen also felt that external factors that stimulate motivation should be factored into school selection.

DETAILS OF SELECTED SCHOOLS

Details of the selected schools are provided below, including information on the infrastructure at each school, and the learners involved.

KWAZULU-NATAL

School IT infrastructure Learners Activity levelMostly from the Mpophomeni schools

• No computers - used Mpophomeni Computer Centre at the Zenzeleni Community Centre

• 8 of the 22 computers (including server) were stolen 2 days into the course;

• 2 were not practicing educators

• Personal problems were experienced including, health issues (pregnancy), family deaths

• Quite active - thanks to on-site support from Pam Robertson and others from the Centre

10


KWAZULU-NATAL

School IT infrastructure Learners Activity levelthe Internet connection was restored but a mess was made of the e-mail registrations and e-mail remained unreliable for some time

• 2 teachers were discouraged by the theft; once they returned, they had further e-mail problems and were again discouraged but did eventually progress

WESTERN CAPE

School IT infrastructure Learners Activity levelVista High • No Internet connection;

telephone had been suspended for some time due to a dispute over bill; network had lost its settings

• 2 teachers – 1 no longer taught at the school and the principal was the postmaster

• Neither teacher started the course

Sinethemba Secondary

• Dial up connection and teachers registered with e-mail, but incorrect address provided on registration

• 2 teachers were nominated

• 1 very keen but got frustrated – wrote to Janet, but never made contact with mentor because incorrect address

Kulani High School

• Gerald arranged for Schoolnet to be installed and WCSN responded promptly, but no learners registered; no communications reached them

• 3 teachers (share a mailbox)

• Not really interested in the course and seem to have been coerced to attend

Silverstream High

• Computer room with Internet dial up, but locked most of the time

• 4 educators • All active to some degree

NORTH WEST PROVINCE

School IT infrastructure Learners Activity levelSoshanguve High School

• Well-resourced computer centre with dial up

• 2 teachers, very capable computer users

• Both active

Ngaka Maseko High School

• Well-resourced computer centre with dial up, but dial up was out of action for a few weeks

• • 1 person active

Micha Kgasi High School

• 2 computer centres, but no Internet connection

• 1 teacher carried a computer home to dial up

• Discovered that the CD did not operate in the dial up machine and that stiffy disk drive also non-operational; could not send activities

• • 1 person stayed active

11


Based on the above, Gerald provided a list of reasons why learners dropped out of the course:• Lack of contact from the mentor.• No e-mail at school (modem out of order; no connectivity; telephone line dispute).• No or poor access to computers (live far from school; do not work at school; no

computers at school but at a community centre; computer room locked; no Internet access at school).

• Workload at school. • Did not know there would be a distance course (thought they were only going to a one

day course).• Lack of interested in modules on offer.• Personal problems (health, death in family).

Of the 29 teachers who attended the introductory course, nine (31%) never commenced the distance education programme, ten (34.5%) started but never completed the course, and ten (34.5%) persisted and completed the course. According to Gerald, this somewhat disappointing participation is a reflection of the poor school selection process.

RECOMMENDATIONS FOR SCHOOL SELECTION

Based on the above discussion, the following points are highlighted as issues to be considered, amongst others, when selecting schools for these types of projects:• Technical infrastructure, including existing networked and online computers, and e-mail

addresses and functional e-mail facilities. • On-site information technology support and back-up. • Support for the project from school administration and principal. To achieve this, school

administration and management should be involved in the development and initiation of the project as far as possible.

• Time allowed for educators to access computers and any other potential restrictions.

Stephen Marquard has, however, also pointed out that that the schools finally selected did all start with a better-than-average chance of success compared to a randomly selected school with computers. As he notes, there is merit in a pilot project of this nature in testing training approaches like this under such conditions, as it raises a series of potential implementation problems that need to be taken into account as such interventions go to scale. Thus, it is worth noting that problems such as those outlined above need to be expected and catered for in the design of future projects.

12


CHAPTER THREEThe Introductory Course


This section describes the introductory course that was conducted prior to the distance education component of the pilot project. Two of the trainers wrote brief reports on the introductory days they ran (in the Western Cape and Kwazulu Natal), and an observer also provided detailed reports on each of these sessions. There was no trainer’s report for the course conducted in the North West Province, but comments made during interviews with two of the attending learners were recorded, and a telephonic interview was conducted with the introductory course trainer. Nine feedback forms on the introductory course were completed and returned by learners, while other learners provided comments in e-mails. All of the above documents can be found in Appendix Four.

DESCRIPTION OF THE INTRODUCTORY COURSE

According to Schoolnet’s guidelines for the structure and organization of distance education courses, learners should attend an induction workshop where the following details of the process are explained:• Course objectives, concepts, ideas, and learning outcomes.• Clear indication of course deadlines and expectations, i.e. implications and

responsibilities of participating (learning contract).• Applied outcomes and assessment criteria.• Expectations regarding interaction.• Portfolio requirements.• Sufficient practical induction training with regard to use of course materials and

communication technologies.• Technical requirements.

The pilot introductory course took place at three venues. Details of these introductory courses are shown below.

Cape Town Kzn North West ProvinceVenue Cape Town

Teachers’ CentreMpophomeni

Computer CentreNgaka Maseko

Date 9 September 2 September 2 SeptemberTrainer John Fourie Pam Robertson Philemon KotsokoaneInitial number of learners 11 13 6

The purpose of the introductory course was to introduce the learners to the tools they needed in the distance education component, namely:• Word processing (writing a basic document and saving it)• E-mail (sending, replying to e-mail, attaching a document to e-mail)

13


• The forum and chat tools (to happen live on the Internet)• Copying and pasting text between the word processor and the e-mail

The learners were also introduced to the resource CD and were given information about the distance education component of the programme.

The introductory courses took place during a full day, designed according to the schedule in Appendix Five. Learners were provided with manuals for the course, as were the trainers/ facilitators. The aims and objectives (in terms of the outcomes and purpose) of the introductory course are made explicit in the manual, the outcomes being defined as follows:

• Switch on and start the computer • Hold and move the mouse to perform tasks • Create, save and print a file in Paint • Insert and navigate the whole CD with the course materials and

supporting resources • Use Help on the CD to obtain additional support and guidance • Switch between applications while using the CD • Create and save a reflective journal in the MS Word application • Send and receive e-mail to and from their student study groups and

mentors • Communicate with their student study groups using the forum and

chat facilities • Copy and paste text into their reflective journals • Shut down the computer

The outcomes (which clarify what learners should be able to do by the time the course is over) are relevant and worthwhile in that they provide learners with the basic skills they need to continue with the distance education modules. The major problem with these, however, is that they reflect an over-emphasis on learning technical skills at the expense of using expensive face-to-face interaction to introduce learners more thoroughly to the learning approach adopted in the modules and generate discussion between learners and educators about why teachers might use ICTs to support classroom practice at all.

INTRODUCTORY COURSE ISSUES

The comments and recommendations made by those involved in the introductory course, including trainers, observers and learners, are discussed below.

SCHEDULE AND OVERALL PROGRAMME

Many of the learners felt that a full day course was too long, mainly because they were not accustomed to looking at a computer screen for such an extended period. Some suggested doing the course over two or three days, for shorter periods. It was mentioned in particular, that learners became tired from having to do so much reading during the day. Furthermore, Pam Robertson felt that there was too much to absorb in a single day, believing that if the learners had all been completely new to computers, they would not have coped. Philemon Kotsokoane, who trained six teachers in the North West Province, supported this view, believing firmly that teachers who were beginners with computers would never have coped

14


with the content or pace required for the introductory course. Philemon felt that, for complete beginners, too much was crammed into the single day.Philemon also noted that, for his training session, he did not adhere to the schedule for the introductory course provided by Schoolnet, mainly because his learners were not new to computers. He therefore omitted several sections of the course, particularly the beginner elements that were too easy for his learners. He nonetheless conducted a full day of learning, concentrating on the more demanding sections of the course.

STIMULATION AND ENJOYMENT

Despite finding the day long, many of the learners thoroughly enjoyed the experience. In the feedback, learners described the course as ‘interesting’, ‘thought provoking’, ‘informative’, ‘fun’, and ‘enjoyable’. The section on learning to use the mouse (in Paint) was particularly pleasurable and provided an essential opportunity, early on, for learners to relax and interact with one another.

It is very significant that the learners also showed a great deal of appreciation for being given the opportunity to learn more about computers, and some commented on the effort made and patience shown by the trainers:

Thank you very much for your patience with us on Saturday. It is much appreciated.

Many also spoke about the anticipated future benefits of this learning opportunity and clearly looked forward to the distance education component of the programme. Philemon’s group in particular found the course interesting because they developed new ideas about what they could introduce into their own classes, and how this could be done with greater creativity. Therefore, the course was successful in creating enthusiasm amongst learners and some participants even commented on the positive impact that the acquired knowledge would have on contributions they could make in their working environments:

• I wish I could carry this computer home to go practise all that I have learnt today. I have really gained some new things.

• I enjoyed the course very much - I hope to continue and learn more about computers.

• I can’t wait for the next session so that I can master it.• I am very eager to start with the real stuff, it seems is going to be

interesting.• It will sharpen our methods of teaching and help us to go along with

change as it is a very important aspect in the new South Africa.

Unfortunately, having created enthusiasm and appreciation, the delay between the introductory course and the commencement of interaction between many learners and their mentors (as long as ten days), seemed to dampen this interest considerably.

15


LEARNERS AT DIFFERENT LEVELS

Problems can arise when training learners in a subject on which their level of knowledge varies considerably. Most of the learners clearly had at least some experience of computers and generally knew basic computer skills, like how to use the mouse. The few first-time users appeared to have greater difficulty in following the course, particularly with certain sections:

I would say that the course was a little bit difficult, but interesting especially to a first computer user like me. My biggest problem is with typing, it takes me ages to find letters from keyboard.

At the same time, no learners complained of getting bored as a result of covering topics about which they already felt knowledgeable. Many experienced users willingly assisted those who knew very little, thereby raising the level of collaboration and interaction taking place during the course:

Although I could do some of the sections there was still many areas where I benefited as I felt free to ask questions which I needed to be answered.

PROBLEM AREAS

Difficulties experienced by certain learners were manifested in the desire to have more opportunity to practice what had been learnt. This was particularly true of the final sections (chat and forum), where most learners became confused, and several complained that it was covered too quickly:

I do not feel very comfortable with the last section of the course on Saturday. The chatting and the forum.

Therefore, some students felt that more activities were needed, and others suggested running the introductory course over two days so that more time would be available for completing activities, thereby helping to create a deeper level of confidence in the tools.

• I enjoyed the whole thing only that I have to give myself more time to actually do it on my own. I think we still need more activities to get us properly used to distance learning.

• I feel that I would have had more confidence if the Introductory Course was held over two sessions.

Philemon Kotsokoane’s group were unable to access the chat or forum facilities and so had to leave out these sections. A few members of his group also experienced problems in finding certain items on the CD, but generally they did not find any of the other sections of the course problematic.

MODULE SELECTION

An important objective of the introductory course was to introduce learners to the three modules, so that they could decide which one they wished to follow. It was generally felt that insufficient attention was paid to providing details on these modules. Although the students were given an opportunity to review the materials for each of the modules on the CD, most seemed to base their decision on the short description provided in the introductory course material and on what the trainer said about each module. There seemed to be a general

16


perception that the Word Processing module was for beginners, and, because it made sense for questioning to come before information skills, Questioning Skills was considered intermediate and Information Skills advanced. Therefore, in some cases, modules were chosen on the basis of the level of computer knowledge of a learner.

OTHER COMMENTS

Detailed comments made about the training manual are highlighted in the recommendations section below. Overall, Janet commented that the design of manual lent itself to being closely followed by both trainers and learners, without telling learners exactly which buttons to press. In other words, it provided a good balance between supporting learners and encouraging them to find out for themselves. Janet therefore felt that the manual effectively created the expectation amongst learners of the need for independent learning habits.

RECOMMENDATIONS FOR THE INTRODUCTORY COURSE

As we did not have first hand experience of the course most of the recommendations below are based on issues that were raised, or suggestions made, by the various stakeholders who were involved (trainers, observers, learners). Their practicality would, of course, need to be tested against the realities of project implementation.• Try to cover basic e-mail in-house before the introductory course.• To ensure that the training centre is in full working order, the infrastructure should be

checked and operational before training commences.• To prevent having to experience start-up problems, the computers should be ready for

logging in when the learners arrive (switching off and on should be covered at a later stage in the course).

• Trainers should be careful about making assumptions about learners’ level of knowledge of computers, and should err on assuming that all learners are new to computers so that complete beginners do not get left behind.

• There should be increased focus on the following: - The CD itself, to improve familiarity with the materials;- The modules on offer and their purpose, so that learners are better equipped to choose

a module (a covering document summarizing each option would be useful);- How the course will proceed beyond the introduction, in order to improve transition

between the introductory course and the distance education component of the programme.

• It was suggested by some that the forum interaction should be dropped, because it caused confusion and impacted negatively on the learners’ confidence.

• ‘Click methods’ should only be introduced in context.• Learners should work in pairs for the section on getting to know the help resources.• Some felt that the course was too long and needed to contain less information.

The materials for introductory course (as they are appear on the CD) are discussed in more detail in Chapter Five. In terms of the print version of the introductory course materials, it is worth noting the recommendation that the trainer notes be on the facing page of the materials and not in a separate document with different page numbers, and should include more guidelines on regarding navigating the CD.

17


18


CHAPTER FOURMentor Support


In this section, the mentoring process is described and analysed. Since mentoring was conducted electronically, the main source of information on this aspect of the course was the e-mail discussions between the mentors, and between the mentors and learners. Other material used included data obtained from interviews with the learners, as well as with a number of other stakeholders, including the mentors themselves.

DESCRIPTION OF THE MENTORSHIP MODEL

Following the introductory course, teachers were divided into three groups according to the modules they had selected: Word Processing; Asking Questions; and Information Skills. Four mentors were selected to support these groups as follows:

Course Word Processing

Asking Questions

Information Skills

Information Skills

Tutor Gerald Roos Janet Thomson Sandy Zinn Trudi van WykNumber of learners 10 10 5 5

Each mentor had his or her own discussion list, so that, for example, one e-mail address could be used to send the same e-mail message to everyone in Gerald’s group. There was no general discussion list including all four mentors and all 30 learners because it was thought that this would generate too much e-mail.

Each mentor was expected to initiate contact with his or her group by first sending out an introductory message. This message introduced the learners to the mentor, welcomed them to the course, and then directed them to the relevant materials on the CD. The mentors also discussed timing issues, reiterated the need to submit assignments via their reflective journals, and encouraged them to write to the list and to the mentor, as well as to each other individually. In particular, the message encouraged the learners to write to the mentor when they experienced difficulties or had questions.

There was no training programme or manual for the mentors, but their anticipated role was defined in a document on the Schoolnet website. To the learners, the role of the mentor was described on the CD, as follows:

The mentor will not direct the activities - the course materials will do this. The mentor will negotiate deadlines and course schedules with you and your group. Other than that the mentor will play a pure supportive role. This will be in the form of comments, encouragement and advice by e-mail from time to time. Your collaborative group members will also react to some of your e-mail by passing on ideas and comments. The mentor has to assess your

19


contribution to the course and your successful achievement of the outcomes by monitoring interaction between the learners on the course and by reading your reflective journal and other documents that you may be asked to submit as part of a portfolio of work. Most importantly, you should see the mentor as person who can help you to succeed. The mentor should be assisting you with problems if neither you nor your group are able to solve them satisfactorily.

Therefore, the mentors were expected to support their learners as they work through the materials on the CD, assisting where problems arose and encouraging collaborative learning within the group. The learners were expected to work through the materials independently and to submit assignments to their group via a reflective journal.

The reflective journal formed part of each learner’s portfolio. The reflective journal was to be used to document the cycle of thoughts and experiences that is an essential element of the reflective learning process, and to develop collaboration amongst learners based on ongoing and open communication with the group. Therefore, learners were encouraged to discuss how they were finding the work in the module, what they had learnt and how they had applied (or wished to apply) it in their classroom. They were also told to share frustrations and problems, as well as positive experiences. The learners were encouraged to develop the journal in a word processing document, to which they should add their reflections on a daily basis.

The portfolio also consisted of completed assignments. Submission of these assignments allowed the mentors to gauge learners’ progress in the module and provide feedback. Therefore, the final reflective journal was expected to provide material on the basis of which the learners’ participation and progress with the course could be evaluated. Assessment could take place continuously as well as on completion of the course.

MENTOR/LEARNER INTERACTION

In order to extract the key issues that arose during the mentorship process and to review the effectiveness of the approach, the e-mail discussions were analysed quantitatively in order to establish the extent to which administrative, educational or personal issues were discussed. The content of these electronic conversations is documented on a qualitative basis in order to reveal how the process progressed and what issues arose. Therefore, the e-mail interaction provided an opportunity to assess the quality of mentors, as well as the response of learners.

INTERACTION BETWEEN MENTORS

The table below provides a quantitative analysis of e-mail messages between the mentors.

Messages Between… Number of Administrative Messages

More than two mentors 4Gerald and Sandy 2Janet and Sandy 9Janet and Trudi 34

20


All the messages between the mentors were administrative; there were no personal messages although some administrative messages included personal comments.

The discussions between mentors primarily centred on efforts to understand and resolve problems experienced in the mentoring process. A major problem was that e-mails frequently were not received. Reasons for this related to issues with the group e-mail addresses, use of incorrect addresses, or technical/hardware problems (such as the theft of computers from one of the schools and incompatibility problems). Failure to receive messages caused confusion and uncertainty, and in some instances, tensions arose between mentors. The resending and resultant duplication of some mails also caused confusion.

In the case of Trudi’s group, confusion arose because her learners initially sent their mails to Janet. The learners seemed to be confused about the use of the group address, and clearly felt more secure about sending messages to Janet, with whom they had interacted during the introductory course.

A number of administrative messages with an instructional component were relayed mainly from Janet to Trudi and Sandy, since these two mentors were not involved in the conceptualization of the course, and therefore had less understanding of their expected role. This guidance was also necessary as there was no mentor manual or training, although Janet suggested that they refer to information provided on the Schoolnet website. Janet also provided information and guidance herself on the recommended mentoring approach. She sent Trudi and Sandy the introductory message that she had sent to her group, in order to give them an idea of what information this message should contain. Janet also had to provide clarity on the composition and use of Sandy and Trudi’s mailing lists.

These differing levels of involvement in conceptualization of the course resulted in a variance in mentoring styles in the pilot. In particular, issues arose between Janet and Trudi regarding these variations. Trudi’s approach to mentoring varied from the preferred Schoolnet style in that she became actively involved in providing information and assigning tasks to learners. Schoolnet, by contrast, views a mentor as a reflective practitioner who provides support and assists in interpretation and analysis (where the materials provide most of the information and challenges). Therefore, Janet pointed out to Trudi:

I suspect you are thinking that you have to do a lot more work than you really need to. Rather just talk about the activities in the materials rather than give them ones of your own.

The above issue again relates to the lack of clarity on Trudi and Sandy’s parts on their expected roles. According to Trudi, she was not briefed on what to do or on the expected duration of the process. Clearly, Trudi and Sandy should have been better informed about how to mentor in an electronic environment, and they should have had an opportunity to work through the CD materials before mentoring started.

The mentors also discussed the lack of activity and seemingly poor progress on the project. This resulted from the delay between the introductory day and commencement of communication within the groups (in some cases as long as ten days), which caused confusion and made it difficult for the project to gather and maintain momentum.

21


With the exception of the interaction between Janet and Trudi, on the whole, there was little interaction between the mentors. This begs the question of whether there would have been value in encouraging sharing of interaction between the mentors by providing a group mentoring mailing list.

INTERACTION BETWEEN MENTORS AND LEARNERS

A quantitative review of the e-mail discussions between mentors and learners is provided below.4 The messages were categorized as administrative, educational, or personal. In some cases only one or two mails were received from a learner. This was when learners went no further than submitting a registration form and/or reflective journal introducing themselves to their groups.

ADMINISTRATIVE EDUCATIONAL PERSONAL TOTALGERALD: Messages to:>1 learner 8 13 3 24Learner 1 0 7 1 8Learner 2 4 7 1 12Learner 3 6 11 1 18Learner 4 0 0 1 1Learner 5 2 0 0 2

Gerald Total 20 38 7 6531% 58% 11% 39%

JANET: Messages to: >1 learner 4 0 1 5Learner 1 6 2 1 9Learner 2 5 12 1 18Learner 3 2 6 1 9Learner 4 0 2 0 2Learner 5 0 2 0 2Learner 6 0 0 1 1

Janet Total 17 24 5 4637% 52% 11% 28%

SANDY: Messages to: >1 learner 4 0 1 5Learner 1 4 7 3 14Learner 2 2 4 2 8Learner 3 1 0 0 1Learner 4 1 0 1 2Learner 5 0 0 1 1

Sandy Total 12 11 8 3139% 35% 26% 19%

TRUDI: Messages to: >1 learner 4 1 0 5Learner 1 8 0 1 9Learner 2 0 4 3 7Learner 3 1 0 0 1Learner 4 2 0 0 2Learner 5 0 0 1 1

4 This analysis reflects all e-mails forwarded to the evaluators by the project team. It may exclude further mail communication that took place, but copies of which were either not saved or forwarded to the evaluators.

22


ADMINISTRATIVE EDUCATIONAL PERSONAL TOTAL

Trudi Total 15 5 5 2560% 20% 20% 15%

OVERALL TOTAL64 78 25 167

38% 47% 15%

Administrative MessagesAs with communication between the mentors, many discussions between the mentors and learners centred on efforts to resolve administrative and technical problems. Again, these related to mails not being received, for various reasons, with the result that (contrary to the intended design of the course) communication between mentors and learners was frequently irregular:

• This message is being sent in order to find out if we are all still around. It seems like the mailing lists that we set up are not working...

• I am a bit worried that you are not receiving my mail - I will try to find out what the problem is and hopefully get back to you again tomorrow.

• It seems as though we may be having some problems with this mailing list. In order to test that it is working please can you send a message to the list above by clicking on the reply button and just saying that you have received it.

• I am worried about the silence.... are you still there? Do you have problems? Did you receive my previous messages? Please communicate.

In one case, the mentor was not using the correct group address, and therefore learners failed to receive prompt feedback on work submitted, thus becoming frustrated and discouraged. Communication was also slowed by the occurrence of holidays, since teachers were generally dependent on using computers at their schools.

As with the mentor interaction, because mails were not received they frequently had to be re-sent, and this led to duplication and further confusion:

When you wrote to me earlier this week I sent you a response immediately. I will send this response again in a separate mail.

Many of the technical issues were difficult to solve due to the remoteness of the schools and lack of technical infrastructure and support. A few problems arose as a result of virus infections on computers and files, while some were related to incompatibility between programmes:

I forgot that your computers had an old copy of Word. I have attached a file that will be able to be read by your computers.

Progress on the project was initially prompted by means of a ‘welcoming letter’ from the mentors to their group members. Frequently mentors had to re-send these messages, and, sometimes, additional messages of encouragement were sent in order to get learners started:

I thought I would just write to enquire how you are getting on with the Word Processing distance course. I have heard nothing from you since the course started and I received your registration form. Is there anything I can do to help?

23


Educational MessagesA large proportion of the educational messages involved submission of completed assignments by learners. Mentors then relayed comments on these assignments and generally encouraged the learners to keep progressing:

• What can I say? The perfect 10. You could not present a better-looking examination /test paper.

• Good luck with the worksheet. I think it looks good. Once you have completed that you can go onto activity 4.

According to Schoolnet, encouragement and acknowledgement is a vital component of mentoring. Mentors are expected to facilitate ongoing group interaction, and encourage and counsel when it is required. Encouragement from mentors was frequent and incorporated essential motivational techniques to promote participation from learners:

• Some of you have been sending me your activities very regularly and that is wonderful but we do not seem to be writing to the whole group via the mailing list.

• You really have been working - I am very impressed - well done and keep them coming.

• This is great! Thanks so much for proceeding with the course.

In line with this encouragement, mentors also volunteered their support on an ongoing basis, making it clear that they were always available to assist. However this support was carefully balanced with the need for learners to develop self-help/learning skills, which is essential as the mentor should be a guide, not a provider of knowledge:

• Please contact me if you experience any difficulties with the course.• By the way, you did not have to copy the document into the e-mail.

You could just attach it to the e-mail. If you have forgotten how to do that, you could consult the Help on the CD and look under Computer Skills, Pegasus Mail, Attachments.

Learners also frequently asked questions of the mentors on how to complete certain aspects of assignments or regarding other issues. Questions sometimes arose because skills learnt on the introductory course were forgotten (such as how to send an attachment). Mentors sometimes directed questions to learners, thereby providing a strategic balance of challenge and support:

I’m now busy with activity 4. I am a little confused. Must I type my whole examination paper or can you give me a little guidance. Enclosed find a sample of how far I am. Let me know if I’m on the right track.

Generally, learners were satisfied with the level of feedback, amount of encouragement, general response time, and frequency of mails received. The encouragement and assistance provided by mentors was generally appreciated by learners, many of whom openly expressed enjoyment in having an opportunity for interpersonal communication, networking, and learning. Regular mailing list interaction helped to develop a sense of community between the learners. There were even instances where peer support took place (often regarding ICT skills).

• …we enjoyed what we did up to so far, and we are looking forward for having more fun with this programme.

• Keep writing - I’m beginning to enjoy your company!• I am so pleased to hear encouraging words from you, it shows that I

24


am important in this communicating process.

Personal MessagesIn an effort to encourage the use of interpersonal skills to build mentor/learner relationships, learners were encouraged to send an introductory letter introducing themselves to the group (this was carried out at the introductory course). These were generally quite personal in nature, describing issues of work and home life, as well as interests. These introductory messages formed the first activity of the reflective journal, but, in reality, few reflective journals were developed beyond this point. Many learners submitted activities to their mentors via attachments to e-mails (i.e. they did not copy and paste into their reflective journals). Furthermore, very few learners spontaneously discussed their thoughts on the course or activities, or on their experiences of introducing activities into the class. Therefore in practice, very little ‘reflection on practice’ seems to have taken place. Failure in this area seems to support Stephen Marquard’s view that expecting teachers to become reflective requires a major mind shift on their part.

There were not many other personal mails, although many messages included personal comments by learners.

STRENGTHS AND WEAKNESSES OF MENTORING

Notwithstanding problems experienced during implementation, the pilot project has provided sufficient evidence to suggest that e-mail mentoring provides workable pedagogical models of supporting learners, even those with limited computer experience. In terms of its strengths, the mentoring model, as well as the peer support, proved to be a motivating factor in that it encouraged learners to persist. Immediate feedback and a friendly, encouraging tone helped to achieve good mentoring, and was perceived by learners as important factors of effective mentoring. Some learners valued the meaningful, friendly relationship they established with their mentor and other learners.

A further positive element of the tele-mentoring used in this project was the extent of flexibility in learning it afforded. Learners could proceed at their own pace, they could determine their own style of learning, explore the resources, use the programmes and do the activities as and when they chose to, also selecting when and what they wished to practice. Learning on one’s own is thought to help retention of knowledge and to create a feeling of responsibility for one’s learning. This advantage was expressed by a number of learners when they were interviewed. This enhances the adult style of learning that is one of the key principles of Schoolnet’s educator development framework.

A key weakness of the mentor system relates to the logistical problems of e-mail interaction (mainly technical). Furthermore, confusion is prevalent, either when e-mails are not received or when the tone or information are misunderstood. Weaknesses in this area probably contributed to the sometimes poor response rate amongst the groups, although the dropout rate once learners had commenced with the course work was low (most of the ‘dropouts’ were people that never started the course, as opposed to people that fell out along the way).

A further weakness of the mentor system was the need to keep up the disciplines of e-mail conversation. Stephen Marquard points out that consideration needs to be given to how long

25


it takes between learning to use e-mail (technically) and when people get really comfortable communicating via e-mail This mentor system seeks to fast track this therefore, there is a need to continuously evaluate how it is working. Stephen also points out that it is difficult for inexperienced users to read emotional cues in e-mail messages. Sandy Zinn recognized this problem, suggesting that because communication in an electronic environment was so new to the teachers, they should perhaps have been brought together on more than one occasion.

Alixe Lowenherz also noted that details of the participants were minimal and received too late. She felt that the mentors would have benefited from a more systematic analysis of participants, best obtained by someone actually visiting the schools where teachers were expected to carryout the majority of the work on the course.

The problems of differing mentoring styles and lack of understanding regarding expected roles of mentors is already being tackled in the mentor-training course, which is currently being run to support the next phase of teacher training. The course is designed in the same way as the modules, and, in Gerald Roos’s opinion, is proving very successful.

According to Gerald, mentors play an important role in the total support package of this approach to learning. He does not believe that the programme is at risk if a mentor turns out to be less than perfect. Although it is significant, the peer support and contextualized activities can be sufficient to make the learning process effective. However, it is still clear that good mentoring can significantly enhance the quality of the programme.

RECOMMENDATIONS IN TERMS OF THE MENTORING PROCESS

Based on the above discussion, the following recommendations to improve the mentoring process are made:• Mentors clearly need to receive specific training on their anticipated role and what is

expected of them. This would make the mentoring model more uniform amongst the mentors and would bring their approaches more in line with Schoolnet’s requirements.

• Mentors should receive adequate details on who their learners are, particularly in terms of learners’ experience with computers and the particulars of their working environments. This information should preferably be gathered via visits to the schools involved.

• Mentors should be given adequate time to review the course material and activities before the mentoring process commences.

• Mentoring would be significantly improved, and confusion and tensions reduced, if technical issues were resolved (this relates to school selection criteria). E-mail addresses should also be checked and verified before the course commences.

• Mentors should make contact with their learners immediately following the introductory course in order to keep up the momentum of the course, and to encourage learners’ motivation to take part in the distance education component of the programme.

• Interaction between the mentors should be encouraged by including a mentor-group e-mail address, thereby facilitating mutual support and learning amongst the mentors.

• There should be more direct and persistent encouragement from mentors for learners to be reflective about the course and activities. If learners do not comment on processes followed or experiences had, mentors should ask for this information directly.

26


The biggest problem with the mentoring system is, in the view of the evaluators, not intrinsic to the mentoring approach itself, but rather linked to the central role that it plays within the overall learning process. From this perspective, the e-mail mentoring is still a potentially serious ‘killer factor’, in that its successful implementation is so central to the overall success of the training intervention. This is reflected in the strong voice of mentors in the materials, the importance of mentors in encouraging and – as comments above illustrate – formatively assessing progress of learners through various tasks and activities. Where communication between mentors and learners was weakest (for technical or other reasons), motivation tended to be lowest and non-participation or frustration highest. For these reasons, we believe that critical to the success of the mentoring system will be to build up more ‘safety nets’ around it, thus ensuring that, where it is not working successfully, the impact of this is minimized. There are four areas in which this might be achieved:• Ensure that school selection criteria minimize occurrence of technical problems.• Re-think face-to-face contact with learners to ensure that a greater focus is placed on

preparing learners to participate in the distance education component of the programme.• Establish tracking mechanisms to assess learner participation in online mentoring

discussion. Where problems arise, ensure that alternative communication strategies – not involving e-mail – can be initiated to make contact with learners and find out reasons for their non-participation.

• Strengthen the capability of the materials themselves to be used as stand-alone resources, not as the primary strategy but as a fallback option if mentors do not perform their functions adequately.

27


CHAPTER FIVEThe Learning Process


The aim of this section is to evaluate the extent to which the learning process was effective in practice. It was Schoolnet’s intention that the learning would be experiential and reflective, with peer support and collaboration adding to the learning experience. Furthermore, it was hoped that mentor facilitation would support and encourage both the learning process and the group interaction.

Gerald noted that the learning process used in the pilot had strong support from the educators who experienced it. It was even supported by those who did not fully experience it, but saw it operate with their colleagues.

Broadly speaking the learning process included three key elements:• The learners doing the contextual activities;• The community of learners that enhance that learning; and, • The mentor that supports and facilitates that process.Information on the learning process was gathered primarily from notes written by Gerald Roos, based on his interviews with ten participant learners.

ELEMENTS OF THE LEARNING PROCESS

The effectiveness of the learning process is dependent on smooth functioning of the triangle of three elements described above, and therefore an understanding of the success of the learning process begins with analysis of these three elements: the contextual activities; community of learners or collaborative group’s support; and the mentor’s facilitation.

CONTEXTUAL ACTIVITIES

The importance of the active component of the course is evident from the extensive use of activities in each of the three modules (Word Processor = 9 activities, Questioning Skills = 15 activities, and Information Skills = 14 activities). The activities clearly provided learning opportunities. Some learners confirmed that learning took place best during these activities, and that the activities were meaningful because they led learners to think about how to design activities for their own learners. For example, as a result of the course, one learner has started thinking about designing a snakes-and-ladders game and using Scrabble with the word processor.

The activities clearly create a sense of independence and transfer the responsibility of learning to the learners themselves. In fact, many learners stated that they recognized that they themselves held the responsibility for their learning. Some learners acknowledged that this independent style was better in the long term than having instructors (who were not available for support after the course) telling learners what buttons to press.

28


This independence of learning was created through the design of the course, i.e. use of modules on the CD and the resources available on the CD. Learners were able to explore and help themselves, and developed a sense of independence in finding answers for themselves, before asking questions. In this sense, there was great appreciation for the ‘backup systems’ offered by the CD.

This independent learning process also created a degree of flexibility in the process. Certain learners felt that, as long as they were motivated, there were advantages in the self-controlled and self-paced nature of the course, and they appreciated that there was something new to learn about and do every time they worked on the CD. Therefore, in offering flexibility, learners felt less pressured on this course than they would have on a workshop-based course. This in turn gave them the opportunity to explore and to gain confidence in the work being followed. One learner realized that she needs to discipline herself to work regularly and to ensure that she understands what she is doing.

On the whole, many learners were very much in support of the course being CD-based and mentor-assisted rather than being workshop, lecture-style, face-to-face learning. Thus, generally, learners expressed enthusiasm for a mentor-support model, with content and activities on a CD and preferred this to a workshop model that excluded ongoing support. Other learners commented that the combination of materials on CD and a mentor had a significant influence on their learning. For many, the lack of support following a facilitator-led workshop led learners to support the distance-based approach. However, some felt that the course could have been enhanced had there been a number of tutorial-type visits by someone who could observe learners applying new skills in the classroom situation (i.e. a combination of face-to-face and distance learning beyond the introductory course). Such a combination would be particularly relevant where a large number of learners on the programme were new to computers.

COLLABORATIVE GROUP SUPPORT

As already mentioned, a key element of the learning process stems from group collaboration and support. According to both Gerald and Janet, the aim of the process was to build a community of learners, with learning taking place primarily through this community, not via the materials. The problem with trying to create a community via e-mail is the problem of e-mail itself. Technical problems, as well as lack of confidence with e-mail, tended to inhibit interaction. Trudi also noted that effective e-mail use is dependent on a good infrastructure, sufficient time, and regular access over a relatively long period (for example, a year). There is also some doubt about the extent to which the community of learners persists beyond the course (which this pilot project has not been able to test).

For some learners, the collaborative experience was good and clearly greatly enhanced the learning process. Many experienced a definite sense of community, and communicated individually with other members of the group, often getting to know them quite well. One learner found the fact that she has been able to collaborate with other members of the course on e-mail very useful. For instance, she realized that they had not all interpreted Activity 5 of the Word Processing module in the same way, and were able to solve the problem and support each other by e-mail. Consequently, she feels that this contact enhances learning because the way in which people think and approach problems differs. One learner felt that

29


the CD gives them enough guidance, but this has been enhanced by collaborative contact. Thus, where the ‘e-mail’ problem mentioned above did not dominate, the mailing list proved itself to be an effective tool for creating this sense of community.

Some learners received assistance on the course via e-mail where they otherwise could not have proceeded. One learner described the communication on the course as ‘fantastic’ and was not bothered that he did not have face-to-face contact with the peers in his group. Instead, he felt confident that he would meet them in the near future because he felt that he had made friends. Another learner was also not bothered by the fact that he was communicating with people whom he had not seen because he felt that the process was about much more than mere e-mail messages, it was about the knowledge being gained. The learners’ confidence was also clearly enhanced by the community of learning created by the process, particularly where it involved peer support on site. Different levels of skills in the group created the opportunity for more advanced learners to help beginners and to make additional contributions.

But not all learners’ experiences were positive, or at least what was intended. For some, the sense of community only existed amongst those who were on site, and in some cases working on the same module. For example, there were seven teachers from Mpophomeni who registered for the Questioning Skills module, and these teachers worked closely together. Therefore, according to Gerald, it appears as if the online community is better developed if the collaborative group is composed of teachers from different schools. If they are all at the same place they tend to support each other to the detriment of the other members of the group who are in other parts of the country. Certain learners supported this, claiming that they found more value in a diverse community of learners than in having ten educators from the same school taking the same module. At another school, there were four learners who were fairly actively involved in the pilot and who worked together and supported each other. They had little contact with other learners on the module, but those other learners found their submissions to the mailing list very valuable and were able to use their ideas.

For yet other learners, there was absolutely no collaborative learning experience. There was no sense of community on the module and no communication between the learners. In some cases, this occurred even where colleagues at the same school were taking the same module. The lack of community amongst some learners was caused by low involvement and interaction from members of a group. This is potentially very problematic, particularly if a learner is isolated and lacking in confidence.

One aspect of the collaborative process worth mentioning relates to the issue of communication taking place via the written word, in most cases not in learners’ home languages. Generally, learners felt that having to write was not a problem, and pointed out that it gave them good word processing practice. One learner indicated that it was possible to get to know people by the style in which they wrote. Another writer claimed that he did not have a problem writing in English, but stated that it is more demanding. He noted that e-mail does allow personalities to emerge, and he thinks that writing in English is good because it is developmental for all concerned.

30


MENTOR’S FACILITATION

As was already mentioned, the mentor-based approach was generally thought to have greater value than a face-to-face course with a facilitator in the room. It is clear that the materials were intended to be self-explanatory, but the mentor is still vital. As one learner put it, the materials are like the teacher, but in some senses a poor teacher who is unable to provide feedback. It is in this sense that mentor support is essential. Similarly, one learner identified a significant learning process taking place from the materials, but he recognized that he could learn more by asking his mentor, as well as his group.

The problem with this approach is that the learning process is dependent to a large extent on the effectiveness of the mentor. One learner, for example, felt that she received poor feedback from her mentor since most of here submissions simply received a ‘thank you for sending the reflective journal’ and no valuable feedback. Similarly, when the mentoring process is inadequate, learners are also dependent on the feedback and input they receive from the rest of their group (as discussed below) - some learners had very poor interaction from their groups, as well as being dissatisfied with their mentors.

However, those who had good experiences were strongly in favour of the mentor-supported process with interaction from their groups.

OTHER ASPECTS OF THE LEARNING PROCESS

Various other elements are important to the effectiveness of the learning process.

INFLUENCE ON PRACTICE

A critical aspect of the learning processes is the extent to which the learning experienced will influence future practice. In the pilot project, it was intended that the combination of course materials, mentorship, and collaboration with other learners would influence the learner in developing strategies for improved classroom practice. This aim was developed further by the experiential nature of the course and the fact that teachers were encouraged to seek context in their current practice and forthcoming lessons and tasks.

It is almost impossible to assess the impact of the pilot from this perspective, as there were no opportunities for evaluators to go into classrooms before and after the project to assess shifts in teaching practice. Hence, we have had to rely entirely on statements by participating teachers, which provide only a limited yardstick for measuring impact on practice. Nevertheless, several teachers claimed that the programme had had a positive influence on their classroom practice, one of the main objectives of the materials. According to Gerald, the educators were all able to relate how they had adapted, or planned to adapt, their style of teaching, based largely on the approach of the learning materials on the CD. Some claimed that they had even used the CD with their learners, who had discovered and made use of the help resources.

Most of the ‘Word Processor’ group improved the scope of their work with the word processor both administratively and in integration with the curriculum. One learner realized that computer courses for learners could be improved. Previously these courses were, for

31


example, just MS Word courses, but the educator indicated her intention to develop practices that involved activities and that required use of several applications for genuine needs.

Another learner indicated that the course had influenced her view of how to teach ICT skills. She realized that it is not necessary to give explicit instructions, but that the approach of the CD (whereby process and activities are mapped out) causes unnecessary teacher talk to be replaced by more facilitative action, as the learners work more independently with resources. Similarly, others claimed that the course approach had affected the way in which they teach, since they were making their lessons more activity-based, with the ICT skills being taught ‘just-in-time’. These teachers, in particular, used the ideas in Activity 7 of the Word Processing module to guide them. They had encouraged learners to use the computer as a tool and to think of and use teachers and staff members as support. Another learner found the course very practical and effective in introducing him to using the word processor as an educator. For example, he described how he types examination questions and papers, keeps a ‘data-base’ and types CVs for learners (and stores this as a template). This course has also given him the courage to explore, and he is now using Excel to generate his mark sheets.

The Questioning Skills group were more aware of the need to build questioning strategies. One learner had become more conscious of the role of questions, and had started planning questions to a greater extent and structuring his lessons. Previously, he did not plan questions and consequently they did not tend to encourage thought.

The Information Skills educators gained skills in finding and evaluating information. One educator felt that the course has influenced the way in which she worked with information on a website. When interviewed, she gave an example of a website that she is designing with her business class. She has made the class aware of the importance of the aim of the site and the credibility of the information they put on it. Another teacher, who also took the Information Skills module, has developed a tendency to encourage learners to be independent when seeking information. She no longer provides a lot of the information to her learners herself, but now directs them to information sources.

However, for some teachers the issue of influence on practice is less important because they are not practising teachers and therefore their circumstances are not conducive to experiential and reflective learning. This again highlights the importance of effective school and learner selection.

The value of continuing to evaluate the extent to which educators use the skills acquired on an ongoing basis should also be mentioned, since the tendency to revert to old methods or to forget what has been learned may become a factor over time, particularly when the school environment itself is not changing to accommodate new teaching styles and approaches. In turn, this raises the importance of integrating strategies to transform the entire school environment into interventions of this nature, rather than simply the practices of individual teachers. Introducing ICTs into schools has a wide range of potential implications, the majority of which affect the school as an organization not just the teachers as individuals. In next phases of design, it will be important to build in strategies (perhaps by adding new modules) that focus on the school as a whole.

Nonetheless, in terms of ongoing benefits and long term influence, whereas some had forgotten what was learnt during the introductory course (due to the ten day delay before the start of the distance learning programme), they felt that what they were learning during the

32


distance education component was more effectively retained because of the ongoing nature of the course and support from the mentor. One teacher was very grateful for this initiative, and hoped that the few who benefited would realize their moral obligation to help others in the vicinity with the same materials.

MOTIVATION

The issue of motivation was significant for this pilot because of the independent nature of the learning process. Learners had to have sufficient interest in the course and what they were learning in order to work through the materials on their own and to participate in the interaction.

Some learners were clearly motivated by the fact that they were increasing their knowledge and developing, through becoming more computer literate. They were learning something new nearly every day, and, furthermore, what they were learning had relevance to their working context. Some were motivated by a particular interest in the module they had chosen. For example, in the Word Processing module, learners were enthused because the course had made administrative activities easier, and therefore made them more efficient in their work. Another educator was motivated by her own interest in computers and their possibilities, and this was enhanced when the Word Processor module revealed more of the value of word processing. Therefore, for some learners, the opportunity to learn particular skills was evidently significant, as well as the flexibility to choose a module that was personally relevant and important.

Some learners were also encouraged by the support from their mentor and from fellow learners. Therefore, they were generally motivated by the e-mail communication and the friendships that have developed as a result, and were often eager to hear from others in their group. The practical element of the course was also clearly motivational, as was the flexible nature of progressing on the course.

Several learners indicated in interviews with Gerald that certificates help motivation (although, no one mentioned it spontaneously). However, it was noted that many educators tend to value paper more than things that are being said. One learner indicated that most teachers attend a course for the sake of the certificate, although this particular learner maintained that he valued the quality of a learning programme more than a mere certificate. It was, however, also noted that the prospect of being able to advance in their careers through having a better qualification was also inspirational. However, while some teachers were adamant that they were not interested in certificates, others were described as simply being ‘lazy’ (especially those who failed to continue with the course), and, for them, certification may have made a difference to their levels of motivation and persistence on the course.

Clearly therefore, for some, motivation was an issue. One learner did not feel at all motivated to work on the course. When asked what would help him to be more motivated, he suggested that a reward such as a certificate would help him, as would a face-to-face tutor (since he was still struggling with the keyboard and basic functions, such as ‘insert’ and ‘backspace’). For this learner, his lack of motivation may have been influenced by his need for more assistance with basic PC literacy skills, as well as a number of technical issues that he had experienced. These factors emphasize the extent to which this course was not effectively aimed at beginner computer users, since those without computer experience clearly had greater

33


difficulty with modules. Also highlighted was that a good, stable infrastructure, which provides the correct software, storage space and reliable e-mail and connectivity, is needed to maintain interest in the programme.

SUPPORT

Support while working through the course materials came in several forms for learners, some of whom made use of all these support options, while others relied primarily on one or two. These included the following:• Mentor• Face-to-face tutors• Peers • CD (Help)

Many learners felt well supported by their mentors, and some used their mentor as their primary source of support. This support was initiated through mentor guidance on the CD and activities, as well as through their ongoing encouragement. Learners expressed satisfaction with the fact that the mentors’ support was ongoing. Some learners were fortunate enough to have on-site support from a facilitator (Pam Robertson). According to Gerald, the mentor is not a pivotal figure in providing support, although this is not to say that the mentor need not be supportive. The evidence of this pilot indicates that the more supportive the mentor is, the better motivated the learners seem to be.

Support was also forthcoming for several learners from their peers. As was already mentioned, in some cases this was particularly true if there was more than one teacher on a module at a school. However, those who did not have any group interaction clearly missed out on this source of support. For some, peer support was a problem because it necessitated meeting at a computer centre, and this seldom happened.

Several learners were aware of and satisfied with the support offered by the CD itself, namely the Help facility, finding it suitably informative and useful, but acknowledging that it required more initiative than simply asking a facilitator or peers for help. Some felt they needed time to get used to the Help, mainly because of certain logic and language issues. Problems were experienced with some of the vocabulary used. For example, when looking for help on a specific skill, it was hard for some to categorize the issue and therefore, to know where to look (for example, terms like ‘rows’ and ‘columns’ were confusing). This issues indicates the value of a ‘Search’ CD function, which was not included on the CD. Similarly, when told during an interview that there was a glossary, one learner was surprised, but noted that the term ‘glossary’ in itself could be confusing.

Some learners, who were aware of the Help facility, had not used it, tending to rely instead on other forms of support. Problems experienced with Help are probably particularly relevant to learners who had never used a computer before, and these individuals would clearly benefit from more one-on-one facilitation. Through such facilitation, they could be shown simple things, like how to work on the keyboard and how to solve minor problems that stand in the way of their completing the activities. Therefore, such one-on-one facilitation would clearly help to develop the confidence of first-time computer users.

34


For some teachers, the schools were not very supportive of their learning. For example, the computer room was often locked, making access to the room, and therefore flexibility in the programme, a major issue.

CONCLUSION AND RECOMMENDATIONS

The effectiveness of the learning process is primarily based on three elements, namely, learners doing the contextual activities; the community of learners that enhance that learning; and, the mentor that supports and facilitates that process. There was a large number of activities in each module that encouraged independent and flexible learning. Teachers were also encouraged to take these activities into the classroom, thereby putting what they were learning into context. For many learners there was a strong sense of community and considerable interaction within the group. However, some learners only interacted with other educators at the same school doing the same module, and still other learners experienced absolutely no collaborative learning or sense of community. Satisfaction with the effectiveness of support offered by mentors varied somewhat, depending on the amount and quality of feedback given by the mentor.

According to Gerald, this three-tiered process means that the risks are evenly distributed. He noted that good learning took place with minimal community input (the activities should be contextual and reflective), whereas a strong community requires less mentor input. In his opinion, a weak and inactive mentor and an inactive group did not prevented some of the learners in the pilot from getting on with the course and gaining benefit, making the activities and contextualization of the materials the pillar in the process.

A further element of the learning process included the extent to which what was learned influenced practice. Many of the learners who were interviewed described various ways in which they planned to use what they had learned in their working environments. Motivation to work on the course was driven by a number of elements, namely, the opportunity to extend knowledge and to advance professionally, and support from mentors and peers. Support was also significant and was offered by a number of sources, namely mentors, on-site facilitators, peers and the CD’s Help facility.

Given an understanding of the overall learning process and the different elements that contribute to its effectiveness, it is worth mentioning again the level of learner at which the programme is aimed. Although the intention was to direct the programme at computer novices, very few of the learners on the pilot were actually beginners. With more beginners involved it is likely that many aspects of the learning process may not be effective using the current format of the course. This concern is supported by several learners’ belief that the programme was good for educators who had some basic IT skills.

Recommendations for improving the effectiveness of the learning process include the following: • A combination of distance and one-on-one learning should be considered if learners on

the project are completely new to computers. Such facilitation will clearly help to improve the levels of motivation of learners, and will also assist new users in learning about other support facilities available to them, such as the Help facility, are used.

35


• If learners on the project are new to computers it is also essential that they are given a longer period during which to familiarize themselves with how to use e-mail and how to communicate effectively via e-mail.

• On the project, it is suggested that fewer learners from one school participate (or at least do the same module). The involvement of a range of learners from a range of different schools will help to create a greater sense of community online. However, if the learners are new to computers, the practical difficulties of this approach should be considered since many new users may be highly dependent on having on-site peer support.

• The importance of the school selection process is again highlighted, since teachers not actually teaching at the time of the project should probably not be involved in the programme.

• There should be ongoing evaluation of the extent to which educators use the skills acquired on an ongoing basis, since the tendency to revert to old methods or to forget what has been learnt may be a factor, particularly in environments where the school as a whole is not supportive of changing teaching practices.

• Certification may be an effective measure to motivate individuals to continue working on the course.

• A ‘Search CD’ facility should be included, particularly in order to enhance the usefulness of the Help facility.

36


CHAPTER SIXThe Course Materials


The objective of this section is to evaluate the course materials (namely the resource CD), looking mainly at how effectively it catered to the role of the learning and mentoring processes. The analysis is based on first hand exposure to the CD, as well as on comments made by various stakeholders (introductory course trainers, mentors and learners), during interviews or in evaluation documents.

MATERIALS DEVELOPMENT

DESCRIPTION OF THE DEVELOPMENT PROCESS

The development of the materials was initiated at a four-day session held in Pretoria in July 2000 (the agenda for this meeting is in Appendix Six). During this meeting, the design of the course was discussed, including collaborative development of the outcomes, and writing of the materials commenced. Five individuals were involved in different aspects of the materials development, as follows:• Cathy Zandee – Introduction to the course, and introductory course materials.• Gerald Roos – Word Processing module.• Janet Thomson – Questioning Skills module.• Trudi van Wyk – Information Skills module.• Alixe Lowenherz – editor/language monitor, especially for second language writers

(Alixe was, however, largely unable to perform this function as deadlines were ultimately too tight to allow her time to review the materials).

Gerald Roos was responsible for general coordination of the development process and overall editing of the materials. Following the Pretoria session, materials development continued. Having completed a draft of a section, developers then circulated copies amongst the group for comments.

PROBLEMS AND ANALYSIS

Various problems surfaced during the materials development process. In particular, tensions arose between certain individuals as a result of conflicts around material formats and deadlines.

Different material development approaches and styles emerged, with early drafts in some cases being considered poorly structured, too long and repetitive, and not instructionally packaged. However, incorporation of feedback and continued work on these drafts eventually produced materials more in line with Schoolnet’s model. Furthermore, some of the developers did not adhere to the deadlines for submission of first drafts, their feelings being that these deadlines were unreasonably tight. They therefore missed them and worked

37


according to their own deadlines. This suggests that mutual agreement on the deadlines, and in some cases the format, was not achieved. These problems more than likely emerged because the development process was too rushed and management strategies were unclear. This clearly impacted negatively on the overall quality of the materials produced.

As a result of these problems, relationships between the different developers were at times strained, thus highlighting the importance of establishing sound professional relationships between materials developers and other stakeholders. Again, the rushed nature of the development process may have impacted negatively on these relationships. Some problems could also have been avoided if the course materials writer and the course materials coordinator functions did not reside with the same person, as this meant that no-one within the materials development team was able to take on a mediation function effectively when problems arose.

A further problem arose due to time constraints on the language editorial process; Alixe reviewed certain drafts, but in some cases did not have time to edit final versions once changes had been made. This again suggests that more time should have been allocated to developing the materials.

DESCRIPTION OF THE COURSE MATERIALS

The materials for the course were CD-based, with limited links to the Internet. The CD provided learners with all the course materials and supporting resources. On loading the CD, the Home Page automatically opens, and here the Schoolnet logo and Schoolnet mission (‘Creating communities of learners and educators using computers and the Internet’) are displayed. The following main headings, or links, also appear on the Home Page. A brief description of each of these sections of the CD is provided below:

About this CD: This section introduces learners to the CD, and puts the CD into context in terms of the introductory course and the distance-learning programme. The different headings on the Home Page, some of the tools for navigating the CD, activity icons, and links to other information are explained in this section.

Introductory Course: In this section, a copy of the introductory course manual is provided (so that someone with previous experience of computers, but who missed the introductory course, can use the CD and go through the materials). The manual material introduces learners to the course, explains what they will be getting at the course (CD and face-to-face tutoring), the expected outcomes of the course, and its purpose. By navigating through the ‘next’ buttons, the learner is taken through each section of the introductory course, these being: • Getting started (switching on, using the mouse)• The resource CD (inserting, navigating)• Reflective journal (creating the journal, purpose)• E-mail • Interaction tools (forum and chat, copy and paste)• Feedback (and shutting down, what next)

About this Course: This section introduces the distance-learning course and how it relates to the introductory course, under the following headings:

38


• Nature of the course (including, use of e-mail, mentors, mentoring groups, reflective component)

• Modules (short description of each module) • Mentors (role of)• Interaction (how it takes place, and why it is important)• Feedback (why its needed and to whom)

Help: A number of help tools are provided, including:• FAQ (three frequently asked questions)• Technical (technical help desk number)• Computer skills (How do I…, MS Windows, Word 97, Word 2000, E-mail, Web

browsing, Forum, Chat)• Help yourself (solving problems concerning computer skills, not technical problems, by

searching this disk (planned but not operational), using the programme’s Help menu, using the CD’s Help menu or asking peers/learners)

• About this course (links to this section - see above)• Glossary (list and meanings of technical words)

Feedback: Here learners are encouraged to give feedback on mentors (to the mentor coordinator), on course materials (to the course materials coordinator), and on customer satisfaction (to the Schoolnet project manager). Relevant contact details are provided.

Forum/Chat: Initiates dial-up connection to forum and chat facilities on the Internet.

Word Processor: This section introduces the learner to this module explaining what will be learnt, the outcomes, and assessment, and takes the learner through the information and activities, under the following headings: • Finding help• Your needs• Basic document• Educators’ use• Learners’ use• Templates• Self-assessment• Course evaluation

Questioning Skills: Introduces the learner to this module explaining the outcomes, and purpose, and takes the learner through the information and activities, under the following headings:• Why ask questions?• Types of questions• Setting questions• Planning questions• Managing questions• Self assessment• Course evaluation

Information Skills: Introduces the learner to this module explaining the outcomes, and purpose, and takes the learner through the information and activities, under the following headings:

39


• Evaluation• Search skills• Self assessment• Course evaluation

ANALYSIS OF THE RESOURCE CD

General aspects of the CD are discussed first, including its design, ease of use, language, and activities. The individual subsections are then analysed, followed by the three distance learning modules.

GENERAL FEATURES

The Role of the MentorStephen Marquard pointed out that the notion of self-standing materials emerged as a major point of difference throughout the materials development process. In other words, should the materials be self-standing or should they be directly related to the mentoring process? Should learners technically be able to work through the material without the help of a mentor, or should a mentor be a necessary ingredient of the process? This is never quite clear; the materials seem to attempt to evoke a high level of independence, in particular the Word Processing module, by not having all the answers available, yet dependence on the mentor, at least in terms of certain activities, seems to remain quite high (based on the e-mails between mentors and learners).

There are clearly points where this confusion creates quite serious pedagogical problems. As has been outlined in our discussions on mentors, the mentor is intended to function as a supportive facilitator, but this role is undermined to a large extent by having some of the materials written as if by the mentor. For example, in the opening section of ‘Word Processing’, the voice shifts between the mentor and another person, which can only be confusing for learners:

These would be good solutions and you could use any of these. However, I must warn you that, as your mentor, I will not just tell you what keys to press, but rather point you in the right direction to find help from resources that will build your capacity as a lifelong learner. If the answer is not available in any other way, of course your mentor will help you out. I do not want you to feel uncomfortable or unhappy. However, you should know that every single person who has learnt to use the word processor has experienced frustration. I hope to make this less frustrating for you than usual.

At points in this paragraph, the material attempts to ‘speak’ with the voice of the mentor, but it becomes equally clear in the third last paragraph that the learner’s mentor will not be the same person as that who wrote the materials.

More problematically, though, the excerpt above reflects a dangerous tendency that surfaces at certain points throughout the materials. The learning process is based on the pedagogical assumption that – in general – there are not ‘correct’ answers to problems, but rather different ways of solving similar problems in changing contexts. The underlying logic of the

40


materials is to equip learners with tools and information to enable them to solve their own problems. However, excerpts such as the one above create a dialectic that contradicts this undermines this approach, because it communicates such a clear message that the mentor knows the answer but is requiring students to guess it before divulging it. This is made even worse in the above example when the following question is later asked:

Have you been able to open that file? Hopefully one of the above approaches to helping yourself become an independent learner did help. If all else fails, e-mail me, your mentor.

In this example, the mentor is being set up not as a support system, but rather as the final and most reliable source of knowledge, the very opposite of the learning approach intended for the materials.

For these reasons, it becomes clear that the role of the mentor within the materials needs to quite seriously re-considered. This is not simply because too much weaving of the mentor into the materials undermines their capacity to be used independently, but more importantly because the role of the mentor voice within the materials currently undermines the broader educational philosophy underpinning the course as a whole.

DesignThe design of the CD is relatively straightforward and easy to follow, although it is clear that a familiarization process is necessary (such as took place during the introductory course). The main sections are headed on the Home Page and a link to Home is visible on every page of the CD so that users can always return to this original page. On going into each of the main sections, subtitles are displayed and highlighted on the left hand side of the screen so that it is easy to establish where one is on the CD. The look of the pages is generally consistent in terms of colours, icons and diagrams. Arial font is used primarily, which is simple and easy on the eye, but some inconsistency emerges since Times New Roman is sometimes used. Use of italicised text is avoided (generally more difficult to read than plain text).

Diagrams are used effectively to create interest and to demonstrate practical elements of the course, such as how to hold the mouse. There are not too many diagrams and they are stored locally and are limited in size, and therefore do not slow the functionality of the CD. Similarly, the lack of multimedia ensures that the CD loads information fast.

On the whole there is good use of ‘white space’ on the screen, which facilitates easy reading. However, in several places there are long paragraphs of continuous text that are difficult and tiring to read on screen. Alixe Lowenherz confirmed this design problem. She felt that the structure of the materials was fine, but that there was too much detail in some areas and a certain amount of unnecessary repetition. The inclusion of too much information on some screens suggested to her that the materials were more suited to print media. Alixe suggested that more click-through links to see additional information should have been included. Bullet points and bolded headings are used in some placed on the CD, but would clearly help to break up the long paragraphs that appear on certain pages. Similarly, elsewhere on the CD, there is good use of coloured blocks to distinguish quotes/comments and activities, and these help to create interest and to avoid monotonous text.

Finally, switching from a 1024 x 768 screen resolution to 800 x 600 revealed that some screens have not been designed assuming an 800 x 600 standard, requiring users to scroll from left to right in the browsers. In a South African context, it is probably still most sensible

41


to assume 800 x 600 as the screen resolution standard (particularly as it does not really negatively affect 1024 x 768 users to design in this way).

Ease Of UseSchoolnet’s intention was to produce materials that were self-explanatory, easy to follow, accessible, and which directed the learning process effectively. Interviews with learners confirmed that most of those who proceeded with the distance-learning programme generally found the materials easy to use and the instructions clear. Learners also maintained that they were satisfied with the information given on the CD, and felt that it was adequate and sufficiently clear even for new users of computers. Some learners said that they could not think of any suggested improvements to the CD. These comments are supported by the e-mail discussions during the mentoring process, which suggested that the only problems experienced related to specific tasks or activities, rather than to issues of trying to follow the materials.

The CD is navigated by clicking on headings and subheadings. Hyperlinks lead to additional documents and diagrams, and ‘next’ and ‘back’ arrows move the user between pages. However, although these navigation links are uniform, the page widths often are not. Some pages are too wide for the screen, thereby necessitating scrolling to the left and right. This makes reading the CD difficult and slow, and obscures certain text blocks and navigation tools (such as the ‘next’ and ‘back’ arrows at the bottom of some pages).

Although this is unlikely to have been picked up on by users, it did appear in some places as if the above ease of use had been achieved predominantly by creating a resources that is navigated in a linear fashion. Aside from not taking advantage of the digital platform, this also creates the impression that learners are – rhetoric aside – simply navigating a pathway of learning mapped out for them by educators who know what it is that they need to learn and who expect them to follow a single, uniform to develop pre-identified knowledge. Although this impression is not overwhelming, greater consideration of lateral and alternative navigation pathways would help to overcome a nagging sense that learners are still predominantly expected to do what they are told by the educators on the programme.

Further, the point has been made above that some long, unbroken text might have been reduced by using click-through links. Likewise, there are many places in the CD where greater advantage may have been taken of the digital environment by ‘chunking’ material into smaller pieces and designing a less linear set of navigation pathways. This design improvement could add considerably to the quality of the CD as a whole, as well as opening opportunities for different kinds of users to harness the resource in ways possibly not imagined by the designers. A good example of this might be the teacher who found aspects of the resource useful for teaching her own students. Greater navigation flexibility – where it does not compromise the pedagogical flow of the materials – could add significantly to the value of the CD.

LanguageThe language of the CD is clear and relatively straightforward. Most of the learners who were interviewed confirmed this, as they generally felt that the language was easy to follow, even though it was not their home language. They were able to read and understand the course content and instructions without difficulty, and few problems were encountered.

42


However, some learners complained that certain terms on the CD were not immediately familiar, such as ‘currency’ and ‘validity’, and therefore a dictionary was sometimes necessary. This was particularly true of words and language encountered on some of the linked websites. The term ‘reflective journal’ was also originally confusing for some. It was also true of some of the quotations included that use high-level words and language that are probably lost on individuals for whom English is not a home language. For example, the quotation in the Questioning Skills module that reads

He knows he must not jump in too quickly, thereby robbing the student team of authentic learning. He is aware that synthesis often requires some incubation and struggle.

This suggests that quotations should be more carefully selected to suite the level of learner likely to use the CD.

Another area – possibly most problematic – where language is sometimes confusing is in the outcomes and associated information. Although most are simple to understand, some statements – such as ‘prepared and conducted a lesson with the integrative use of the word processor’ and ‘Formulate questions that stimulate interaction with information’ may unnecessarily fill learners with a sense of inadequacy about their ability to achieve goals that sound much more complicated than they actually are.

One final example illustrates the danger of overly complicated, abstract language. This is the table used in Questioning Skills section called ‘Why we Ask Questions’. Here the problem is not so much with language use itself, but rather with the level of abstraction expected by learners reading about such concepts without the benefit of home language. Examples such as this would benefit greatly from more sustained attempts to simplify language and, more importantly, by efforts to integrate more practical, illustrative examples into the text.

In most instances, though, the tone of the language is comforting, yet up beat, and was described by some learners as ‘welcoming’. Possibly, in order to enhance enjoyment in the learning process, there could have been increased usage of humour; the comment about making mistakes (‘congratulate yourself on being normal’) was greatly appreciated by at least one of the introductory course groups.

Given all of the above, another final consideration worth entertaining would be to translate at least aspects of the materials into one or two other languages. This might greatly facilitate understanding, as well as helping learners to identify more easily with the course materials.

ActivitiesActivities or assignments are used throughout the materials to reinforce the learning process and to drive interaction between learning-group members. In fact, several learners confirmed that the materials were successful in that they emphasized practice more than the theory. Through most activities, the materials create opportunities for collaborative sharing and problem solving. The intention was to present these activities as problems to solve. Learners were then expected to use the CD as a reference from which they could find the answers to these problems, thus learning some independence in seeking help. Therefore, it was hoped that the materials would lead the teacher through all the activities without additional help or input by mentor (although, this does not imply that the mentor is unnecessary).

The activities are clearly delineated on the CD pages in pink/beige-coloured text boxes and are marked with icons that indicate what the activities are about. These icons are explained in

43


the ‘About this CD’ section, but it is felt that their real purpose and benefit is not very clear. The icons make the appearance of some pages more interesting, but learners are unlikely to memorize or refer to them. On the whole, the instructions for the activities are unambiguous and easy to follow, and for the most part, there appears to be sufficient information provided before learners are asked to perform an activity.

Learners’ responses to the activities were generally positive. It was felt that the activities effectively enhanced or complemented the contents of the CD, and were interesting and diverse enough to prevent boredom. The activities also reinforced what was being learnt by forcing learners to refer back to sections previously (or not yet) covered. Some learners felt that there needed to be more activities for people who had not used the computer before, for example, a keyboard primer to help beginners become more familiar with the keyboard by introducing them to the various keys in context. Another suggestion was that different levels of activities be introduced for ‘beginner’ and ‘more advanced’ learners.

Activities were also generally effective in drawing the learning experience into the classroom; certain assignments required introducing new concepts and activities into the classroom, and then reporting back in the reflective journal, on their effectiveness. This experiential learning in the classroom should be effective in driving improved teaching practice, as well as in stimulating new, innovative ideas in teaching approaches.

However, despite this generally positive response, some problems with the activities were experienced. For example, some activities were found to be too complex. Similarly, although the materials were premised on the notion of self-exploration, the expectations of certain activities were found to be unclear and there was sometimes insufficient content leading the learners to find answers. Problems were also sometimes encountered where activities required accessing online sites, because some learners experienced difficulties in getting online or had restricted access to the Internet.

SUB-SECTIONS

Home PageThe layout of the Home Page is simple and uncluttered. Only the key headings (and a few subheadings) are displayed, as well as the Schoolnet logo. The page therefore provides important information, and serves as a central reference point which can be returned to from any page on the CD and which provides access to the various sub-sections of the materials.

About this CDThis sub-section ensures that learners have immediate and upfront access to information on how to navigate the CD. The information is clear and concise, and good use is made of the actual headings, icons and textboxes, so that learners can easily relate the explanations to these elements as they are appear on the pages.

Introductory Course Since the introductory course was discussed in detail in Chapter Two, issues relating purely to the materials on the CD are mentioned here. Copying the introductory course materials onto the CD provides an important reference for learners who attended the one-day course but who wish to review what was learnt. An introduction is also provided for users who did

44


not attend the introductory course (but who hopefully have some experience of computers). Given these roles, it is appropriate that this section of the CD is self-explanatory and easy to follow. Good use is made of step-by-step explanations and of photographs to explain practical elements or to illustrate where things are located on screens or drop-down menus.

However, a few omissions in this introductory course section should be noted: • When using the hyperlink to view the different types of mouse-clicking actions, it is not

clear where one has moved to on the CD (i.e. to Help) and that learners should use the ‘go back’ arrow to continue with the introductory course. Therefore, there should be a ‘back to introductory course’ link at the end of the table on mouse-clicking actions for users who have accessed this section via the introductory course.

• When the concept of the reflective journal is introduced, it should be specified that the journal must be e-mailed to the mentor coordinator (this is only mentioned near the end of the introductory course).

• It is not explained in the forum section that learners need to respond to others’ comments.

• In the ‘What Next?’ section on the final page of the introductory course materials, learners are not encouraged to start working through the CD as soon as possible; it is suggested that they wait to hear from their mentors. Given the delays that were experienced between the introductory course and commencement of the distance-learning programme (often due to problems with e-mail), learners should be advised to start working on the materials while the introductory course is still fresh in their minds. This will help to keep up the momentum of the course and will prevent learners from getting left behind if e-mail problems need to be sorted out.

• One learner felt that that there should be more on file management aspects, and Windows Explorer and making folders should be introduced at some point in the course.

About this Course: Here, important background information on the distance education component of the programme is provided, and learners find out what will be expected of them. In some areas, large areas of text appear, which are quite difficult to read. Click-through links or alternatively formatted text should perhaps be used.

HelpLearners seemed to appreciate the supportive nature of the CD, in particular the relatively comprehensive Help section. Nonetheless, some learners who were interviewed reported that they did not have to use this facility (this was probably dependent on their level of previous exposure to computers). All new terms are adequately explained in the glossary section.

The following changes or additions to the Help section should be considered:• On the first page of Help, the ‘help tools’ on the left hand side of the screen should be

pointed out in the introduction to finding support.• ‘Frequently Asked Questions’ should be written in full, as many learners did not know

what FAQ stood for.• Steps for opening Pegasus should be included in the section on e-mail.• More information should be provided on technical issues and in the FAQ section.• There is sometimes inconsistent use of Times New Roman font, and this should be

avoided.

45


• Some problems were experienced in classifying where help information lies, and therefore ‘CD search’ section should be included as learners would not need to explore sections of Help in order to find what they are looking for (this was confirmed by some of the learners who were interviewed) .

FeedbackThe purpose of, and need for, feedback is simply explained, and the contact details are made easily accessible to users. Nonetheless, very little spontaneous feedback was obtained from learners on the pilot course. The use of more reminders throughout the CD, asking learners for feedback on different sections, might have produced a better response. In addition, simple forms that can be converted into e-mail messages might also have facilitated more user feedback.

Forum /ChatGiven the problems experienced with this section on the introductory course, when accessing Forum/Chat from the menu bar, there should be a prompt to go to help for instructions on how to use these facilities.

MODULES

The modules should be designed with a consistent structure, unless there are clear reasons for using different structures. For example, there are possibilities for improving standardization in terms of introductory headings such as ‘purpose’ and ‘assessment’.

All the modules encourage a high level of interaction between learners within the same learner-group. This helps to create the community of learners that should extend beyond the mentoring process. However, there is some concern regarding the effectiveness of this interaction, given that learners are asked to compare their answers on different activities, yet they clearly progress at different rates through the work. Many learners experienced time constraints or had limited access to, or problems accessing, the Internet:

I’m battling to do some of the things which need Internet since our internet is not working, we only have one computer with internet so we are sharing but I will try them soon.

No doubt, this lessens the extent to which interaction on the different activities can take place, since learners arrive at and complete activities at different times.

The final sections of the modules are consistent (‘self-assessment’ and ‘course evaluation’) and the table format is easy to read and complete. These tables provide an important opportunity for reflection and feedback. However, learners are not encouraged to e-mail this information to their mentors, despite the opportunity for the course designers to obtain valuable input on the process and materials. Technically, this should not be too difficult to incorporate into the tables themselves (although some consideration may need to be paid to the implications of using Java Script and other high-end browser functions in the materials). The end of each module could also include a suggestion that learners attempt to work through the other modules not already covered, in order to maximize their gains from the course and CD. Having worked through one of the modules in a supported format (i.e. with a mentor), most learners should be able to continue more or less unaided.

46


Concepts such as ‘just in time’, that have meaning for all the modules, should also be explained at the beginning of each module (currently, it is only explained in the Word Processor module).

Word ProcessorThe Word Processor module concentrates on making learners independent users of the word processor as educators. Independence is encouraged because users are not immediately given all the answers on how to do the activities and are therefore forced to look for help elsewhere on the CD or from other sources. For example, in Activity 1, learners are encouraged to use ‘help’ to find out how to open the ‘establishing your needs file’. This activity was clearly a challenge to learners, some of whom had to resort to asking their mentor for help. On reviewing this task however, it was found that the information in ‘help’ is in fact insufficient, since the explanation on ‘opening a current file’ does not explain how to ‘browse’ to find the file location, or to change the file type from HTML to ‘all files’. This meant that there was insufficient study guidance and this made it difficult for learners to achieve the objective of the activity. However, the process of searching for the information does lead learners to explore the support facilities on the CD, thus teaching them independence in finding help for other problems. This meets the aim of trying to concentrate on the educators using the computer as a tool rather than on then learning about the computer.

The module draws on skills learnt in previous sections on the course, such as how to copy and paste, therefore encourage learners to use what they have learnt. This reinforces concepts through practice and by requiring learners to refer back to sections covered previously, or to use Help.

In addition to creating independence in learners, this module also concentrates on helping educators to look for contexts in their work in which the word processor would be a useful tool. For example, in Activity 2, learners are required to consider and prioritize the documents that they currently create in their work as teachers and later they learn to produce one of these documents on the computer. Therefore, these activities clearly put the word processing course into a teaching context, but should also make education processes more efficient by teaching educators how to produce documents they need on the word processor. Furthermore, in Activity 8, learners are encouraged to use the tools being learnt in the context of the classroom, i.e. with their own learners.

The process of producing teaching-related documents is made even easier by introducing the concept of templates. However, some learners had trouble with this activity, and one commented that the expectations were not clear. Nonetheless, another learner pointed out that she would have liked the educators to be required to be creative with templates and design their own before being shown where to find templates that are pre-designed.

Problems were experienced with tables in the activities, but this was because some learners had Office 95 and the materials are written based on the assumptions that the learners would have access to Office 97 or later.

In general, this module appears to cater for beginners, yet it is sufficiently challenging to be of interest to those with some exposure to computers. The materials hang together logically, flow well, and are easy to follow. The skills taught are highly relevant to a teacher’s work environment, and both the content and activities are appropriate to the objectives. The tone of

47


this module is informal but clear, although some sections include long, wordy paragraphs (such as the explanation of the ‘just-in-time’ concept) and should preferably be broken up with formatting, such as bullets or bolded text. Elsewhere, good use is made of click-through links to introduce additional, relevant information.

Questioning SkillsThe Questioning Skills module helps learners develop skills in asking questions that make them think, and encourages them to provide situations in class whereby learners can ask questions. The intention is that this enquiry will place the Worldwide Web and its information sources in context as a valuable resource in schools. Learners are therefore armed with questioning skills that enhance learning in the classroom. The outcomes and purpose of the module, as summarized above, are clearly stated at the beginning of the module.

Like the previous module, the Questioning Skills section puts question-asking in the context of the classroom by requesting a list of types of questions that teachers ask in the classroom. In this way, the learning context of the materials was rooted in the current classroom practice of the learner. Interestingly, ‘types’ of questions have not yet been introduced (they appear later), yet the learners do not seem to have had any problem in listing types or categories of questions that they use in class. Similarly, learners are encouraged to take this process into the classroom by getting their own learners to think about types of questions asked in a classroom context. Excellent use is made of examples, such as the example discussion in Planning Questions, and quotations are used extensively to make the module more interesting and to reinforce points. Therefore, the examples, analogies and case studies seem relevant to learners’ interests and are sufficiently illuminating.

There is noticeably less opportunity in this module for learners to use what they have learnt or to seek assistance from ‘help’ or other resources. More could have been made of this opportunity perhaps, although it is recognized that Questioning Skills relate less closely to the computer skills learnt in the introductory course than do Word Processor Skills. Reference is made now and then to ‘children’ when talking about learners that educators teach. However, it should be remembered, that not all learners are children, and that they should rather consistently be referred to as ‘learners’.

Similar to the previous module, there are some sections with long paragraphs. Although this text seems relevant and clearly has a positive effect on improving teaching skills, using more click-through links, or simply highlighting or bulleting parts of the text could improve the layout.

Information SkillsThe Information Skills module concentrates on the skills required for using search engines on the Internet and helps learners to identify and use criteria for evaluating the reliability and value of information found on the Internet. The outcomes and purpose of the module are clearly explained and interesting, creative graphics are used. On the whole, long wordy paragraphs are avoided by using shortened paragraphs, bullets, and diagrams. The layout of the module is appealing, though the changing page width is very irritating to work with and should be amended. This module is quite long, which seems appropriate if the impression that it is for more advanced learners is correct. However, there should be greater clarity as to the appropriate levels for each of the modules. It is the evaluator’s opinion that the

48


information in this module is very useful, as many users of the Internet do not understand the difference between various search engines or the specifics of how to use them. Therefore, the information is very current and relevant to teachers in the modern era.

One of the major problems experienced with this module related to access to the Internet, which was clearly vital for progression with the notes and activities. This is clearly a factor for consideration under school selection; many learners had limited time to work on the Internet, or had difficulties when trying to access the Internet.

At the moment I am busy with activity 2. (Preparing a lesson and using the Internet to do so.) I am finding difficulty in doing this, as I do not have my own computer to surf the net at home. As you know, at school we have very little time to spend on the computer. At the moment it is only the 15-minute interval time and sometimes after two in afternoon. That is, if there is no staff meeting.

A few other problems were experienced with the activities: One learner did not know what they should send to the learner group in Activity One, and requested help from her mentor. Another learner could not find the information on when the Pretoria Union Building was built. A spelling mistake was noticed in Activity 11B, Questions 5: ‘seen’ not ‘seem’.

CONCLUSION

In conclusion, there seems to be some need for changes to the materials, although they clearly represent a very good first attempt to create a different kind of learning experience for teachers. Broadly speaking, some changes to the uniformity of the structure of the modules is necessary, certainty of adequate support being available from the resources (without the mentor) must be obtained, and certain long paragraphs of text should be broken up using click-through links, bulleted or bold text.

Given the generally positive response to the materials from participating teachers, it is not surprising that they were very much in favour of the idea of creating a similar resource with activities for their own learners.

The CD could be adapted to cater for different levels of learners, say ‘beginners’ and ‘more advanced’ by grading the sections or modules according to these levels, and providing different levels of activities. In this way the CD would cater for a larger audience of learners. Alternatively, two different CDs could be developed catering for the different levels, which would allow content and activities to be focused on different levels.

RECOMMENDED CHANGES TO THE CD

Detailed, recommended changes to the CD are listed below.

General • Re-consider the portrayal of the mentor within the learning materials to ensure that it

does not contradict the underlying educational philosophy of the programme.

49


• Ensure that all learners undergo an adequate familiarization process with the materials. This is particularly true for beginners, since they will clearly battle to follow the materials if not given initial guidance More support of this kind should also be integrated into the resource itself.

• Remove inconsistencies in the font used, where this is not justified stylistically.• There should be more encouragement and tools throughout the CD for learners to send

feedback to the various contact individuals described in the feedback section.• Work on long paragraphs of continuous text that are difficult to read, by making

increased use of bullets, bolded text and click-through links.• Ensure that the page width is consistent with a computer screen width to negate the need

to scroll left and right.• Avoid quotations and sentences that make use of high-level language and terms and

ensure that all potentially difficult terms used on the CD are adequately explained.• Review activities that caused difficulty and ensure that they are not too complex or that

the expectations are adequately explained. The option of introducing different levels of activities for beginners or experienced users should be considered.

• Exclude or significantly improve the Chat and Forum sections as many problems were experienced with these facilities.

• Provide greater clarity as to the appropriate levels for each of the modules.• Undertake thorough editorial to check for and correct spelling, grammatical, and typing

errors.

Introductory course • Include a ‘back to introductory course’ link at the end of the table on mouse-clicking

actions for users who have accessed this section via the introductory course.• When the concept of the reflective journal is introduced, specify that the journal should

be e-mailed to the mentor coordinator. • In the ‘What Next?’ section, encourage learners to start working through the CD as soon

as possible. This will keep up the momentum of the course, thereby hopefully encouraging a greater level of motivation amongst learners.

• Add information on file management aspects, and Windows Explorer and making folders.

Help• On the first page of Help, the ‘help tools’ on the left hand side of the screen should be

pointed out in the introduction to finding support.• ‘Frequently Asked Questions’ should be written in full, as many learners did not know

what FAQ stood for.• Steps for opening Pegasus should be included in the section on e-mail.• More information should be provided on technical issues and in the FAQ section.• Some problems were experienced in classifying where help information lies, and

therefore a ‘CD search’ section should be included as learners would not need to explore sections of Help in order to find what they are looking for (this was confirmed by some of the learners who were interviewed).

Modules• There should be improved standardization in terms of the introductory headings, such as

‘purpose’ and ‘assessment’.

50


• Learners should be encouraged to e-mail the self-assessment and course evaluation tables to their mentors.

• The end of each module should include a suggestion that learners attempt to work through the other modules not already covered, in order to maximize their gains from the course and CD.

• Concepts such as ‘just in time’, that have meaning for all the modules, should be explained at the beginning of all the modules.

• Word Processing module - Activity 1: there should be more guidance to learners on how to open a current file.

• Expectations in the templates section should be made clearer.• Problems of different versions of Microsoft Office should be addressed and understood

by both learners and mentors.• In the Questioning Skills module, there should be a section that explains the World Wide

Web and the Internet, so that users know what these are when they are making use of them.

51


CHAPTER SEVENRecommendations

RECOMMENDATIONS

Based on the various issues that have been highlighted during the evaluation in the preceding chapters, a number of broad recommendations have been formulated and are discussed in detail below. These recommendations should be considered in addition to the individual, detailed recommendations given at the end of each of the chapters.

LEADERSHIP AND PLANNING

As part of the planning process it is clear that careful consideration needs to be given to the timing of such a project. The timing of the pilot was poor since it commenced during the last week of a school term when many educators were very busy marking examinations, organizing events, and so on. The holiday period caused a lack of momentum in terms of the skills that had been developed during the introductory course, since few learners were able to work on the distance learning programmes during the holidays. Therefore, holiday times and very busy periods of the year should be avoided where possible (certainly for the initial stages of the project).

Focused, well-prepared leadership is an essential element of an effective education programme. In this project, absence of leadership seems to have been a significant issue, which had a number of consequences. Therefore, in a project such as this, the leadership needs to focus on achieving the following: • Providing clarity on the level of expertise required by developers involved in the

process, and selecting skilled individuals to take part. (In this project there was generally an insufficient range of expertise).

• Defining clearly the various roles of different stakeholders.• Providing clarity on the level of computer literacy required by learners involved in the

course.• Spelling out any assumptions made during the development of the course and ensuring

that they are understood by all stakeholders.

SCHOOL SELECTION

The school selection process needs to be well planned and managed, including well-developed and clearly specified selection criteria that take into account the following issues:• Presence of a good technical infrastructure, including existing networked and online

computers, and e-mail addresses and functional e-mail facilities.• Presence of on-site information technology support and back-up.• Support for the project from school administration and principal. To achieve this, school

administration and management should be involved in the development, initiation, and ongoing sustainability of the project as far as possible.

52


• As part of the above, ensure that time is allowed for educators to access computers and that other potential restrictions are removed.

As already mentioned, as part of the school selection process, there should be clarity on what level of experience of computers, learners on the course are expected to have (beginners or people with some experience of computers). Most of the learners involved in the pilot had some experience of computers, and, of those who were actively involved, there were no complaints of modules or activities being too easy, or boring. Having said this, it was confirmed from a number of sources that individuals new to computers would have had great difficulty, both with the introductory course and with the distance education component of the programme. Therefore, it is vital, not only that the level of learner is agreed upon and defined, but that the introductory course and course materials are developed in line with this level of learner.

INTRODUCTORY COURSE

In line with the above point, the information covered in the introductory course should be reduced as the current schedule tries to cover too much, particularly for first-time users of computers. Alternatively, the course should be spread over two days. This latter option would have the added benefit of providing learners with more opportunity to practise what has been learnt, thereby helping them to develop these skills and their confidence.

MENTORING PROCESS

Training of mentors is vital (as has been mentioned on numerous occasions throughout this report). In particular, mentors require clarity on what is expected of them and they need to be well informed on how to mentor in an electronic environment. Mentors should also be given adequate opportunity to work through the CD materials before mentoring starts. Progress has been made on addressing this issue as a mentor-training programme has already commenced. Clearly, Trudi and Sandy should have been better informed about their role and responsibilities.

MATERIALS

Detailed changes to the resource CD are discussed in Chapter Five. Overall, it should be mentioned that the development process of the course materials should not be rushed. Adequate time must be allocated to the writing and editing of the materials in order to ensure that high quality materials are produced. This is particularly important in a distance education project, where the quality of the materials impacts significantly on the learning process. It should also be noted that the course materials writer and the course coordinator functions should not lie with the same person, given the problems this can cause when conflict arises.

EVALUATION

In terms of the evaluation process, it is felt that a more in-depth and thorough evaluation would be possible if the person responsible for writing the evaluation report was able to

53


conduct first-hand interviews with key stakeholders and to observe key sessions such as the introductory course. Because this type of first-hand interaction was not possible, much of the evaluation is based on second-hand information from individuals who carried out interviews. Furthermore, in some cases, the interviewers were not independent of the pilot, and therefore may not be able to point out issues and weak points in the process objectively. It should therefore be remembered that this analysis is highly dependent second-hand, possibly biased information. It should also be noted that interviews conducted with learners on the course primarily involved learners who had participated in the distance education programme. These learners probably had a more positive perception of the course than those who dropped out very early.

The evaluation would also be improved if there were better records of the documents available for the evaluation (for example, a list of the documents, their headings and summary of their content). In particular, to facilitate the evaluation process there needs to be a system for keeping track of e-mails sent between various stakeholders in the project, and this system should be in place before the course commences.

54

evaluation strategies - nba professional de… · web viewfor example, in the word processing...

Documents