Student Support – a Critical Factor for SuccessDesign and Evaluation of an Integrated Student Support System for E-learning

Torstein Rekkedal and Svein Qvist-EriksenNKI Distance Education, Norway

IntroductionThis paper discusses the pedagogical basis for the support measures built into the NKI distance education system and organisation to help students complete their online learning programmes successfully. The paper also reports from an evaluation study among students carried out as part of the project, Student Support Services in e-Learning, supported by the Socrates Minerva Programme of the EU Commission. Similar evaluation studies have been carried out by the other partners of the project. For reasons of comparison between different countries, institutions and systems, the quantitative part of the evaluation instrument was, except for language differences, identical with the questions used in the studies conducted by the other four project partners, Ericsson Competence Solutions and Cork Institute of Technology in Ireland, FernUniversität in Germany, University of Rome III in Italy. The paper concludes with some recommendations for priority developments of student support and also describes some important improvements of the support system implemented during 2004.

DefinitionsThis project deals with the role of student support services in e-learning. In the literature there seems to be as many definitions of e-learning as there are writers on the subject (see e.g. Rekkedal & Qvist-Eriksen (2003), Súilleabháin (2003)). In the project ‘e-learning’ is defined as the provision of education or training electronically, via the Internet. The term ‘student support services’ is used for those parts of a distance or electronic learning system, which are additional to the provision of course content (Thorpe 2001).

NKI Online Distance Education

NKI online distance education (or Internet/web based e-learning)NKI was probably the first European online college, and it has offered distance education online every day since 1987. Few - if any - online colleges in the world has been longer in continuous operation.

NKI Distance Education has today over 400 courses and more than 60 complete study programmes on the Internet. May 2004 we had approximately 6,000 registered active students. Contrary to many other educational providers, where the Internet is used as a supplement to face-to-face teaching or other forms of distance education, we have followed the philosophy that in principle all communication can be taken care of through the Internet, and ideally no obligatory physical meetings should be required. (This does not mean that the students are not free to communicate by post, phone or fax or that study materials includes print, audio or video technologies.)

NKI Internet College ’4 Generations’ of developmentThe NKI Internet College has been developed through 4 systems generations:

1987 - 1994: ‘1st generation’ based on the conferencing system ’EKKO’, a menu based conferencing system designed by our in-house systems developers no other options were available that could be installed on the mini computer in NKI at that time. The idea was that we through ‘electronic means’ could establish a virtual school and be able to simulate electronically all communication needs previously organised through solutions of combined distance teaching and local face-to-face classes.1994 - 1995: ‘2nd generation’ – ‘the open electronic college’ with the underlying philosophy of offering a system as ’open as possible to other networks and services based on the Internet, e-mail and Listserv conferencing system’.1996 - 2001: ‘3rd generation’ – the introduction of graphical interface and the WWW, taking the step from ‘small scale experiments to large scale Internet based distance education’, introducing courses and programmes below university and college level.2001-: ‘4th generation’ - In our experience, it is the step from small-scale to large-scale operation that involves the greatest challenges. Putting some teaching material on the Internet and offer one single course is not a very difficult task. The great challenge is to develop and administer an Internet based teaching organisation offering a large number of courses with high quality to a large number of participants on a continuous basis. This is why we experienced a great leap forward when we March 2001 launched what we characterize as the ‘4th generation’ with the introduction of SESAM (Scalable Educational System for Administration and Management). SESAM is our internally developed learning management system completely integrating the teaching-learning system on the WWW with our overall student administrative system (STAS). The learning management system and the student administrative system together form the basis for the complete system of student support services.

Didactical solutions for distance teaching and learningThe basic philosophy for the development of online education at NKI is founded on accepted theory and research from the field of distance education, included our own research.

Independence and autonomyMichael Moore is specifically known for his development and refinement of the theory of distance education as independent learning. The main dimensions are ‘transactional distance’ and ‘learner autonomy’. It is clear that in his earlier writings Moore put more emphasis on autonomy – as distance teaching programmes by their nature require more autonomous behaviour by the learner. To succeed in such programmes, the learner must be able to act independently and autonomously.

Transactional distance is not the same as physical distance but built up of the two qualitative and continuous variables labelled ‘dialogue’ and ‘structure’. The dialogue describes the transactions between teacher and learner, but is not used synonymously with interactions, as dialogue is described as interactions having positive qualities. The structure of a programme is determined by the nature of the media being applied and by the teaching philosophies of designers and constraints imposed by the educational institutions. Structure describes to which degree the programme is able to be responsive to individual student’s needs. According to Moore the transactional distance of a programme increases when level and quality of dialogue decrease and structure increases. Programmes with low transactional distance have high dialogue and low structure (Moore 1991, 1993).

2

Guided didactic conversation – teaching-learning conversationLong before the term distance education had been established and the terms for this concept were correspondence education, home study and independent learning Börje Holmberg argued in favour of a conversational approach to course development (Holmberg 1960 pp. 15-16) and later followed this up by attempts to formulate what can be called a theory of distance education in which empathy between the learner and the teaching organisation was assumed to favour learning. In his earlier writings Holmberg used to denote his theory of distance education as ‘guided didactic conversation’. Now he prefers the term ‘teaching-learning conversation’ (Holmberg 2001).

When analysing the teacher-learner conversation, Holmberg stresses that the conversation includes both non-contiguous conversation between the live teacher and student and also learning activities, such as thinking, processing information and other cognitive processes taking place when the student interacts with the pre-prepared learning materials including its ‘built-in tutor’. He specifically refers to the educational institution as the supporting organisation.

Continuity of concern for studentsSewart (1978, 1993, 2001) has worked with central and local support services at the UK Open University since 1973. His theoretical approach to teaching at a distance can be summed up as a ‘continuity of concern for students studying at a distance’. He discusses the dilemma between the efforts of some course developers to produce the ‘hypothetically perfect teaching package or put more resources into the support system for students during study. He finds the perfect package to be unrealisable, and however perfect the pre-produced material is, the teacher, tutor or tutor-counsellor as well as student advisors are necessary as intermediaries between the learning material and the individual student.

Sewart (1978) also maintains that the success of the Open University largely is a result of its support and counselling systems.

Cooperative learning and constructivism Learning theories based on social constructivism (see e.g. McConnell 2000) have dominated the ground, at least in Scandinavian academic institutions and have had considerable influence on higher education institutions when embarking on distance education. It is not difficult to agree with the ideal theories of learning as a social process where students help each other to develop understanding in an enjoyable and stimulating context and not only as an individual pursuit concerned with accumulating knowledge. The learning is process-driven and learners must be involved in the social process and pay attention to this process to achieve their desired goals. The outcomes are not only academic, but involve increased competence in working with others, self-understanding and self-confidence. Learning is seen as a construction of meaning in interaction with others (teacher and fellow students). Knowledge is constructed in social groups.

However, according to NKI research and evaluation studies we must acknowledge that for most distance learners the primary social arena for learning is often not the group of fellow students, but their local society and environment including family, friends and colleagues. Our main challenge may seem to find solutions in designing learning methods stimulating individual learning constructed in interaction with the student’s own social environment, colleagues, friends, family and others.

3

FlexibilityFigure 1 illustrates six dimensions of flexibility that many individual students want. Many institutions (among them NKI) have put major emphasis on designing online courses to be flexible concerning time and schedules. It is a great challenge to develop online learning environments that support this individual freedom as well as collaborative learning. This challenge is discussed in the theory of cooperative freedom (Paulsen 1993, 2003). There is no doubt that design and administration of student services is related to how the teaching learning model emphasizes individual freedom in learning relative to collaborative learning.

Figure 1. The hexagon of cooperative freedom (Paulsen 1993, 2003).

It has been demonstrated through a number of evaluation studies that individual flexibility is the most important aspect for the majority of NKI online students (Rekkedal & Paulsen 1998, Rekkedal, 1998, Rekkedal 1999). This means that learning methods heavily based on co-operation and collaboration with fellow students in real time and/or paced and rigid progression schedules do not seem to be in line with the needs of the NKI distance learner.

To balance between the students’ need for individual freedom, autonomy and flexibility and solutions based on collaborative learning methods, we have chosen the following basic philosophy for the development of Internet based education at NKI:

Flexible and individual distance teaching with the student group as social and academic support for learning. Within the framework of individual flexibility and free progression schedules we try to stimulate cooperation between students without requiring formal group work.

NKI recruits thousands of online students every year. These students may enrol in any of the more than 60 study programmes or 300 courses or in any combination of courses at any day of the year and progress at their own pace. This flexibility does not exclude group-based solutions in cooperation with one single employer, trade organisation or local organiser.

The overall choice of didactical solution does have some specific consequences for the design and operation of some aspects of the teaching-learning system. For instance, the NKI Internet courses generally put more emphasis on individual studies and individual work on exercises

4

and assignment for electronic submission to a tutor for comments, evaluation and feed-back. Although it has been a main objective from the very beginning to exploit the possibilities for learning support from the learning group through interaction and discussion with fellow students, in most courses the majority of students have not taken the advantage of using the course Forums very actively.

5

Student support in the NKI Online Distance Education SystemThe table below was presented by Rekkedal & Qvist-Eriksen (2003):

Time Support needs Component responsible Tools/applications

Prospective phase

Information about courses Administration Print, WWW, print/ broadcast media etc.

Guidance concerning choice of courses and programmes

Administration Phone, e-mail

Financial questions, loans, grants Administration Print, phone, e-mailGuidance on practical matters Administration Print, phone, e-mail

Start-up phase

Dispatch of printed and other physical learning materials

Administration Surface mail

Registration/information/user identity, passwords etc.

Administration e-mail

Introduction to online learning techniques

AdministrationFaculty

Phone, e-mailPhone, e-mail

Initial follow-up AdministrationFaculty

Phone, e-mailPhone, e-mail

Technical support Administration Phone, e-mail

Learning phase

Teaching/tutoring Faculty Phone, e-mail, Forum, WWW media

Academic support Faculty Phone, e-mail, ForumOrganisation of learning Faculty Phone, e-mail, ForumSocial support Faculty Phone, e-mail, ForumAssessment Faculty Phone, e-mail, ForumPractical support, economy etc. Administration Phone, e-mail, ForumFollow-up Administration Phone, e-mail, surface

mailTechnical support Administration Phone, e-mail, ForumResources/library Administration Print, WWWLearning group support Fellow online

studentsPhone, e-mail, Forum

Local learning support Local facultyClassmates

Face-to-face

Local administrative support Local administration Face-to-face, phone, print

Local technical support Local facultyLocal administration

Face-to-face

Local social/practical support EmployerFamily

Face-to-face

Graduation Diploma/accreditation Administration Print, face-to-faceAfter graduation

Counselling on further study Administration Print, e-mail, WWWCounselling on job opportunities Administration WWW, ForumAlumni services Administration e-mail, WWW,

6

ForumFigure 2. Framework of student support services for NKI online distance students

In the table we have included the following components of the system:Administration:

Marketing and sales staff, course coordinators, counsellors, advisors, office staff

Local administration (study organisation, employer, local office)Faculty:

Senior faculty and internal academic staff, external and internal tutors Local teachers

Fellow students: Students in same course, in other courses and classmates in local learning

groupsEmployer, family and colleagues:

Not usually included in analyses of educational systems, but may be seen as (the most) important support system for online distance students (e.g. Bajtelsmith 1988)

As the pre-produced course materials (e-learning package) (Thorpe 2001) is not considered to be part of the student support services, office and academic staff is not included among the support components as in the model suggested by Aoki & Pogroszewski (1998).

The support services for NKI e-learning students were described in more detail by Sjaastad et al. (2003), where the ‘Prospective’ phase in the table above was described as ‘Information/Guidance Phase’.

Student support services in different phases

Information Phase/Guidance PhaseInformation on the InternetInformation about the programmes/courses offered by NKI is available on the NKI web site on the Internet. There is an efficient search engine for courses, special pages for individuals and employers and a FAQ page and access to guidance and counselling be telephone or e-mail. The web pages also present general information about research and other relevant news to prospective students. There is also has a demonstration programme (in the form of a guided tour) on Internet based distance learning.Information in printNKI produces a print catalogue, which is revised once a year. From spring 2003 the print catalogue is generated from the web site (that is continuously updated). The print catalogue is distributed to potential students and to individuals on their request. NKI also produces brochures with more extensive information for all major learning programmes.AdvertisementsNKI advertises courses and programmes in newspapers, weekly and monthly magazines and professional journals and through television.Representatives and prospective student advisorsRegional representatives visit companies and study organizations throughout the country to inform about programmes/courses offered by NKI. There is also counselling staff at the NKI main office for guidance of prospective students, mainly through e-mail or on the phone on choice of courses and programmes.

7

Registration phase/Start-up phase:Registration by mail or on the InternetStudents submit their applications to NKI by regular mail or through the Internet. If some information is lacking, the applicant is contacted through e-mail. If the student is qualified he/she is registered as soon as sufficient information is received. Practical study informationThe same day that the student is enrolled in a course/programme, all practical information regarding the programme and how to make use of our web site is sent by e-mail to the student. This information includes initial study counselling about study progression and assessments, information about financial conditions, such as payment and student loans and grants, and necessary contact information. The student receives her user name and password via e-mail the day after registration.Information on the web site On the personal web page the student has links to all courses that she is signed up for, a link to a survey of progression and the grades she has obtained and a link to the page where the student can change her registered personal data. There are also links to news from NKI and other resources the student may need during the programme. On NKI`s web site the prospective students and students can also get a demonstration through a guided tour of how to use the NKI web sites in their online education. Some programmes (mainly on lwer levels) include a course in study techniques for online distance education, in which the student gets individual guidance from a tutor specialized in the field. Technical supportTechnical support is available to all students day and night. They can contact NKI’s technical staff through e-mail or by telephone (between 8 am and 4 pm). Printed and audiovisual learning materialsPrinted and other physical learning materials are sent to the student within a few days after registration. Follow-up of new studentsA few weeks after registration the students receive a follow-up letter from NKI encouraging student activity. The content varies depending on whether or not the student yet has submitted any assignments.Telephone or e-mail support and guidanceNKI has a specific department specialised on student support and guidance. Each student advisor is responsible for support services to students enrolled in certain programmes. The student advisors communicate with the students by mail, telephone and fax. The need for personal support is often quite large during the first phases of study.

Learning phaseE-learning and physical learning materials (print, audio and video tapes, CD-ROMs etc.)Study materials in most courses consist of both physical learning materials and online materials. (In this connection efficient services means efficient logistics in distribution to students.)Help desk and advisors/student counsellorsThroughout the programme the student is allocated to a personal advisor/counsellor who can be contacted by e-mail or telephone about any practical issue or questions about studies or learning techniques. Academic staff and instructional designersThe senior academic staff members are academically responsible for content, literature of the programmes and are available to answer questions regarding all academic matters.

8

Instructional designers are responsible for the web pages and the links and resources on these pages. Discussion forumsAll courses are attached to a forum. Forums enable communication between students and also between students and the tutor. Some course tutors are very active in the course Forum. In other courses the Forum is mainly for student interaction. The student advisor/counsellor regularly uses the Forum for necessary information about assessments, news about courses, news regarding the web sites and any other information that needs to reach all the students that attend the course. In some programmes the academic staff and instructional designers also participate in forums with information and problem solving related to their work. The students may at any time on their web site change their e-mail address, password, personal presentation and the frequency of receiving messages from the forum.The personal tutorThe personal distance tutor is probably the most important support agent in the NKI e-learning system. The tutor is responsible for commenting, assessing and giving feedback to individual students based on their submitted assignments. On the web site of each course the students find a presentation and a picture of their tutor in the given course. The tutor participates in the course Forum and also tutors each student individually by e-mail or by telephone. Students may phone or e-mail the tutor when they have questions regarding the academic contents of the course.Face-to-face teachingIn some programmes face-to-face sessions are organised as a supplement to the teaching and learning on the Internet. Such sessions can for example be held weekly, on irregular intervals or once or twice a semester.Follow-up of inactive studentsStudents who are inactive are regularly contacted by mail. They are encouraged to get on with their studies or to contact their advisor/counsellor if they have problems related to the programme or other problems that interfere with their studies. Evaluation and assessmentAssessments are done on several levels throughout the programmes. The students are encouraged to do self-assessment through exercises that are included in the materials. Some courses have automated assessment through multiple-choice exercises and in some courses the students comment on each other’s work in forum. Assignments for submissionAt the end of each unit the students will normally have to complete assignments for submission. The assignments are sent to the tutor for feedback, comments and assessment by e-mail. The tutor is expected to answer the student via e-mail within two days. Some courses lead up to an exam, the final assessment before the student can obtain her certificate/diploma. Grading and information about gradesAll grades achieved during the programme are registered at NKI and the student has access to a survey of all personal registered grades on her personal web page. Each tutor can register grades directly on the web. The tutor decides the final course grade according to the student’s achievements – submitted assignments, contribution in forums etc,. Bulletin board and the Internet college newspaperNews from NKI is given on the bulletin board on the web site. NKI publishes regularly an Internet newspaper for students, which also is accessible to the general public.Resources and library servicesNKI has a library with full time staff as a service to both full time and part time students. The e-learning courses are designed so that the most important and necessary literature and learning resources are available on the Internet or they are sent to the student at enrolment or.

9

If the student would need other materials these can be supplied, or the student would be helped on how to get hold of such materials.Students’ association and integration into the teaching institutionFor many years NKI helped distance student practically and financially to organise a ‘distance students’ association’. The association was closed down some years ago. It proved difficult over time to keep the association active. For most part time e-learning students their role as students seems to be more peripheral and provisional than other aspects of their lives such as family, job and social engagements. This fact does not make student support services less important, and NKI sees most activities as efforts make students feel attached to the institution. ExaminationsThe NKI Internet College includes a system for examination. Students may take their exams centrally at NKI, at co-operating local institutions or individually at a local school. Students abroad may take their exam at a Norwegian embassy or consulate.

GraduationCourse diplomas and certificateStudents who have completed a course or programme are entitled to receive a course or programme diploma. If there are, as in most cases, mandatory examinations as part of the programme, students who have passed all examinations will receive a certificate.Graduation ceremonyEfforts are made to make graduation to a memorable event. In co-operation with local study organisations or employers/employers’ association NKI arranges graduation ceremonies for students who have completed certain programmes successfully.

After graduationAdvice on further studyWhen a student has completed her course or programme NKI takes contact, by ordinary mail of e-mail, and supplies information and advice on further learning possibilities. Such information may be sent also at later stages.AlumniNKI does not organise an alumni association for former students. However, we try to keep contact with students by supplying information on the web pages of general interest for life long learners to spur students and former students to apply NKI web pages as a central source of information on learning and competence development.

10

Research MethodologyAs mentioned, this research was carried out as one important activity of the Socrates Minerva project, Student Support Services in e-Learning. The other partners of the project have carried out similar surveys. The questionnaire was developed in co-operation according to a ‘grid of possible student support services during the different phases of an e-learning process’.

Qualitative and quantitative surveyWe have continuously carried out quantitative surveys among NKI distance online learners. In our experience quantitative questionnaires may be well suitable for getting overviews and trend developments concerning customer attitudes and satisfaction, but often these studies are less suitable for collecting more valid information for revisions and quality development. We have also experiences that students are less and less willing to answer questionnaires, response rates tend to become quite low. Thus, in this project we had a dual aim, firstly we wished to collect information that was comparable between the partner institutions through the quantitative instrument developed co-operatively. And – we also wished to collect more substantial information as basis for quality development of our own support systems for e-learning.

Thus we decided to apply the questionnaire as developed by the partners and to construct a number of open ended questions to get more comprehensive verbal responses. The common project questionnaire is presented as Appendix 1, and our additional questions as Appendix 2.Consequently, we chose to collect both quantitative and qualitative data.

Collection of dataThe method chosen for data collection was a structured telephone interview. The questionnaire was originally constructed to be used as a paper and pencil questionnaire to be distributed to students via mail or at a learning site. At NKI we chose to contact the students by telephone and fill in the questionnaire during the conversation. The open questions were asked at the end of the interview.

Qualitative interviews require generally that the interviewer is more competent and experienced than what is required in quantitative surveys. This is because the interviewer should be able to take advantage of the situations that arise during the interview to get as much valid information as possible. In many ways the interviewer is an instrument and the competence of the interviewer is fundamental for the validity and reliability of the instrument. In this study 8 different interviewers with much varying experience and competence carried out the interviews. The interviewer group consisted of 4 people from the R & D Department, 3 from the Department for Student Support and Counselling, and 1 from the Marketing Department. Before starting the interviews all interviewers met for instruction and discussion to make the interviews as similar as possible. Before this preparation meeting the researcher responsible for the evaluation project revised the preliminary questionnaire based on a try out 6 interviewees.

The reason for engaging a group of interviewers was to firstly to involve different persons and departments NKI with different responsibilities in the project and its focal issues to get maximum force in quality development of student support services. Secondly, we hoped that people with different experiences and responsibilities would be attentive to different problems that could be mentioned by the interviewees. Using a number of interviewers with different

11

backgrounds and areas of focal attention may also increase validity and reliability of the results. On the other hand, there is no doubt that the experience and competence in telephone interviewing varied a lot among the people involved. When reading the answers (Appendix 2), one sees clearly that it has been difficult to get the interviewees to give extensive information. They give a general impression of having had few problems and that they generally are satisfied with their experiences. So in many instances the open questions did not produce much more information that the attitude scales.

Selection of intervieweesWe decided to interview a sample of students with recent experiences from the NKI Internet College and ordered a list of all students that either had completed an e-learning study programme during the last 6 months, or were actively studying defined by have submitted at least on assignment during the last 6 months. For practical reasons we included only students that were registered in a programme and not single course students. The lists were distributed among the 8 interviewers who selected randomly whom to ring, but try to reach students studying different types of programmes at different levels. Those 47 active and completed students who were reached by the 8 interviewers during a 3 hours period one afternoon were included in the sample. Only some very few students claimed to be too busy to be interviewed at that particular time when they were reached on the phone. Only one student gave a negative impression and refused to be interviewed, also with reference to difficulties with time available.

Data processing and interpreting answersThe student answers both on the quantitative and qualitative part of the interview were recorded on individual questionnaire word document forms during the interviews. The quantitative data was processed by the FOSS (File Oriented Statistical System) programme package (Amundsen 2001). The data from the quantitative part of the questionnaire is presented in tables where ‘Importance’ of the support elements is presented together with the ‘Satisfaction’ with the same elements. The mean values of the students’ answers are presented in connection with each table. (Note that the scale for ‘Satisfaction’ goes from 1-4 with a mean value of the scale 2.5 (as “not used” is not included when calculating mean values) while the ‘Importance’ scale goes from 1-5 with a mean value of the scale 3.0.)

Open answers have been interpreted by the researchers during writing of this report. In the following chapter the results are presented with quantitative data integrated with discussion and comments.

Results - need for and satisfaction with student support services

The Information PhaseThe quantitative questionnaire contained questions on the need for and satisfaction with support elements such as information about course availability as e-learning, about the total programme and individual courses, pricing, financial questions such as payment schemes, loans and grants, technical questions, possibilities of contact with student advisors, web information with FAQ pages etc.

The structured interview covered the same areas on how the students had experienced the information and their contact with NKI advisors. It was very clear that thorough information

12

on all aspects of the e-learning programme before enrolling is seen as ‘important’ or ‘very important’. Generally, the students were also very satisfied with the information they had received. Both the quantitative and qualitative part of the interview showed that the possibility for direct personal contact with advisors is considered to be important, and the students were also generally very satisfied with this support element, e. g.: “Very satisfied, the advisor was very nice on the telephone, and I received the help and everything I needed” or “I was in contact and got answer from people that knew their job”.

Registration Phase/Start-up PhaseThe questionnaire and interview on the start-up phase contained questions concerning real time technical support services, registration issues, and online learning techniques. Research has shown that the first phase of studies can be critical for study success or drop out from distance study (see e.g. Rekkedal 1972). There is reason to believe that a similar relationship is present in e-learning.

NKI has put large emphasis on technical support services by e-mail and telephone and on systematic follow-up of newly enrolled students. In this study a large majority found technical support to be important, but, in fact, quite few said that these services were ‘very important’. Most of the students had not actually needed technical support. Those, who had used the support service, were generally satisfied. Concerning online learning techniques it seems that the students fall into two separate groups, either they find this support element to be really important or not to be important. Very few are indifferent to this question. NKI has for many years offered an introductory course in study techniques to students taking lover level courses. It seemed clearly that respondents with low-level previous education and little experience from independent study would have liked to get more support concerning learning techniques: “I missed some information about study techniques. That was lacking”. On an open question on whether the start of studies functioned satisfactory, the majority answered positively. However, some had experienced problems. Especially, it seems to be frustrating if something is lacking in the package of learning materials at the start: “Some books were delayed. I lost some motivation as it took some weeks to get all the learning materials”. Some students also miss a closer follow-up scheme in the start-up phase: “Some more deadlines and follow-up in the starting phase would have been good, so that one does get started. It is too easy to postpone the first necessary efforts”.

Learning PhaseThe support elements covered concerning the learning phase was access to and support from student advisors, tutor access in general, the possibility of contacting tutors on phone and e-mail, feedback on assignments, possibility to communicate with other students on e-mail and phone and discussion forums. This study confirms previous NKI research that easy access to the tutor and quick feedback on assignments are ‘very important’, while interaction with students is seen as less important by many. The students seem to be very pleased with support and help from their personal student advisor at NKI during their studies. Very few students find communication with fellow students as ‘very important’ for their learning. The students seem to be ‘satisfied’ and to a large degree ‘very satisfied’ with the services they have said to be most important, i.e. communication with their tutor(s) and speed and quality of feedback on assignments. It should be noted, however, that a large group of students who have experience with more than one tutor, report large differences, and often they have been dissatisfied with at least one of their tutors. It also seems that the tutors, who receive low ratings, supply low quality substance in their feedback, and they are also slower in returning assignments. There is certainly a challenge of following up tutors better to assure an evenly

13

high and more predictable standard of tutor work. And – although communication with fellow students are rated as being less important, there is need for developments that make the course discussion forums function better that they do today for students who wish to take advantage of communication, discussion and collaboration with fellow students.

Graduation and After GraduationAs not all students had completed their study programme, we could not expect that the question was relevant for all. On the other hand, many students may have had need for information on requirements for course certification, requirements for access to exams, procedures in connection with completion etc, and further studies before they actually have completed the programme. 75 percent looked at information and advice concerning accreditation and further study as important. Of those who have received/used such advice (22 students (49%)), only 3 students expressed dissatisfaction with the advice they had got.

Summary and conclusionsThe overall impression from both the quantitative and qualitative part of the evaluation survey on support services in e-learning at NKI was that the students find all the support services supplied as important or very important.

The table below gives an overview of mean values for importance and satisfaction with all the support elements examined in the qualitative part of the questionnaire.

Table 1. Mean values for Importance and Satisfaction with the support elements.Support element MImportance MSatisfaction

Information regarding course availability 4.47 3.39Information regarding course or module content 4.56 3.47Information regarding the larger programme to which the courses belong 4.41 3.25Information regarding pricing 4.11 3.45Information relating to course costs, grants etc. 3.49 3.18Possibility to contact NKI by phone, e-mail etc. 4.41 3.39Information on the web on registration, access etc. 3.72 3.17Access to real time technical support services 4.04 3.11Support regarding registration issues 3.84 3.43Information regarding online learning techniques 3.91 2.79Possibility to contact tutors via e-mail telephone etc. 4.65 3.59Possibility to other students via e-mail telephone etc. 3.20 3.16Discussion forums/bulletin boards 3.60 3.00Online tutorials 4.18 3.39Tutor access 4.70 3.31Feedback on assignments submitted 4.89 3.52Advice on accreditation, certification and further study 3.78 3.00Importance Scale: Not important at all=1, Unimportant=2, Don’t know=3, Important=4, Very important=5, Scale Mean=3.00 Satisfaction Scale: Very dissatisfied=1, Dissatisfied=2, Satisfied=3, Very satisfied=4, Scale Mean=2.50

14

Figure 3 below shows the relationship between mean values of answers to the questions of ‘Importance’ of each support element and the students’ ‘satisfaction’ with the same support element. The figure clearly illustrates that the students express a high degree of satisfaction given by all the mean values, except one, being 3 or higher. The figure also clearly illustrates that there is a relationship between importance and satisfaction showing that generally the students are most satisfied with the support elements that they perceive to be most important.

Figure 3. Relationship between “Usefulness/Importance” of the support elements and “Satisfaction” with the support elements.

In line with the fact that the students express a large degree of satisfaction with the support services experienced from the prospective phase until graduation, the following quotations are typical answers to the final question on overall opinion on support services:

“I am very satisfied. I am impressed by the totality of services. I thought Internet based learning was complicated until I started my studies at NKI.”“ All in all I am satisfied. According to my experiences I could take another course – and recommend it to others.”“ I am satisfied. One can always take contact with the tutor and receive good answers. Specifically the tutor is to great help.”“ I am very satisfied. I have received good information, and I know that there are people there who answer when I need it.”

15

“I have absolutely been satisfied – I will gladly return to other courses if I decide to continue to study.”

The survey ends with the following recommendations:

Support services offered to the NKI e-learners today are generally seen as important and satisfactory by the students – thus, the developments should continue according to the same philosophy of serving the individual learner and his/her need for flexibility.Priory should be given to:

Close follow up and support during the start-up phase concerning how to get started and how to make personal progression plans.

Introduction to efficient techniques for online learning, specifically for students with little experience from independent learning.

Follow-up and guidance of tutors who do not satisfy requirements concerning turn-around time and quality of comments/feedback to students.

Developing course forums to function better for students who want more interaction with fellow students, without requiring too much participation for students who prefer more individual studies.

Postscript – Some recent developments during the projectBased on previous experiences, research and strategic priorities, and on the project evaluation study presented here, these major system developments have been undertaken during the last few months to increase the quality of the student support system:

Student planning and follow-up systemAn important facility was introduced in May 2004, “The Student Planning and Follow-up System”. On the student’s personal page the student is recommended to plan his/her study schedule/progression by filling in planned dates for submissions. These dates may, of course, be changed, at any time. See figure 4, which shows the courses that the student is studying at the moment including planned submission dates, and courses completed with final grades.

Details are presented in figure 5. One can see that in the first course, the student submitted the last assignment 21.06.04. It is still shown as a link, which means that the assignment has not been evaluated and can still be revised by clicking the link. The previous assignment was planned to be due the 6th June, was submitted 28th May and was registered 1st June with the grade 6.

The system gives possibilities for the student to register in and out of classes and forum conferences. It also makes it possible to inform fellow students about progression of other students to make co-operation between students at the same stage in the courses more efficient. It also caters for information to tutors when they may expect to receive assignments and how many assignments to expect from students in different courses during specified times. The system also opens for automatic e-letters welcoming students into classes and Forums when the student is ready for studying a specified course. Further, the system automatically sends reminders and follow-up letters to students who are behind their personal study schedule.

16

Figure 4. Planning facility on the student’s main page

Web-based submission systemIn the second-generation distance education system with assignments for submission sent by ordinary mail, the corrected and commented assignments were sent via NKI for possible control of tutor comments and control of turn-around time. Until now, submissions by e-learning students have been sent directly to and from the tutor as e-mail with or without attachments. May 2004 NKI also launched the web-based submission system giving new and better possibilities for quality control of tutors’ work and turn-around time. As seen from the student’s planning page there is a button for submission. E.g. when clicking the button for the submission of Unit 5 of the SPICE 601 course, which is due 30.06.04, the page shown in figure 6 appears.

The student normally would submit his/her assignment as one or more attachments possibly with a message to the tutor. The submitted assignment and also the commented and graded assignment are stored in a database. The stored files can be used for portfolio grading and other purposes. The tutor may also register grades directly into the SESAM Learning Management system by filling in the grad in the e-mail from the student.

Course ForumsThe evaluation study also demonstrated that many students were not fully satisfied with how course forums functioned. During May and June 2003 we also have developed administrative system solutions to survey course forums. The system now gives reports for every forum and better possibilities for following up tutors and students and how forums actually function and what goes on in the forums. The forum participants have also got a possibility to organise the forums and themes according to their own priorities. However, as a result of NKI’s emphasis on individual flexibility and student autonomy we may expect that many students will find that forum discussion and interaction actually interfere with their learning schedules and

17

possibly learning style. Forum activity is a means for learning and not an end in itself. The aim is authentic learning and participation in the course forum must be perceived by the individual to foster this aim in an efficient way.

Figure 5. Detailed view of the student’s planned schedule.

18

Link to submission not received back from tutor

Last submission

Planned submission date for Unit 5

Button for submission page for SPICE 601 Unit 5

Figure 6. Web page for submission of Unit 5 in the course, SPICE 601.

Accessibility, introduction to study techniques and dyslexiaDuring the project period NKI has put great resources into making e-learning universally accessible by developing solutions for synthetically produces sound specifically for people with visual handicaps and reading difficulties. Thus, after experimenting with a number of different software solutions (see Mortensen 2003, Paulsen 2003, pp. 312-313, Mancinelli et al. 2004), we introduced summer 2004 a system for “Talking web pages”, which reads all the web pages of the NKI Internet College with an acceptable (?) synthetic voice. The system also makes it possible for the students to store the pages as mp3 files.

Figure 7. Web system for synthetic voice generation.

Parallel with these developments we also carry out a project on dyslexia – or adapting the organisation, systems and support systems to help students with reading and writing

19

disabilities to succeed in distance education and e-learning. Also the NKI course in study techniques for distance study is under revision to include e-learning study techniques, as this was one aspect found not satisfactory attended to especially for students with low previous education and little experience with adult study.

ReferencesAmundsen, A. (2001): FOSS – Fil-Orientert Statistikk-System. Veiledning for brukere.

Aoki, K. & Pogroszewski, D. (1998): Virtual University Reference Model: A Guide to Delivering Education and Support Services to the Distance Learner. http://www.westga.edu/~distance/aoki13.html

Bajtelsmith, J. W. (1988): “Predicting Distance Learning Dropouts: Testing a Conceptual Model of Attrition in Distance Education”. Report to the ICDE Research Committee, presented at the 14th ICDE World Conference in Oslo, 1988.

Holmberg, B. (1960): On the methods of teaching by correspondence. Lunds universitets årsskrift. Lund: Gleerup

Holmberg, B. (2001): A Theory of Distance Education Based on Empathy (unpublished paper).

Mancinelli, E., Moretti, M., Dondi, C., Delrio, C. & Slaatto, T. (2004): E-learning – me included. How to use e-learning as a tool for social inclusion. Report S2Net Project. Bologna: Scienter and Norwegian Association for Distance Education.

McConnell, D. (2000): Implementing Computer Supported Cooperative Learning. London/Sterling: KoganPage.

Moore, M. G. (1991): Editorial: Distance Education Theory. Am. J. of dist. Ed. 5, 3, pp. 1-6.

Moore, M. G. (1993): Theory of transactional distance. In: Keegan, D. (ed.): Theoretical Principles of Distance education. London/New York: Routledge.

Mortensen, I. (2003): Universell tilrettelegging av nettbasert studium I “Ledelse og organisasjon. Report to Norway Opening Universities. Bekkestua: NKI. http://www.nettskolen.com/forskning/soffrapport_universell.pdf Retrieved 24th June 2004.

Paulsen, M. F. (1993): The Hexagon Of Cooperative Freedom: A Distance Education Theory Attuned to Computer Conferencing. DEOSNEWS Vol. 3 No. 2.

Paulsen, M. F. (1998): Teaching Techniques for Computer-mediated Communication, Ann Arbor, Mi, UMI Dissertation Services. Pages: 295.

Paulsen , M. F. (2000): Online Education, An International Analysis of Web-based Education and Strategic Recommendations for Decision Makers, NKI. http://www.nettskolen.com/pub/artikkel.xsql?artid=131 Retrieved 24th June 2004.

Paulsen, M. F. (2003): Online Education and Learning Management Systems. Global e-learning in a Scandinavian perspective. Bekkestua: NKI.

20

Rekkedal, T. (1985): The personal tutor/counsellor in the system of distance education. Project report 2: Final report. Oslo: NKI.

Rekkedal, T. (1993): Evaluering av Teknisk fagskole. Group interview with exam candidates. Internal evaluation paper.

Rekkedal, T. (1998): Courses on the WWW - Student Experiences and Attitudes Towards WWW Courses. An Evaluation Report Written for the MMWWW Leonardo Online Training Project. http://www.nki.no/eeileo/research/eei/Rekkeval.htm Retrieved 24th June 2004.

Rekkedal. T. (1999): Courses on the WWW - Student Experiences and Attitudes Towards WWW Courses - II (1999). Evaluation Report Written for the MMWWW Leonardo Online Training Project. http://www.nettskolen.com/pub/artikkel.xsql?artid=147 Retrieved 24th June 2004.

Rekkedal, T. (2001): The NKI Internet College Based on 30 years of research in distance education. ICDE Prize Lecture. Holmenkollen Park Hotel, Oslo. 3rd December 2001. http://www.nettskolen.com/forskning/ICDE%20Prize%20Lecture.pdf Retrieved 24th June 2004.

Rekkedal, T. & Møystad, E. (2003): Evaluering av Interiørskolen 2003. Evalueringsrapport. Bekkestua: NKI. http://www.nettskolen.com/forskning/evaluering/InteriorEval2003.pdf Retrieved 24th June 2004.

Rekkedal, T. & Paulsen, M. F. (1998): NKI Internet Students - Experiences and Attitudes. The Third Generation NKI Electronic College. An Evaluation Report Written for the Leonardo Online Training Project. http://www.nettskolen.com/pub/artikkel.xsql?artid=149 Retrieved 24th June 2004.

Rekkedal, T. & Qvist-Eriksen, S. (2003): Internet Based E-learning, Pedagogy and Support Systems. http://learning.ericsson.net/socrates/doc/norway.doc Retrieved 24th June 2004.

Sewart, D. (1978): Continuity of concern for students in a system of learning at a distance. ZIFF. Hagen: FernUniversität.

Sewart, D. (1993): Student support systems in distance education. Open Learning, 8(3), 3-12.

Sewart, D. (2001): The future for services to students. In Student Services at the UK Open University. Papers presented at the 20th ICDE World Conference, Düsseldorf, Germany (pp. 7-12). Milton Keynes: The Open University.

Sjaastad, J., Akre, A.-K., L. & Rekkedal, T. (2003): NKI Distance Education - Case Study Student Support Systems in E-Learning. http://learning.ericsson.net/socrates/doc/norwayp2.doc Retrieved 24th June 2004.

Súilleabháin, G. O. (2003): Principles, Structure and Framework of e-Learning. http://learning.ericsson.net/socrates/doc/cork.doc Retrieved 24th June 2004.

21

Thorpe, M. (2001): Learner support: A new model for online teaching and learning. Paper to the 20th ICDE World Conference, Düsseldorf.

22