videoconferencing in a language learning application

interacting with Computers ~018 no 2 (1996) 207-217

Videoconferencing in a language learning application

Patrick McAndrew, Sandra P. Foubister and Terry Mayes*

Can videoconferencing substitute for face-to-face contact sufficiently well for collaborative task-based learning to take place at a distance? The paper reports on the use of videoconferencing in the context of remote users’ learning of a foreign language. Video-mediated communication was placed at the centre of an integrated system for the learning of business French, and students used it in performing collaborative role-plays, leading to the joint enactment of a communication task (such as the setting up of a subsidiary company in the Nord Pas de Calais). Observations of the system in real use are reported, and some positive conclusions are drawn about the potential role of videoconferencing in language learning.

Keywords: collaborative task-based learning, videoconferencing

Broadband communications allow many users to communicate, but there are few applications that depend crucially on this. In that sense, broadband communications can sometimes appear to be a technology looking for an application. However, where human communication is central to the performance of a task, and especially where the quality of that communication is a determinant of task performance, then we may expect to find a real added-value. The project described here has developed an application for language learning in which video-mediated communication is central. The main point of the trial described below was to demonstrate that the system is effective and acceptable as an alternative to face-to-face communication in the learning of a language. It is worth noting that it was not an aim of the project to demonstrate that there was no difference between face-to-face and video-mediated communication, but that the overall system of material and communication was sufficient for effective task performance. Since there are obvious advantages in being able to substitute

Institute for Computer Based Learning, Heriot-Watt University, Riccarton, Edinburgh EH14 4AS, UK.

Tel: +44 (0)131451 3286. Fax: +44 (0)1314513283. E-mail: [email protected] [email protected] *Glasgow Caledonian University, 3rd Floor, St. Andrew House, 141 West Nile Street, Glasgow G12RN, UK. Tel: +44 (0)141 331 1271. Fax: +44 (0)141 332 8214. E-mail: [email protected]

0953-5438/96/$15.00 ~Q 1996 Elsevier Science B.V. All rights reserved

PII SO953-5438(96)0102%4 207

video-mediation for the requirement to be physically co-present, then the achievement of this goal would carry considerable implications for distance education and training.

Having adopted a basic pedagogical rationale for how the users would learn a language, the project needed to work out how to embody this in an advanced network-based multimedia application. It was necessary to confront many technical issues about how to find and convert suitable multimedia material, and how to allow access to that material. This system then had to be implemented on suitable hardware, with conferencing facilities on a broadband network. The resulting system incorporates servers of various kinds of multimedia course material, including video and sound, together with real-time video and audio communication tools. This paper outlines the project as a context for discussion of how the videoconferencing facility in the final system was used in the trials.

The HIPERNET project

HIPERNET is a European Union funded project “High PERformance NETworked multimedia for distributed language learning”. The project was funded under RACE @X2115), a research programme for “Research and Technology development in Advanced Communications technologies in Europe”. It involved seven partners in six countries with an additional three subcontractors. The Institute for Computer Based Learning at Heriot-Watt University (ICBL) acted as a subcontractor to Cambridge University Language Centre (CULC). ICBL’s main involvement was in the pedagogical specification, the conversion of data, and the evaluation of the system.

Technical motivation Some of HIPERNET’s aims were driven by technological needs. The system was designed to make use of an existing Metropolitan Area Network, the GRANTA Backbone Network, installed around Cambridge to connect together many of the different institutions of Cambridge University. This provided the project with access to “dark fibre”: unused fibre connections over which any chosen protocol or signal could be used. This was a strong motivation, as such access was rare at the time of specification in 1993. The project aimed to develop new high speed devices to allow data to be transferred over the network. To underpin this technical motivation, the project was tied to a language learning application with a need for the high bandwidth that the technology was designed to offer.

Language learning requirements Dialogue is important in second language learning, as, without practice and reinforcement of dialogue skills, such learning is likely to take place less efficiently, if at all. Synchronous feedback to the speaker is a crucial component of oral practice. This is direct motivation for the provision of audio connections between participants in a computer-based language learning system. It seems intuitively reasonable to assume that for dialogue to occur naturally, and with maximum effectiveness in terms of reciprocal understanding, it is helpful if

208 Interacting with Computers uol 8 no 2 (1996)

participants can also see each other, both to motivate the dialogue and to allow visual cues to be used. Communication could be between any two users, but an existing feature of the way in which language was taught at CULC was the use of an adviser. The possibility of replacing this face-to-face interaction between student and adviser was a major incentive for including videoconferencing in HIPERNET. High quality videoconferencing in conjunction with such an adviser is likely, for example, to promote good pronunciation, as the adviser and the learner can watch each other as they try to pronounce the words.

Task-based collaborative learning An early aim in the project was to use the computer to integrate multimedia sources. Motivation for learners to use the system would come through the introduction of some self-testing exercises. A “Help Desk” was envisaged as an adjunct to this, to offer text, audio and video based communication with a trained human adviser.

Reassessment of the initial design led to a change in the philosophy behind the system. Instead of the focus being the provision of material to learners, a set of tusks was designed as a central theme for HIPERNET. The tasks need to be supported by the material and the facilities available. To encourage dialogue and the use of resource material, the tasks involve role-play so that learners take part in conversations in character. Fulfilment of a task also involves creating a joint presentation incorporating such role-play. The material within the multimedia course thus forms a background resource for the fulfilment of tasks. The tasks themselves provide a reason to explore and use the system. The Help Desk communication tools offer a way to incorporate collaboration with fellow learners as well as with an adviser.

The system as implemented Employing video-mediated communication as the key functionality set the need for high-bandwidth and fast response. The implemented system emphasises the tasks, and the original concept of a Help Desk is extended to become a “Help Desk and Collaborative Workspace Environment” allowing text-, audio- and videoconferencing between learners, in addition to between the learner and adviser. The redesign also incorporates the sharing of materials, to allow each learner to see the same part of the available material, though for technical reasons the implementation used in the final trials did not in fact use this course sharing.

Videoconferencing is central to the use of the system. Although the multimedia learning materials may be explored independently by the learners, fulfilment of the tasks depends on collaboration. This is both for the management of the allocation of subtasks, and for the rehearsal of the role-play dialogue for the final, assessed, presentation.

The multimedia course material is provided in an integrated shell. The material comes from French Means Business (King, 1993), a BBC produced course, authored by Anny Ring from CULC. This is linked to a multimedia glossary which is limited in scope, but directly relevant to the course material. The glossary includes spoken pronunciation of each word and in some cases a graphic, or even illustrative material extracted from the course video.

McAndrew et al. 209

The two tasks incorporated into the system involve the role-play of simulated business situations. These require the participants to be familiar with specific aspects of French business culture: in one case the Minitel information system, in the other the facilities for encouraging the setting up of commercial companies in the Nord Pas de Calais, and Lille in particular. The participants were to prepare the role-play for presentation to French teachers/assessors, who might also assume roles if necessary. Description of the task requirements are available to the users on-line. Each task has links to component subtasks, and these have further links into the course material.

Technologies for HIPERNET Low-cost PC clients were used (486DX2/66) with 16Mb of memory. Commercial cards with customised software were fitted to these to allow delivery of multimedia and conferencing. Hybrid servers with a Unix workstation front- end fetching data from a PC based back-end server were used to deliver the material. The material itself was translated to linked multimedia following an SGML DTD (Standard Generalized Markup Language Document Type Defini- tion), with audio and video digitised into MPEG-1 format.

Videoconferencing between the clients used a Bitfield I3261 conferencing card with customised software to operate over the network. Communication over the network was ATM using two different configurations developed within the project. This operated at either 1.25 Gbit/x or 622 Mbit/s for the backbone connections with final delivery to each workstation at 25.6Mbit/s. Each workstation was equipped with a flexicam, desktop microphone and head- phones. The videoconferencing software offered functions that included a self- view, an audio-only option, and settings of sensitivity and audio level.

The HIPERNET system was designed to work in a very controlled environment but developments in the networking infrastructure around Europe mean that alternative ways of implementing the methods of HIPERNET can be considered. In the development process a conversion from the SGML DTD markup to HTML mark-up used by Web browsers was produced. This means that the same material used in the HIPERNET course can be presented world-wide working directly from the same mark-up. The interface is different and at this stage the multimedia delivery is not satisfactory, but an indication of the course element of the system can be seen at: http:/ /www.icbl.hw.ac.uk/hipernet.

Potentially this can be linked with the Internet conferencing tools which are becoming available for each of text-, audio- and videoconferencing. The Internet does not offer guarantees of bandwidth, but in areas where the infrastructure is known, for example SuperJANET and the Scottish MANS, it may be that reasonable performance will result. This approach to delivery is being examined within other projects to take advantage of the mark-up to allow course material to be converted once only allowing presentation to be updated quickly and uniformly as technology develops.

User trials Ideally formative evaluation would have taken place, with a refinement cycle of trials with potential users in conjunction with development phases. Unfortunately

210 Interacting with Computers ~018 no 2 (1996)

in this case much of the final system depended on implementation on the hardware at the trial site in Cambridge. The only chance to assess the fully operational system therefore occurred towards the end of the project within a formal six-week trial. Nevertheless there had been some opportunity to test separate elements such as early versions of the Help Desk interface and of the Application Shell and presentation of course material. Improvements deriving from these earlier assessments were incorporated into the trial system wherever possible.

The trials themselves took place in Cambridge University over six weeks during October and November 1995. Six workstations were placed in three locations: Cambridge University Language Centre, the Engineering Department, and Churchill College, these sites separated by about 7km. An adviser workstation was also installed, but only operated towards the end of the trials. Twenty-four students were recruited and asked to work in pairs at different sites.

To allow assessment of the effect of computer mediation, both of the provision of course material and of the communication between participants, a ‘no-computer condition’ was used. In this condition students worked together at the same location with traditional materials, rather than at different sites with computer- accessed materials and videoconferencing.

The two tasks and two conditions allowed a balanced repeated measures experimental design, though it was anticipated that understanding of the language learning process would benefit more from qualitative measures, including interviews and questionnaires, than from statistical results deriving from the experimental structure. In all cases students would have experience of both systems, so that judgements could be expressed regarding them, and comparisons be drawn between them.

Videoconferencing versus face-to-face

For the purpose of this paper we are extracting and summarising here results from the user trials that relate to videoconferencing and the overall performance of the students in the HIPERNET system.

Investigative techniques A mixture of quantitative and qualitative measures was used both in the assessment generally, and in the assessment of the use of the videoconferencing facility in particular. By the end of the trials all students had experienced the two conditions, of working remotely from their partner and videoconferencing, and of working together with their partner at the same location (i.e. face-to-face discussions in the same room).

The presentations were assessed by French language experts at CULC. No significant difference was found between the presentation scores of pairs of students who had used HIPERNET and those who had worked under the no-computer condition. This suggests that collaborative task-based learning is adequately supported by videoconferencing, with the important implication that such methods may be appropriate for distance learning.

Time-lining was used to structure some of the observation of the HIPERNET learning sessions. There is a theory that people spend more time using features of

McAndrew et al. 211

a system that are the most “acceptable” in terms of usefulness, usability, and making work easier (Collis and Verwijs, 1995). During the trials over 55 hours of use of the system were observed in this way, taken from 64 student-sessions. Occasionally both ends of a collaborating pair were simultaneously under observation. For each five-minute period all activities were noted. More than one activity may occur at the same time, for example reading a text while listening to the audio version, and several activities may occur consecutively during each five- minute time-slice. It turned out that 34% of the time-slices included videoconferencing. This was second only to those that included reading available text (43%), though this figure may be artificially raised as, naturally, there was something displayed whenever the system was in use. These figures match well to the design of the system, in that it is to be expected that, primarily, learners will interact with the material through the text, and communicate with each other using the videoconferencing facility.

Students were asked to rate various features and potential features of HIPERNET on a five-point scale ranging from ‘very useful’ (5) to ‘quite unnecessary’ (1). The mean usefulness rating for videoconferencing was the highest, at 4.71. This was strong confirmation that they appreciated this feature, supporting Tang and Isaacs’ conclusion that, for multimedia technology to support collaborative work, “Users want video connections” (Tang and Isaacs, 1995). Our set-up did not allow a direct comparison with audio-only communication, as this was but a little used option of the videoconferencing facility, nor with text communication, as this was intended, and used, as a back- up, to enable some contact when the video link failed.

Results It was not expected that any improvement in the measures of competence in French would be detectable after just three two-hour sessions. Ratings on a 5-point scale, made on the occasion of the presentations, for each of the measures of pronunciation, fluency, grammar and confidence were subjected to ANOVA (see Table 1). The analysis was applied to a comparison of the assessment after each of the three week blocks. The overall result here is quite clear: with one minor exception, there was no detectable difference between the HIPERNET and no-computer conditions, even after the effect of the order in which the conditions were used was removed.

When the subjects made their role-play presentations in pairs, there was no difference between the presentation marks for the two conditions (i.e. the presentation after using HIPERNET was not distinguishable by the assessors from the presentation after the no-computer condition).

Because of the expectation that competence ratings would be insensitive to any effects which might be detectable over the short time period of the trial, measures of confidence were also included. There were self-assessments of confidence in the initial and final questionnaires, and assessors rated confidence at the beginning of the trials and at both presentations. Confidence has been used before in conjunction with detailed learning objectives (Draper et al., 1994). Here the rationale for such a measure was based on an expectation that only the individual learner would be capable of detecting the kind of subtle benefits that might

212 Interacting with Computers vol 8 no 2 (1996)

Table 1. Statistical analysis from HIPERNET trials 1

Grammar Confidence

Mean assessment HIPERNET Control Mean assessment HIPERNET Control

after: after: Block 1 3.42 3.33 Block 1 3.42 3.75 Block 2 2.92 2.92 Block 2 3.42 3.92

HIPERNET v control: F(l, 22) = 0.07 ns HIPEFWET v control: F(1,22) = 5.85, p < 0.05 Order: F&22) = 3.27 ns Order: F(1,22) = 0.04 ns Interaction F(1,22) = 0.07 ns Interaction F(l, 22) = 0.23 ns

Pronunciation Presentations

Mean assessment HIPEFWET Control

after: Block 1 3.00 3.00 Block 2 2.83 3.25

HlPERNET v control: F( 1,22) = 1.92 ns Order: F(1,22) = 0.36 ns Interaction F(1: 22) = 0.08 ns

Mean assessment HIPERNET Control


HIPERNET v control: F(1,lO) = 0.25 ns Order: F(l,lO) = 0.01 ns Interaction F(l,lO) = 0.69 ns

Fluency Self-Confidence Ratings

Mean assessment HIPERNET Control


HIPERNET v control: F(1,22) = 0.00 ns Order: F( 1,22) = 0.45 ns Interaction F(1,22) = 2.28 ns

Before After

HII’ERNET First 21.08 21.83 HIPEIWET Second 20.67 23.33

Before/after: F(1,22) = 6.34; p < 0.02 Order: F(1,22) = 0.13 ns Interaction F(1,22) = 2.00 ns 1

emerge over so short a time period. For the self-rating, subjects were given a series of seven hypothetical tasks, ranging from booking a hotel room to ‘phoning a plumber, and were asked to rate their confidence in carrying these out on a five- point scale that ranged from “no problem” to “no way!“. The self-ratings of confidence did increase significantly between the first and second questionnaires (F(1,22) = 6.34; p < 0.02).

There was a greater increase in assessor rated confidence over the face-to-face than over the videoconferencing part of the trials (F(l, 22) = 5.85; p < 0.05). These ratings looked for evidence in terms of eye-contact and repair strategies. However the instability of the system during the trials is particularly relevant here, as breakdown in communication caused by system failure could well contribute to a lack of any increase in confidence in communication. Thus, although the difference found may be due to some real advantage in face-to-face, further work is needed to confirm this.

Many of the advantages and disadvantages noted in videoconferencing were reported in a structured interview which took place after the first three weeks of the trials, and further revealing comments were offered in the context of the final

McAndrew et al. 213

questionnaire at the end of the trials. The rest of this section summarises the points that emerged, under the headings: benefits of videoconferencing; problems with videoconferencing advantages of co-location.

Benefits of videoconferencing Videoconferencing is vital to the collaboration needed in the fulfilment of these tasks, where face-to-face communication is not possible. Typical comments included: “. _ . it came to the point where videoconferencing was critical to the session. . .” and “This was indispensable! It was really important to be able to get in touch with the partner in order to co-ordinate and rehearse our presentation”. The participants have to make joint decisions, such as the mutual allocation of subtasks and details of the roles they will assume in the role-play. They also need to rehearse their presentation, which specifically includes a requirement for them to communicate with each other in French. In addition to this the students noted some definite benefits of videoconferencing:

It allows a productive combination of working alone, and working collaboratively. When students are working together in the same room, they are constantly aware of their colleague’s requirements, and take this into account. In remote working the problematic aspects of physical co-working are less prominent. Students can explore the multimedia material at their own pace, listen again to sections that they wanted particularly to hear, and to words pronounced in the multimedia dictionary. After a pair who are working together have decided on an allocation of subtasks, there is a period where they need to work independently, and it is appropriate at this stage also to be working apart. One student commented, “Working with a partner in the same room is not very productive. A partner on the other end of a computer lets you get more work done!“.

The interaction between students is focused because the connection is not always there. On the other hand when they do make contact there is motivation to use the time efficiently, in a focused way. The video call is made for a specific purpose: for example to discuss progress, to clarify a particular point or to rehearse part of the role-play needed for task fulfilment.

It is perceived as motivating. The students explicitly enjoyed the novelty of videoconferencing. Their comments about the facility were almost invariably positive, e.g. “It was great fun!“.

It eliminates physical barriers. Many students mentioned that videoconferencing eliminates physical barriers, and appreciated not having to travel across Cambridge to their partner’s location. Thus even where physical co-location is possible there may be advantages, for collaborative task based learning, in videoconferencing at a distance instead.

Problems with videoconferencing Some of the problems with videoconferencing reflected the occasional instability of the system, others the particular set up. Although the facility served its purpose


in permitting the collaboration necessary for the fulfilment of a task, several minor difficulties were noted:

There is lack of eye contact. The cameras were movable, and the videoconferencing software allowed a “self-view” so that subjects could ensure that their image was appropriate for display. However it was not possible to position the camera to emulate eye-contact without partially obscuring the screen of the monitor. It was suggested to the students that they look at the camera when speaking, and at the screen when listening. This was generally successful, but in a few cases the impression of lack of eye contact when speaking was distracting to them. The problem could be overcome in various ways, including some kind of half-silvered mirror set up such as that described in another paper in this issue (O’Malley et al, 1996).

It is physically demanding to have to wear a headset and look at the monitor. Some students were happy to wear a headset, others found it noticeably uncomfortable. The need for using a headset was to eliminate audio feedback.

Cognitive load in having to make contact to say something, The counterpart to the claimed advantage of focused communication with the video-mediated condition, is that the user has to expend cognitive effort in contacting their partner each time.

It is not yet easy to share material. As the course sharing element of HIPERNET had to be dropped for the trials, the students sometimes had problems sharing materials. They could navigate slowly through course material while videoconferencing, but this was a poor substitute for really seeing and hearing what was on the other’s machine.

“You can’t tell what the other person is thinking”. This reflects the view that desk-top videoconferencing, even with the VHS-quality achieved here, still falls short of face-to-face communication quality. By contrast, the advantage of video communication over audio-only was noted in similar terms by one student in the context of the structured interview: “You can tell what the other person is thinking”.

Advantages of co-location The advantages of working together at the same place to some extent complement the disadvantages of videoconferencing: the problems of lack of eye contact, need to wear a headset, need explicitly to make contact, sharing material and subtlety of communication disappear. But there also appear to be distinct bonuses:

Partners can talk to one another whenever they want. No special effort is required to make some small comment, ask a short question. The partner’s work is not going to be significantly disrupted by such communication.

It is easier to be sure of what the other person wants to do. Particularly where the students had previously been unacquainted, they felt more confident that they

McAndrew et al. 215

were working in accord when they were together rather than when videoconferencing. There seemed less chance of misunderstanding, perhaps because of the opportunity easily to clear up any such misunderstanding.

It is better for sharing information. They found it easier to compare notes and ideas, to prepare OHP foils and to practise the presentation when they were together. This would probably have been the case even if the course-sharing facility had been present in HII’ERNET. A shared whiteboard on the computer system, in conjunction with course sharing, might have gone some way to solving this problem though.

The tusk may perhaps be completed more eficiently. This cumulative set of advantages of co-location implied to some that the task could be completed more efficiently. This was probably the case, but it was not formally established in these trials.

Conclusions

An important finding from the HIPERNET trials is that the videomediated condition was rated by the learners, the advisers, and the observers, as providing effective support for the communication demands of the language learning tasks.

The students liked using the system. In answer to the question, “Given a HIPERNET system that didn’t crash, and a traditional system with CDs, videos, book and dictionaries, which would you prefer to use to learn a foreign language?” Eighteen students voted for HIPERNET and six chose the traditional method. Their comments indicated that some votes for the traditional method were motivated by frustration at the limitations of the version of the system as realised, for example, “Had there been longer video clips, more text and audio clips, then I would imagine that I would have preferred HIPERNET”. Others saw computer-mediation as a viable alternative: “The HIPERNET system is a very good alternative and could be better in certain circumstances (e.g. link ups with a French student in France)“. But there were also very positive comments such as “Anything is better than a book! Perhaps HIPERNET could be linked to France, and then the French person learns English and the English person learns French?“.

Overall, performance was neither better nor worse due to computer-mediation. Thus we can claim that learning through the HIPERNET system was as effective as in a conventional face-to-face setting. That is an encouraging result, and provides an answer to the main question posed by the project. However, to claim a null-result as any kind of finding at all would be misleading. There was insufficient data, gathered over too short a time, to expect any performance differences to show up, and almost none did. The students’ self-rated confidence in speaking French did increase significantly over the whole six-week trial period. We can take this as confirmation of the pedagogical effectiveness of the task-based approach in general, even though when the assessors were rating the apparent confidence of the learners, they rated higher for the effect of the no-computer control. This must cause us to exercise caution in our claims for the effectiveness of video-mediation.


So far as the use of videoconferencing was concerned, mostly our intuitions were confirmed by observation of students using the system. They enjoyed the facility, even when there were very obvious problems in making the connection, and the sound disappeared inexplicably, and so on. The chief exception was in relation to the need for detailed observation of the speaker’s mouth. We had expected this to be very important, but it seemed that the students rarely even looked at the speaker’s face, and to a large extent the videoconferencing facility was regarded as an enhanced audio link - they could look for visual confirmation, a nod, a smile, and they could sometimes show OHPs across the link; and of course the sense of presence was enhanced, particularly when they were rehearsing the presentation. Perhaps greater opportunity to videoconference with an adviser would have made a need to observe detailed facial movement more prominent, or perhaps pronunciation is learned predominantly through aural means.

The HIPERNET trials established that videoconferencing emulates working face-to-face sufficiently well for collaborative task-based learning to be supported. One implication of this is that it may be confidently attempted across national boundaries. This is being exploited in a follow-up project (ACTS AC109) which indeed uses learners in different countries. Many detailed questions remain to be addressed; HIPERNET has not benefited from iterative design, and the trials were unable to probe detailed questions about the differences between face-to-face and video-mediated communication. However we are able to conclude with some confidence that videoconferencing, at the least with the quality obtained with the broadband network used here, has real potential for the distance learning of languages.

Acknowledgements

This work was carried out in collaboration with the HIPERNET consortium partners: Compagnie IBM France, AND Software BV (The Netherlands), ASCOM Tech AG (Switzerland), CAP-SESA Telecom (France), GEC-Marconi Ltd. (UK), University of Cambridge Language Centre (UK) and VTT Technical Research (Finland).

References

Collis, B.A. and Verwijs, C. (1995) ‘Evaluating electronic performance support systems: A methodology focused on future use-in-practice’ Innovations in Education and Traininglnt. 32,1,23-30

Draper, SW., Brown, M.I., Edgerton, E., Henderson, F.P., McAteer, E., Smith, E.D. and Watt, H.D. (1994) ‘Observing and measuring the performance of educational technology - a TILT project report’ Technical report University of Glasgow

King, A. (1993) French Means Business BBC Books

D’Malley, C, Langton, S., Anderson, A.H., Doherty-Sneddon, G., and Bruce, V. (1996) ‘Comparison of face-to-face and video-mediated interaction’ Interacting with Computers 8, 2,177-192

Tang, J.C. and Isaacs, E.A. (1995) ‘Studies of multimedia supported collaboration’ in Emmott, S.J. (ed) Information Superhighways: Multimedia Users and Futures Academic Press, 123-160

McAndrew et al. 217

videoconferencing in a language learning application

Documents