The ESP Journal Vol. 1, No. 2 Spring 1981

A Procedure for ESP Textbook Analysis and Evaluation on Teacher Education Courses

Ray Williams

The article examines the applicability of an approach employed in the Open University’s four-stage “Group Study Skis” sessions to ESP textbook analysis and evaluation (TAE) during a teacher education course. The major factors affecting on-course TAE pro- cedure are suggested, and the OU’s approach to study skills ses- sions is set out. It is suggested that on-course TAE factors lend themselves to the OU approach.

The author describes a recent attempt to apply the four-stage OU approach to TAE during a teacher education course in Manila, the Philippines. In particular, an example is given of a TAE “script” used, and an evaluation is made of the TAE procedure in practice. It is concluded that this four-stage procedure is a distinct advance over the inflexible, predetermined “check-list” approach, and may indeed be equally applicable to other components of teacher educa- tion courses.

Introduction The range of ESP textbooks is now so wide that most courses involv-

ing teacher education - whether postgraduate applied linguistics, or limited duration, ESP teacher-training - include an element of text- book analysis and evaluation (TAE). The aim is not so much to analyse and evaluate specific textbooks, as to offer course participants an ap- proach for doing so - an approach they can continue to apply when the course is over.

Factors Affecting the TAE Procedure What are the factors that should be taken into account’when devising

a procedure for carrying out TAE on a teacher education course? I sug- gest there are three major factors to be considered:

1. Course participants bring with them a wealth of varied teaching experience, in particular in using different textbooks and locally- produced material. It is important that participants’ experiences (successes, failures, reservations, enthusiasms, etc.) in using ESP teaching material are brought to bear on the TAE situation. If a “pooled experience” approach is to be adopted, then a procedure should be used whereby participants’ differing personalities are taken into account, so that everybody is enabled to contribute to TAE, not just those who are the most confident, assertive, dogmatic, articulate, etc.

155

156 The ESP Journal ’

2. ESP textbooks have frequently originated in a particular teaching institution and have then been modified to meet the wider requirements of the published version. A globally-available course emanating from a specific local teaching situation is in a sense a con- tradiction in terms. It is therefore important that the TAE pro- cedure does not demand unrealistically high standards of the text- book concerned, or course participants may unintentionally be in- duced to become hypercritical, and as a result may even be dismissive of ESP textbooks in general. 3. Some ESP textbooks incorporate their own recommended teaching approach. The Reading asd Thinking in English series, for example, advises the teacher to:

. . . manage class activities so that as soon as the students are briefly orientated towards a stage of the practice they should be required to work independently of the teacher. In general, where questions and activities need to be discussed, have the students work in groups of 4-6 . . . . (Moore et al. 1979:xvii)

Similarly, the Teacher’s Notes for the demonstration lesson in Nucleus: General Science advocate:

. The exercise can be done by dividing the class into groups. &udents can work out the answers, correcting one another, and then a student from one group can ask another group the answer for each question. . . Group-work ensures the participation of the whole class. (Bates and Dudley-Evans 1976:3-5)

There is by no means unanimity among the ESP profession as to such a methodology. Nonetheless, series such as Reading and Thinking in English and Nucleus are influential. Such a methodology is new, often alarming, for many participants of a teacher education course. It is therefore valuable to employ a TAE procedue that attempts to reflect a student-centred ESP methodology, and so give course participants a students’-view experience of it, as well as an insight into the different form of classroom management it entails.

The Open University’s “Group Study Skills” Sessions

For a number of years, Gibbs and Northedge of the Open University’s Tuition and Counselling Research Group’ have used a procedure which may be applicable to TAE-although Gibbs and Northedge use it in a dif- ferent situation, in running study skills sessions with Open University students. Before exploring the applicability of the procedure to TAE, it is necessary to describe Gibbs and Northedge’s approach (Gibbs 1977, Northedge 1975).

The purpose of each of their sessions is to focus attention on a particular

1Britain’s Open University is a Distance Learning institution, offering a wide range of degree courses by a combination of mail, radio, TV and short residential courses. It caters for approx- imately 70,000 students a year.

A Procedure for Textbook Analysis and Evaluation 157

aspect of study skills (e.g., “Organising Your Time”, “Using Books”, “Taking Notes”). The intention is a structured, student-centred approach in which participants share experiences, allow problems to surface and be analysed, draw out criteria for success or failure in a particular area, etc. The important elements of each session are:

1. The PURPOSE of the session. 2. A SCRIPT of instructions to students, divided into various stages (see below). 3. The MATERIAL to be used in the session.

Essentially, the structure of any one of their sessions takes the following form:

Stage 1: individual work Stage 2: work in pairs Stage 3: small groups (two pairs) Stage 4: reporting back to the whole group

Stage 1 (individual work)

Northedge (197571) comments on the first stage as follows: “One of the intrinsic problems of group discussions is the multiplicity of demands made on the students. Each student is involved simultaneous- ly in (a) listening to the discussion; (b) thinking about what has been said and formulating points of his own; (c) finding-suitable entry points for his contributions; and (d) attempting to maintain an overall picture of where the discussion is heading (and if necessary intervening to change its direction).” Under such pressure, the topic often takes on a narrow perspective, drawn by a small number of dominant students. Less confident group members are inhibited from expressing an alter- native view, or feel they have nothing relevant to say. Initial individual work, on the other hand, in which each student writes notes on two or three main points, provides an entry point to the topic concerned and requires all students to formulate views.

Stage 2 (work in pairs)

Many students are reticent in groups to express views on which they think they may be attacked, or become increasingly tense while they wait for an appropriate “entry point” to the discussion, until they lose confidence in the potential value of their contribution. Paired discus- sion circumvents these “public speaking” aspects of discussion groups, avoids the possibility of one student being “wrong” and out- numbered, and makes it impossible for the shy student to stay silent. “From the point of view of the group as a whole . . . working in pairs has the advantage of involving everyone immediately . . . and of creating a buzz of conversation so that the atmosphere of the room is one of activity and exchange of ideas rather than awkward silences.” (Northedge 1975:72)

158 The ESP Journal

Stage 3 (2 x pairs) The bulk of productive discussion takes place at this stage, in pair-

ings of pairs from Stage 2. Discussion will now be more free-ranging than in pairs, but “. . . in contrast with the usual group situation, the more nervous students have the support of a partner in expressing the points they have agreed between them.” (Northedge 1975:72) A group of four will generate a wide range of ideas and is small enough for all members to be able to speak. There is no group leader, and each group evolves its own dynamics.

Stage 4 (whole class) “The aims of this stage are firstly to give a purpose to the earlier

discussion, secondly to allow the different groups to become aware of aspects of the topic which they have left uncovered, thirdly to give an opportunity for students to direct questions at the tutor, and finally to foster cohesiveness within the whole group.” (Northedge 1975:73) The tutor acts as chairman and recorder.

Application to TAE of the Open University’s Study Skills Approach

While visiting lecturer at the Third National Seminar/Workshop in Manila, the Philippines, held April/May 1980, I experimented with Gibbs and Northedge’s approach, adapting it for use as a procedure for ESP textbook analysis and evaluation. The PURPOSE, SCRIPT and MATERIAL elements of their approach were all used. Six TAE sessions were held, each concerned with one textbook, with a maximum of 36 participants per session.

The PURPOSE of TAE was self-evident - to enable course par- ticipants to assess the probable applicability of the ESP textbook con- cerned in their own institutions - and there was no need for this pur- pose to be written down. The MATERIAL, of course, was the textbook itself - flyleaf, list of contents, teaching methodology pages and a specimen unit. These materials were given to participants a day in ad- vance, and they were asked to familiarize themselves with them in preparation for TAE the following day. They were told that the TAE emphasis would be on the content of the specimen unit. Figure 1 shows an example of a SCRIPT, the one which was used for the evaluation of Nucleus: General Science. The following comments provide further details and explanation of certain points:

1. The activities of the rm your own-in pairs-in 2 x pairs stages were set out in what I judged to be increasing likelihood of difficulty/opportunity for pooled experience. Thus, the olt your own activities simply ask the participants to “note down” items of con- tent, whereas the in 2 x pairs stage deals with more discussible topics such as the recommended teaching methodology, par- ticipants’ views on the scientific content, etc. 2. Althouth the scripts for each textbook contain many activities in

A Procedure for Textbook Analysis and Evaluation 159

common, each script was slightly different, in that it reflected the content of the textbook being considered. For example, activity 2.5 was included in the script above to see whether participants spotted the incongruity of the diagrams of the very English milk bottle, and the western-style house structure in Nucleus: General Science, Unit 3. 3. The activities under each stage are “guided” in the sense that they direct participants’ attention to certain aspects of the textbook, but not to the extent of influencing their final opinion.

FIGURE 1 ESP Textbook Analysis and Evaluation

NW&US: Geneml Mena Unit 3 1. On Your Own (10 minutes)

1.1 Note down the section headings of this unit. Do you think the sections were put in this sequence for a particular reason? If so, why?

1.2 Note down the language points being taught in each section and subsection. Note also any revision of language items from preceding sections and subsections.

1.3 Note down the exercise types (i.e., the different activities the student is asked to perform).

2. In Pairs (15 minutes) 2.1 Quickly compare (and amend if necessary) your notes under 1 above.

* PLUS 2.2 Which exercises lend themselves to written, and which to oral work? 2.3 Consider the exercise rubrics ( = instructions). Would the student ex-

perience any difficulty understanding what he has to do? 2.4 Can you spot any pitfalls (i.e., language traps) into which the student

might easily fall? If so, how should the ESP teacher guard against such pitfalls?

2.5 Are there any diagrams or exercise items that are culturally inap- propriate to the Philippines? If so, how should the ESP teacher handle them?

3. In 2 2 Pairs (20 minutes) 3.1

3.2

3.3

3.4

3.5

3.6

3.7

What comments do you have on the scientific content of the unit? Would your students respond to it? Can you handle it? At the end of the unit, which language items and skills (in general terms) will the student have learnt? How relevant are these language items and skills to general science? .._.

C. Pknmy (15 minutes) Which points (e.g., language, scientific content, exercise type, exercise rubric, advocated teaching methodology, sequence of presentation) do each group feel strongly about? (We shall tabulate these points under IC and X.)

Quickly compare (and amend if necessary) your notes under 2 above. PLUS Consider the teaching methodology advocated. To what extent is it applicable to your institution? What are the snags? How might they be overcome? Are there exercise types/specific exercises that you think are too easy/difficult for your students? If so, which and why? Is there insufficient/sufficient/too much variety in language content and exercise types? Do you have any general comments on the layout and the presentation of the unit? E.g., is it attractively/satisfactorily/unattractively set out and printed?

160 The ESP Journal

Comments on the TAE Procedure in Practice The following are comments on how this approach to TAE worked

out in practice:

1. The one hour allotted to each TAE session was over-optimistic. In fact, the sessions usually lasted about 90 minutes. 2. Seating arrangements were important. For the first session, I arranged pairs of double-tables facing each other, before par- ticipants arrived. Thereafter, I had participants themselves re- arrange a standard full-frontal seating arrangement into pairs of double-tables - to prove that the seating rearrangement of a real- life ESP classroom can be done quickly and quietly. Groups of four had to be sufficiently separate for unimpaired conversation in stages 2 and 3, but sufficiently close for the whole-class atmosphere re- quired in stage 4 (plenary). Further, desks had to be positioned so that participants could focus on the teacher/board in the plenary stage without moving chairs and tables. 3. In the first TAE session, I had difficulty deciding what to do with myself during stages 2 and 3. Joining the discussion of a pair or four was not successful, as I soon found I was doing too much of the talking. Alternatively, “observing” a pair or four tended to make some participants self-conscious. In the end, I gave up my am- bulatory role during these stages, sat by myself, and pretended to read or write something. 4. At the plenary stage, it was important to ensure that each group had the opportunity of expressing a point they held strong views about. This meant that a group opinion had to be expressed suc- cinctly, and that I had to quickly extract a class consensus on that opinion before adding it to the blackboard “/ or X” summary of points, e.g.:

- selection and practice of common-core scientific lexis

- scientific content not frightening to English teacher

- clear, unambiguous exercise rubrics

-

dangers involved in presenting semantically and phonologically similar structures; risk of hybrids, e.g., is crmnected of limited opportunity for writing beyond the sentence level most exercise types lend themselves to small- group work; (?) manage- ment problems with a class of 45-50 over-emphasis on oral work

A Procedure for Textbook Analysis and Evaluation 161

There was a great temptation to allow the plenary discussion on controversial issues - e.g., in Nucleus: General Science the recom- mended methodology, the balance between sentence-restricted and extended writing - to go on too long. 5. At the plenary stage, also, I had to be careful not to influence the consensus on various points to meet my personal views. On the other hand, I found that if I leant too far in the direction of “impar- tial umpire”, course participants became impatient, and quickly asked “Yes, but what do you think?” Clearly, the tutor has to walk a delicate tightrope between these two extremes. 6. The larger of the two groups with which I used this TAE pro- cedure numbered 36. Because of its participant-centred nature, the approach can be used with much larger numbers, although the plenary session would become difficult to handle if numbers rose above about 50.

Conclusion

The traditional “check-list” approach to TAE is characterized by a predetermined list of questions (usually allowing only YES or NO answers), against which any ESP textbook can be judged. The result is usually a depressingly long list of the book’s faults with little scope for further comment, which in turn often alienates the course participant from ESP in general. In comparison, I found this application to TAE of the Open University group study skills approach a distinct advance over the “check-list” approach, as it capitalised on the conditions of the in-course TAE situation. Advances were particularly apparent in:

- a more active participation in the TAE - more informed, cogent observations - the opportunity to tailor the TAE script to the characteristics of

the textbook concerned - better preparation for TAE beyond the teacher education course - a greater opportunity for pooled experience to be brought to bear

on TAE

In fact, I feel sure that it is a procedure that also has possibilities for other aspects of ESP teacher education courses, such as the evaluation of exercise-type writing and of peer-teaching.

REFERENCES

Bates, M. and T. Dudley-Evans. 1976. General Scbzce. Nucleus: English fi Science and Technology. London: Longman Group Limited.

Gibbs, G. (Ed.). 1977. Learning to Study: A Guide to Running Group Ses- sions. Milton Keynes: The Open University.

Moore, J. et al. 1979. Exploring Functions. Reading and Thinking in English. Oxford: Oxford University Press.

162 The ESP Journal

Northedge, A. 1975. Learning through Discussion in the Open Univer- sity. Teaching at a Distance No. 2. [Page references are to an extract from Northedge appearing in Gibbs 1977.1

Ray Williams is Lecturer in English as a Foreign Language in the Language Studies Unit, University of Aston in Birmingham, UK, where his responsibilities include directing the presessional English and Study Skills course for overseas postgraduates. He formerly taught ESP in Zambia, Malawi and Hong Kong. He has written Spotlight on Technical English for the Far East (Longman, 1978). Re- cent articles include “Faster Reading - One Hundred Years after Javal” (Reading 13,3, 1979) and “Paragraph Writing in ESP” (ESPMENA 1, 1979). His major research interest is prepublication readability.