ELT Voices- International Journal for Teachers of English Volume (4), Issue (6), 138-149 (2014)
ISSN Number: 2230-9136
(http://www.eltvoices.in)
Evaluating American English File Text Books Based on
Cunningsworth’s Criteria
Saeideh Shahriari
Department of English Language, Payame Nour University, Qeshm, IRAN
Amir Reza Nemat Tabrizi
Department of English Language, Payame Noor University, Tehran, IRAN
PO BOX 19395-3694
Corresponding e-mail address: [email protected]
Article reference
Shahriari, S., & Nemat Tabrizi, A. R. (2014). Evaluating American English File based on Cunningsworth’s criteria . ELT
Voices, Volume, 4 (6), 138-149.
Abstract: This study evaluated American English File text books based on Cunnings Worth (1995) model. A sum of 45
English learners and 20 teachers were selected from the total population of the available intermediate and advanced stu-
dents at Shokouh English Institute in Kerman. Then, a checklist in the fo rm of a questionnaire, which was adapted primari-
ly from Cunningsworth's (1995) checklist, was distributed among the teachers and students. The items of the questionnaire
were developed according to 14 categories which were mentioned by Cunningsworth (1995). Using SPSS 16.0, the re-
searcher evaluated both students ’ and teachers’ answers based on the differences in the students answers according to their
gender and also on the differences between teachers’ and students’ answers to the same questionnaire. The study was car-
ried out and data was analyzed using an independent t-test to evaluate the teachers ’ and students' answers to questionnaire
items. The results of questionnaire showed that the American English File textbooks were perceived as adequate books and
they met the standards of a good textbook based on Cunningsworth ’s (1995) criteria.
Index Terms: American English File Text Books, Cunningsworth’s criteria, Textbook Evaluation
1. Introduction
Material evaluation aims to determine the extent to which the objectives in the curriculum object ives are achieved.
Taba-Tyler (1962, cited in Pierson & Friederichs, 1980) argued that a textbook evaluation determines the extent to which
the curriculum objectives are met. The ability to effectively evaluate an English Language Teaching (ELT) textbook is very
important in order to achieve the designated learning goals. Garinger (2001) echoed Hartley's (1992) claim that three con-
tent areas should be addressed when evaluating a textbook: teaching objectives, depth and breadth of material, and the need
for textbook supplementation. Thus, in this study the researcher recommended some appropriate tips for the curric ulum
planners according to results.
Bolitho (1998, cited in Li, 2004) ment ioned "teachers understand their own learners best because they understand their
needs and their preferred learning styles" (p. 258) .On the other hand, students are also well positioned to know their own
learning styles and their curricular needs. Therefore, this study is a valuable research project since it aims to reveal the
strengths and weaknesses in American English File textbooks by asking both teachers and students ideas about that, and
determine whether it is viable, or needs supplementation and/or modification for optimal learning.
ELT Voices-Volume (4), Issue (6), 138-149 (2014) 139
Text books are important resources during the English language learn ing process especially for teachers in assisting
students to learn better. Therefore, according to Azizifar et al. (2009), textbooks serve as the basis for much of the language
input and are considered to be dependable for many learners in Iran. However, there are some groups of teachers who have
a very poor opinion of textbooks. They say textbooks are boring and inappropriate for the class as they want to rely on their
own ideas, reference books, pages from magazines, ideas from the students and other resources. Therefore, some teachers
make decision to do without textbook, but this decision is possible if teachers have enough experience and t ime to provide
a consistent program of work by using their own bank of materials. In contrast, a large group of teachers and learners often
feel positive about textbooks as for the former a textbook gives them a consistent syllabus and also dependable teaching
sequences and for the latter textbook is reassuring and give them a chance for what’s coming and reviews what they have
done. Since implementation of American English Books in Kerman Shokuh Institutes , these textbooks have been crit icized
by supervisors and teachers as well as students for many reasons including:
1) There is a high amount of contents that must be covered while a low time allotted to this a ttempt. Teaching 7 units in one
term with all those overloaded materials is not possibly an easy way of teaching (Elis, 1997, p. 37).
2) American English Book series are overloaded with a lot of new vocabularies and expressions and grammar points which
are difficu lt for learners. Each page has to be taught in 2 sessions so that learning could take place. In order to eliminate this
problem, the manager has divided each book into 4-5 levels.
3) The cultural appropriateness of some textbook's units may be incomprehensible for Iran ian English learners. Thus the
institute's chief executive manager has to modify the pictures and censor all the cross-cultural materials which are so obvi-
ous in the books (Yarmohammadi, 2002, p.125).
The present study carried out an evaluation of a series of American English File textbooks which are used for teaching
English language in Kerman Shokuh Institute. In this study, the researcher intended to investigate criticisms which are
mentioned about these books by evaluating the textbooks based on Cunningsworth ’s criteria. Moreover, it examined the
validity of the above mentioned claims made by some teachers, supervisors and students. This evaluation is significant
because it reveals the strengths and weaknesses of the American English File textbooks and determines how well these
books meet the standards of a good book (Bolitho 1998, cited in Li, 2004) and provides guidance and feedback fo r Ameri-
can English File revisions as well. This study may also enhance teachers own personal and professional development by
encouraging them to adopt a reflective approach to their own teaching practice. Since these books haven’t yet been forma l-
ly evaluated, this task may be a difficult one which needs both hardworking and high motivations . Thus, the result of the
study is hoped to benefit English language teachers, learners, and textbook developers to improve their teaching, learning,
and designing of the textbooks.
2. Review of the Related Literature
2.1. Theoretical background
It takes a lot of t ime and energy to choose or develop an appropriate textbook which meets the requirements of the
institutions and the needs of the learners. Once a textbook is chosen, it should be evaluated by standard (or reliable) chec k-
lists based on acceptable criteria. These checklists, either aimed at evaluating a general EFL t extbook or an EAP/ESP
course book, should be able to lead to a more systematic and thorough examination of potential textbooks, evaluating them
on their underlying instructional philosophy, approach, method, activities, and teacher’s manual (Hamidi & Monta zeri,
2014). There are different views concerning why we evaluate materials used for teaching English. Hutchinson and Waters
(1987) see it as an analytical “matching process: matching needs to available solutions ”. This, they suggested, can be done
according to the sponsor’s needs through a need analysis. While this perspective is important, it is incomplete since it
leaves out the views of other interested parties, such as the teachers and the learners (ib id, p. 97). Material evaluation is an
educational necessity because it shows how a textbook can be improved or justified. Teaching materials have a direct in-
fluence on the process of learning and teaching. Nunan (1988) states that materials are, in fact, an essential element within
Shahriari & Nemat Tabrizi (2014). Evaluating American English File based on Cunningsworth criteria.
the curriculum, and do more than simply lubricate the wheels of learn ing. At their best, they provide concrete models for
desirable classroom practice. They act as curriculum models and at their very best; they fulfill a teacher development role
(Nunan, 1988, p. 98). Weir and Roberts (1994) state two main reasons for evaluating teaching materials: First, it provides
evidence, “which can inform theoretical disputes about directions to be followed in language teaching or in teacher educ a-
tion” (p.11). Second, it is a tool to indicate the suitability of particu lar approaches or techniques under given conditions and
whether they meet the claims made for them.
2.1.1. Different views about evaluation
Hutchinson (1987) suggests an interactive view of material evaluation. He emphasizes th e deeper level of material
evaluation by asking the question why materials are the way they are. He claims that "... Material evaluation plays such an
important role in language teaching that its potential for influencing the way teachers operate is consid erable. Material
evaluation can and should be a two-way process which enables teachers not just to select a textbook, but also to
develop their awareness of their own teaching/learning situation. Evaluation of textbooks is also
considered to function as a kind o f educational judgment. Hu tchinson and Waters (1987, p. 96) give this
definit ion, "Evaluation is a matter of judging the fitness of something for a part icular purpose. Cunningsworth (1984, p. 64)
puts forward the idea"...that the process of evaluation could not be a purely mechanical one and that professional judgment
was involved at every stage. 'He goes on to say," Professional judgment, founded on understanding of the rationale of la n-
guage teaching and learning and backed up by practical experience, lies at the base of evaluation procedure. To conclude,
material evaluation helps us make decisions in selecting textbooks, forming professional judgments as well as rais ing
awareness of or reflecting on our teaching and learning experience.
2.2. Related studies
Literature related to textbook evaluation was reviewed as the basis for a decision on which form and approach of
evaluation would be appropriate and how an evaluation should be implemented. While the theoretical literature review in-
vestigates the latest studies contributing to the theory of textbook evaluation, the empirical literature review briefs some
related experimental studies on EFL/ESL textbook evaluation that have been done in Iran and other countries. As far as
the review of literature is concerned, in Iran several pro jects have been carried out to evaluate textbooks, among which A n-
sary and Babaii (2002), Yarmohammadi (2002), and Amalsaleh (2004) are the typical examples. Ansary and Babaii (2002)
analyzed a corpus of 10 EFL/ESL textbook rev iews plus 10 EFL/ESL textbook evaluation checklists and outlined what they
perceived to be the common core features of standard EFL/ESL textbooks. The major categories comprise approach, con-
tent presentation, physical make-up, and administration concerns. Each set of major features of EFL/ESL textbooks co n-
sists of a number of subcategories. They concluded the article mentioning that not all o f these characteristics would be pre-
sent in each and every textbook.
Yarmohammadi (2002) evaluated the senior high school textbooks based on a revised version of Tucker’s model. He
came to the conclusion that these textbooks suffer from a lot of shortcomings: 1. they are not authentic; 2. English and Pe r-
sian names are used interchangeably; and 3. oral skills are ignored. At the end, some suggestions were proposed to remedy
the shortcomings.
2.3. Research questions
Q1: To what extent does the content of the American English File text books meet the standards of a good book based on
Cunningsworth criteria of evaluating 14 categories (e.g. Content, Grammar, Vocabulary, Phonology, Language Skills,
Methodology, Study Skills, Visuals, Pract ice and Testing, Supplementary Material, Objectives, Content Select ion, Grad a-
tion and Recycling, and the Teacher's Manual)?
Q2: Are there any significant differences in the ratings of each of the fourteen categories when ranked by both teachers and
students?
Q3: Are there any significant d ifferences in the ratings of each of the ten categories when ranked by both male and female
groups of students?
ELT Voices-Volume (4), Issue (6), 138-149 (2014) 141
3. Methodology 3.1. Sample/ Participants
The participants were 20 English teachers and 45 students. All the teachers had BS or BA degree from well known
universities, and also five teachers had got their MA or MS degree recently. Teachers’ age ranged between 24-50 years old.
Furthermore, the number of students who studied in American English File levels at the t ime of doing this study were
about 45 learners both male and female who were asked to participate in this attempt and to fill in the questionnaire care-
fully and honestly and to issue their opinions about the textbooks followed by conducting a short interview with both
teachers and students. Furthermore, students’ age ranged between 17-35 years old. Mostly, they were h igh school or uni-
versity students, also there were some students of post graduation.
3.2. Instruments
Two instruments were employed in this study including:
A. A Textbook Evaluation Tool (TET)
TET or questionnaire was developed primarily from Cunningsworth's (1995) checklist .The items of the questionnaire
were developed according to categories which were mentioned by him. Cunningsworth (1995) remarked that since different
criterias were applied in d ifferent circumstances, it was best for teachers or researchers to identify their own priorities and
draw up their own checklists. TET was used to reflect participants’ judgments and opinions regarding the American English
File textbooks. A four-point forced-mult iple-choice Likert scale format, ranging 1-4, was used to reflect participants' level
of agreement with a list of statements. Each evaluative statement was weighted equally, and with 50 statements. Moreover,
to describe these variables the related scores were categorized into “up” for stro ngly agree and agree choices while “bo t-
tom” for disagree and strongly disagree categories then scores were converted to percentage rankings (0-100%). Besides,
students and teachers-supervisors completed the same version of questionnaire, to allow for comparison across groups, alt-
hough the teachers and the supervisors' version included some additional items.
While both questionnaires contained ten major categories (statements 1-33)- content, grammar, vocabulary, phonology,
skills, methodology, study skills, visuals, and practice and testing- the teachers and supervisors’ questionnaire had four ad-
ditional categories: objectives, content selection, gradation and recycling, and the Teacher's Manual (statements 34-50).
Both questionnaires had an open-ended section, remarks, where participants were offered the opportunity to record their
own comments or suggestions regarding the textbooks.
B. Interviews
There were two steps in conducting this interview in which the interviewers had a list of questions prepare d for
this attempt. In the first step two individual interviews were done for this study with two institute supervisors. The inte r-
viewees’ conversations revealed some important points about the strength and of weakness of these books. In the se c-
ond step, there were some other interviews with other teachers after filling in the questionnaires.
3.3. Procedure
To achieve the goal, four American English File textbooks were evaluated from two different aspects and in two dif-
ferent steps. First, a number of teachers were asked to scrutinize the book with the mentioned criteria in the questionnaire.
Totally, there were 50 questions within 14 categories. All the research questions and hypothes es were tested for “statistical
significance” at the α = .05 level. Using a known statistical evaluation tool SPSS 16.0, the researcher evaluated both stu-
dents and teachers answers firstly, based on the differences in the students answers according to their gender and secondly,
based on the differences between teachers and students answers to the same questionnaire. An experimental study was ca r-
ried out and data was analyzed using statistical means such as Leven Test and Independent Sample T-test. The reliability of
the questionnaire was measured by Cronbach's alpha (α) as referred in Hesham Suleiman Dawood Al-Yousef thesis (2007)
in which the questions were tailored specifically for special studies.
Shahriari & Nemat Tabrizi (2014). Evaluating American English File based on Cunningsworth criteria.
4. Results and Discussion
4.1. Introduction
While both questionnaires contained ten major categories (statements 1-33) the teachers and supervisors’ question-
naire had four additional categories. (Statements 34-50).The 50 Likert-type statements were classified under fourteen cate-
gories: (1) Content, statements 1-7; (2) Grammar, statements 8-11; (3) Vocabulary, statements 12-13; (4) Phonology, state-
ments 14- 15; (5) Language Skills, statements 16- 18; (6) Methodology, statements 19- 21; (7) Study Skills, statements 22-
25; (8) Visuals, statements 26-27; (9) Practice and Testing, statements 28-32; (10) Supplementary Material, statement 33;
(11) The Objectives, statements 34-37; (12) Content Selection, statements 38-40; (13) Gradation and Recycling, statements
41-42; and (14) The Teacher's Manual, statements 43-50. Moreover, both questionnaires had an open-ended section, re-
marks, where participants were asked to record their own comments or suggestions regarding the textbook. All participants
completed the same version of questionnaire with the exception of statements 34 to 50, which were directed only to teach-
ers and supervisors.
4.2. Statistical Analysis
After conducting the present study, statistical means (Independent Sample T-test, Leven Test) were used to interpret
the data obtained. The results, which were obtained from the statistical data, shed light on the aspects of the research ques-
tions of the current study. This chapter presents the results of the statistical computations in tables and graphs, then they
were analyzed and the findings were discussed in details. This study evaluated the American English File text books based
on Cunningsworth (1995) model. To meet this goal, a total of 45 students and 20 teachers all studied and taught in Kerman
Shokuh institute took part in this research. Then a TET or checklist in the form of a questionnaire which was developed
primarily from Cunningsworth's (1995) checklist was distributed among teachers and students. The items of the questio n-
naire developed according to 14 categories which were mentioned by Cunningsworth (1995). Using a known statistical
evaluation tool SPSS 16.0, the researcher evaluated both students and teachers answers firstly, based on the differences in
the students answers according to their gender and secondly, based on the differences between teachers and stu dents an-
swers to the same questionnaire. The data were analyzed using statistical means such as independent sample t-test and de-
scriptive statistics including maximum, minimum, frequency tables, cross tables, and frequency percentage.
4.2.1. Description of demographic criteria
Table 4.2.1.The distribution of the frequency of students’ gender
gender Frequency Percent Valid Percent Cumulative Percent
Valid Boy 17 37.8 37.8 37.8
Girl 28 62.2 62.2 100.0
Total 45 100.0 100.0
The demographic information contained information related to students’ gender. In this study 28(%62.2) out of 45
students were female while 17(%37.8) were male. (Table 4.2.1 illustrates detailed information of students’ gender).
4.3. The research questions analysis
Are there any significant differences in the ratings of each of the fourteen categories when they are ranked by teachers
and students and which of them are ranked higher during evaluation?
In order to answer this question, the results of each of 14 categories in TET are presented below.
H0: There are no differences between students and teachers answers regarding the content of the textbooks.
H1: There are some differences between students and teachers answers regarding the content of the textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.368 and more
than significance level =α 0.05 therefore, at this level H0 is not rejected, and it is concluded that the categories variances
ELT Voices-Volume (4), Issue (6), 138-149 (2014) 143
are equivalent and the parametric test is practical.
Table 4.3.1.The t- test statistic to compare the content of textbooks based on teachers and students opinions
Group N Mean Std. Deviation t df Sig
Content Student 45 19.0889 3.18250 8.98 63 0.001
Teacher 20 12.0000 2.27110
A T-test was used to compare the content of textbooks based on teachers and students ' answers. Since the value of
–p(significance) is 0.001 ,and it is less than the significant level of =α 0.05 therefore, at this level H0 is rejected. Con-
sequently, it is concluded that there is a significant difference between the students and teachers ' opinions about the content
of textbooks and by comparing the means , it is clear that students evaluated the content of the textbooks higher than teach-
ers (Table 4.3.1).
H0: There are no differences between students and teachers ' answers regarding the Grammar of textbooks.
H1: There are some differences between students and teachers ' answers regarding the Grammar of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.299 and more
than significance level =α 0.05 therefore, at this level H0 is not rejected ,and it is concluded that the categories variances
are equivalent and the parametric test is practical.
A T-test was used to compare the Grammar of textbooks based on teachers and students ' answers. Since the value of
–p(significance) is 0.001, and it is less than the significant level of =α 0.05 therefore, at this level H0 is rejected. Con-
sequently, it is concluded that there is a significant difference between the students and teachers ' opinions about the Gram-
mar of the textbooks and by comparing the means , it is clear that students evaluated the Grammar of textbooks higher than
teachers.
H0: There are no differences between students and teachers answers regarding the vocabularies of textbooks.
H1: There are some differences between students and teachers ' answers regarding the vocabularies of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.63and more than
significance level =α 0.05 therefore, at this level H0 is not rejected, and it is concluded that the categories variances are
equivalent and the parametric test is practical.
A T-test was used to compare the vocabularies of textbooks based on teachers and students ' answers. Since the value
of –p(significance) is 0.005 ,and it is less than the significant level of =α 0.05 therefore , at this level H0 is rejected.
Consequently, it is concluded that there is a significant difference between the students and teachers opinions about the
vocabularies of textbooks and by comparing the means it is clear that students evaluated the Grammar of textbooks higher
than teachers.
H0: There are no differences between students and teachers ' answers regarding the phonology of textbooks.
H1: There are some differences between students and teachers ' answers regarding the phonology of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.51and more than
significance level =α 0.05 therefore, at this level H0 is not rejected, and it is concluded that the categories variances are
equivalent and the parametric test is practical.
A T-test was used to compare the phonology of textbooks based on teachers and students ' answers. Since the value of
–p(significance) is 0.001 ,and it is less than the significant level of =α 0.05 therefore, at this level H0 is rejected. Con-
sequently, it is concluded that there is a significant difference between the students and teachers ' opinions about the pho-
nology of textbooks and by comparing the means it is clear that students evaluated the phonology of textbooks higher than
teachers.
H0: There are no differences between students and teachers ' answers regarding the language skills of textbooks.
Shahriari & Nemat Tabrizi (2014). Evaluating American English File based on Cunningsworth criteria.
H1: There are some differences between students and teachers answers regarding the language skills of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.09and more than
significance level =α 0.05 therefore, at this level H0 is not rejected, and it is concluded that the categories variances are
equivalent and the parametric test is practical.
Table 4.3.2.The t- test statistic to compare the language skills of textbooks based on teachers and students' opinions
Group N Mean Std. Deviation t df Sig
Language Skills Student 45 8.56 2.159 6.72 63 0.001
Teacher 20 5.10 1.165
A T-test was used to compare the language skills of textbooks based on teachers and students ' answers. Since the value
of –p(significance) is 0.001 ,and it is less than the significant level of =α 0.05 therefore , at this level H0 is rejected.
Consequently, it is concluded that there is a significant difference between the students and teachers opinions about the
language skills of textbooks and by comparing the means it is clear that students evaluated the language skills of textbooks
higher than teachers. (Table 4.3.2)
H0: There are no differences between students and teachers answers ' regarding the methodology of textbooks.
H1: There are some differences between students and teachers ' answers regarding the methodology of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.81and more than
significance level =α 0.05 therefore, at this level H0 is not rejected ,and it is concluded that the categories variances are
equivalent and the parametric test is practical.
Table 4.3.3.The t- test statistic to compare the methodology of textbooks based on teachers and s tudents' opinions
Group N Mean Std. Deviation t df Sig
Methodology Student 45 8.47 1.984 6.22 63 0.001
Teacher 20 5.20 1.881
A T-test was used to compare the methodology of textbooks based on teachers and students ' answers. Since the value
of –p (significance) is 0.001 ,and it is less than the significant level of =α 0.05 therefore , at this level H0 is rejected. Con-
sequently, it is concluded that there is a significant difference between the students and teachers opinions about the meth-
odology of textbooks and by comparing the means it is clear that students evaluated the methodology of textbooks higher
than teachers (Table 4.3.3).
H0: There are no differences between students and teachers ' answers regarding the study skills of textbooks.
H1: There are some differences between students and teachers ' answers regarding the study skills of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.13and more than
significance level =α 0.05 therefore, at this level H0 is not rejected ,and it is concluded that the categories variances are
equivalent and the parametric test is practical.
A T-test was used to compare the study skills of textbooks based on teachers and students ' answers. Since the value of
–p (significance) is 0.024 and it is less than the significant level of =α 0.05 therefore, at this level H0 is rejected. Conse-
quently, it is concluded that there is a significant difference between the students and teachers ' opinions about the study
skills of textbooks and by comparing the means it is clear that students evaluated the study skills of textbooks higher than
teachers. H0: There are no differences between students and teachers ' answers regarding the visuals of textbooks.
H1: There are some differences between students and teachers ' answers regarding the visuals of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.92and more than
ELT Voices-Volume (4), Issue (6), 138-149 (2014) 145
significance level =α 0.05 therefore, at this level H0 is not rejected ,and it is concluded that the categories variances are
equivalent and the parametric test is practical.
A T-test was used to compare the visuals of textbooks based on teachers and students ' answers. Since the value of –p
(significance) is 0.001 ,and it is less than the significant level of =α 0.05 therefore, at this level H0 is rejected. Conse-
quently, it is concluded that there is a significant difference between the students and teachers ' opinions about the visuals of
textbooks and by comparing the means it is clear that students evaluated the visuals of textbooks higher than teachers.
H0: There are no differences between students and teachers ' answers regarding the practice and testing of textbooks.
H1: There are some differences between students and teachers ' answers regarding the practice and testing of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.93and more than
significance level =α 0.05 therefore, at this level H0 is not rejected, and it is concluded that the categories variances are
equivalent and parametric test is practical.
A T-test was used to compare the practice and testing of textbooks based on teachers and students ' answers. Since the
value of –p (significance) is 0.001 and it is less than the significant level o f =α 0.05 therefore, at this level H0 is rejected.
Consequently, it is concluded that there is a significant d ifference between the students ,and teachers opinions about the
practice and testing of textbooks and by comparing the means it is clear that students evaluated the practice and testing of
textbooks higher than teachers.
H0: There are no differences between students and teachers ' answers regarding the supplementary material of textbooks.
H1: There are some differences between students and teachers ' answers regarding the supplementary material of textbooks.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.116and more
than significance level =α 0.05 therefore, at this level H0 is not rejected, and it is concluded that the categories variances
are equivalent and the parametric test is practical.
A T-test was used to compare the supplementary material of textbooks based on teachers and students ' answers. Since
the value of –p (significance) is 0.001 and it is less than the significant level o f =α 0.05 therefore, at this level H0 is re-
jected. Consequently, it is concluded that there is a significant difference between the students and teachers ' opinions about
the supplementary material of textbooks and by comparing the means it is clear that teachers evaluated the supplementary
material of textbooks higher than students.
4.3.1. The comparison of the study components according to students’ gender
The comparison of the study components according to students’ gender which was done by independent t -test showed
that just visual component is different according to males and females ' opinions and this difference will be analyzed later.
4.3.4. Are the textbooks' visuals different according to male and female students’ opinions?
Ho: The textbooks ' visuals are different according to male and female students’ opinions.
H1: The textbooks' visuals are not different according to male and female students’ opinions.
The Leven Test was used to test the variances’ equivalences. Since the value of –p (significance) is 0.061and more
than significance level =α 0.05 therefore, at this level H0 is not rejected, and it is concluded that the categories variances
are equivalent and the parametric test is practical.
A T-test was used to compare the textbooks ' visuals according to male and female students’ opinions. Since the value
of –p (significance) is 0.022and it is less than the significant level of =α 0.05 therefore, at this level H0 is rejected. Con-
sequently, it is concluded that there is a significant difference between the male and female students’ opinions about the
visuals of textbooks and by comparing the means it is clear that female students evaluated the visuals of textbooks higher
than male.
4.4. The condition of the research variables
4.4.1. The condition of the objectives based on teachers' opinions
To describe this variab le the related scores were categorized into “up” and “down” categories. Therefore, among 20
Shahriari & Nemat Tabrizi (2014). Evaluating American English File based on Cunningsworth criteria.
teachers 8 teachers (%40) evaluated the objectives bottom and 12 teachers (%60) evaluated them up.
To describe this variab le, the related scores were categorized into “up” and “down” categories. Therefore, among 20 teach-
ers 5 teachers (%25) evaluated the content selection bottom and 15 teachers (%75) evaluated them up. To describe th is var-
iable, the related scores were categorized into “up” and “down” categories. Therefore, among 20 teachers 12 teachers (%60)
evaluated the Gradation and Recycling bottom and 8 teachers (%40) evaluated them up. To describe this variable , the re-
lated scores were categorized into “up” and “down” categories. Therefore, among 20 teachers 6 teachers (%30) evaluated
the Teachers Manual bottom and 14 teachers (%70) evaluated them up.
5. Conclusion
The process of language education involves many elements often learners considered as the center. However, this
common belief is rejected when textbooks as sources of providing input are seen to control the instruction to a large extent
(Sarem, Hamidi, & Mahmoudie, 2013). There is a range of textbook evaluation checklists in the ELT literature; most re-
searchers use their experience when developing their own checklist by including those categories and sub-categories that
they seem important. As Cunningsworth (1995) remarks, since different criteria will apply in different circumstances, it is
the best for teacher to tailor their own checklist for evaluating a textbook.
To answer the first research question, the results of questionnaire showed that American English File textbooks were
perceived as adequate books, and they meet the standards of a good textbook based on Cunningsworth’s (1995) criteria
since the mean scores for each category exceeded the score 2 except in one item.
According to the second research question, the findings revealed that there are significant differences in the ratings of eac h
of the fourteen categories when ranked by both teachers and students ,and mostly students ranked them h igher than teachers
but in one item, which is supplementary materials, teachers ranked it higher than students.
Based on the third research question, there are not any significant differences in the ratings of each of the ten catego-
ries when ranked by both males and females groups of students, but just visual component is different according to both
gender answers, since female students evaluated the visuals of textbooks higher than male.
In short, the results of questionnaire showed that the American English File textbooks were perceived as moderately ad e-
quate since the mean scores for each category exceeded 2, except for Supplementary Material (for stude nts). Therefore,
students rated the textbooks more favorably than the teachers in all but one category which is Supplementary Material.
In this study both the quantitative and the qualitative findings showed that the Content, practice and testing of the
textbook were among the categories that gained the most support by both teachers and students answers; since the mean
scores of students and teachers ranking for the content are respectively 19.08 and 12 while practice and testing mean scores
for students and teachers are 13.53, 9.5 respectively. Most respondents agreed that the practice and testing of textbooks
were not only reasonably well produced and attractive, but also an integral part of teaching.
Besides, based on interview results both teachers and students stated that the textbook contains some writ ing and
reading tasks that are too difficu lt for the learners. Thus, syllabus designers and managers could take into account the leve l
of the course, the characteristics of learners, and the degree of preparation of teachers when choosing a particular approach
to syllabus design. Moreover, while students ranked just one category (supplementary materials) unfavorably, the teachers
rated three categories (supplementary materials, phonology and visuals) as the most unfavorable ones. The most poorly
rated category based on teachers opinions was supplementary materials (M=3.05, sd=.887), followed closely by phonology
(M=3.3, sd=1.03).Gradation and Recycling was to some extent not given greater concern by teac hers since the selected
items are sometimes beyond learners' level and are not graded according to the learners' level, as d iscussed earlier in cha p-
ter one according to the interviews' findings. On the other hand, the results of the Teacher's Manual accurately reflected the
teachers’ perceptions. Since all teachers were provided with the Teacher's Manual, the findings of the study showed this
category ranked the highest among the teachers as in these books learning difficult ies are predicted and appropriate advice
ELT Voices-Volume (4), Issue (6), 138-149 (2014) 147
is given in addition to the guidelines for evaluating how well lessons went.
The results of the research question 3 showed no statistically significant differences among variables based on male
and female students ' opinions and their ratings of the textbooks except in one Visual category. Therefore, there is a signifi-
cant difference between the male and female students’ opinions about the visuals of textbooks and by comparing the means
it is clear that female students evaluated the visuals of textbooks higher than male. Moreover, according to the findings in
question 6, which is related to contextualization in the content of the textbooks, 16 out of 17 male students ranked it co m-
pletely favorable that showed the male students care more about contextualizat ion than female who 18 out of 27 of them
ranked it high. Also male students graded “principle of immediate use” , which was categorized in vocabulary category,
higher than female students that revealed the males use language more immediate than females do.
Shahriari & Nemat Tabrizi (2014). Evaluating American English File based on Cunningsworth criteria.
References
[1]Carter, R. (1987). Vocabulary: Applied linguistics perspectives. London: Allen & Unwin.
[2]Cunningsworth, A. (1995). Choosing your course book . Oxford: Heienemann.
[3]Ellis, R. (1988). Classroom second language development . London: Prentice Hall.
[4]Ellis, R. (1988). The role of practice in classroom learning. AILA Review, 5, 20-29.
[5]Ellis, R. (1997). The empirical evaluation of language teaching materials . ELT Journal, 51(1), 36-42.
[6]Gollin, J. (1998). Key Concepts in ELT: Deductive vs inductive language learning. ELT Journal, 52(1), 88-97.
[7] Hamid i, H., & Montazeri, M. (2014). Dictionary of second language acquisition . Retrieved December 25, 2014, from
http://www.iranelt.com/index.php/introduction-to-sla.
[8]Hopkins, D., & M. Nettle (1994). Second language acquisition research: A response to Rod Ellis. ELT Journal, 48(2),
157-161.
[9]Johns, T. (1991). Should you be persuaded: Two examples of data-driven learning. ELR Journal: Classroom concord-
ancing, 4, 1-16.
[10]McCarthy, M., & R. Carter (1995). Spoken grammar: What is it and how can we teach it? ELT Journal, 49(3), 207-218.
[11]McDonough, J., & C. Shaw (1993). Materials and methods in ELT: A teacher's guide. Oxford: Blackwell.
[12]Nation, P. (2000). Learning vocabulary in lexical sets: Dangers and guidelines. TESOL Journal, 9(2), 6-10.
[13]Richards, J. C., & T. S. Rodgers (1986). Approaches and methods in language teaching . Cambridge: Cambridge Uni-
versity Press.
[14]Rivas, R. M. M. (1999). Reading in recent ELT course books. ELT Journal, 53(1), 12-21.
[15] Sarem, S. N., Hamidi, H., & Mahmoudie, R. (2013). A critical look at textbook evaluation: A case study of evaluating
an ESP course-book: English for international touris m. International Research Journal of Applied and Basic Sciences, 4(2),
372-380.
[16]Sinclair, J. (1991). Corpus, concordance, collocation. Oxford: Oxford University Press.
[17]Sinclair, J., & A. Renouf (1988). A lexical syllabus for language learning. Vocabulary and language teaching. Harlow,
Longman.
[18]Skehan, P. (1996). Second language acquisition research and task-based instruction. In D. Willis & J. Willis (Eds.),
Challenge and change in language teaching (pp. 17-30). Oxford: Heinemann.
[19]Skehan, P. (1998). A cognitive approach to language learning. Oxford: Oxford University Press.
[20]Soars, L., & J. Soars (1998). New headway upper-intermediate English course. Oxford: Oxford University Press.
ELT Voices-Volume (4), Issue (6), 138-149 (2014) 149
[21]Stebbins, C. (1995). Culture-specific perceptual-learning style preferences of postsecondary students of English as a
second language. Boston: Heinle & Heinle.
[22]White, R. (1988). The ELT curriculum. Oxford: Blackwell.
Author Bio
Saeideh Shahriari is an English teacher and translator. She has been teaching enthusiastically for about 12 years in different
levels especially advance and adults . Moreover, she has been an active tour leader for over 5 years at different travel agen-
cies. She has participated in d ifferent English conferences and seminars presenting different articles, regarding teaching and
translating. Her areas of interests are textbook evaluation, testing, psycholinguistics, morphology and discourse analysis.