teaching and learning in saudi arabia perspectives from ... · pdf fileteaching and learning...

Teaching and Learning in Saudi ArabiaAm

ani K. Ham

dan (Ed.)

Spine13.056 mm

Teaching and Learning in Saudi ArabiaPerspectives from Higher Education

Amani K. Hamdan (Ed.)

S e n s e P u b l i s h e r s

Teaching and Learning in Saudi ArabiaPerspectives from Higher EducationAmani K. Hamdan (Ed.)University of Damman, Saudi Arabia

Saudi Arabia is witnessing unprecedented progress in the field of higher education.Even though the country opened its first university in 1957, so far there seems to be little English scholarly writing about Saudi education in general and higher education in particular. The current expansion of Saudi Arabia’s higher-education system has put a spotlight on this serious gap in the international literature.

Teaching and Learning in Saudi Arabia helps to fill this lacuna through the work of 16 scholars who have contributed to the development of the Saudi education system. In so doing, the book reveals areas where more research is required and thus provides a useful starting point for education scholars.

This anthology is unique in that it is the first to offer a comprehensive perspective on the current knowledge base pertaining to Saudi higher education as well as to the ongoing efforts to introduce reforms.

Cover photo: University of Dammam Main Campus, with permission from Abdullah Bin Hussein Alkadi (University Vice President for Studies, Development and Community Service)

ISBN 978-94-6300-203-5

DIVS

Teaching and Learning in Saudi Arabia

Teaching and Learning in Saudi ArabiaPerspectives from Higher Education

Edited by

Amani K. HamdanUniversity of Damman, Saudi Arabia

A C.I.P. record for this book is available from the Library of Congress.

ISBN: 978-94-6300-203-5 (paperback)ISBN: 978-94-6300-204-2 (hardback)ISBN: 978-94-6300-205-9 (e-book)

Published by: Sense Publishers, P.O. Box 21858,3001 AW Rotterdam,The Netherlandshttps://www.sensepublishers.com/

All chapters in this book have undergone peer review.

Cover photo: University of Dammam Main Campus, with permission from Abdullah Bin Hussein Alkadi (University Vice President for Studies, Development and Community Service)

Printed on acid-free paper

All Rights Reserved © 2015 Sense Publishers

No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work.

To Lujain, Yousef, Meriam, Ruqaia, and, above all, my best friend and soulmate,

Dr. Mohammed Y. Alghamdi – for their generous support and unconditional love.

We all believe that education is the enlightenment that we all aspire for and work towards … so that our society, too, will be enlightened

through education.

vii

TAbLe of ConTenTS

Preface ix

Acknowledgments xi

Introduction xiii

1. Assessing EFL College Instructors’ Performance with Digital Rubrics 1Reima Al-Jarf

2. A System for Teaching English in Saudi Arabia: A Model for English Language Acquisition with Reference to Study Skills, Multiple Intelligences, Thinking Based Learning and Group Work 31Khadijah Bawazeer

3. Analysis of Systemic Functional Linguistics Preparatory Year Writing in a Saudi University 49Philline Deraney

4. The Power of “I”: Student-Driven Writing at an All Women’s University in Saudi Arabia 71Barbara A. Toth

5. Cooperative Learning: A Case for Creative and Efficient Teaching and Learning Supported by: Analysis of the Present Situation in the Preparatory Year Program in Health Profession Track 101Mona Hmoud Al-Sheikh

6. The Relationship between Statistical Analysis Abilities and the Production of Research among Saudi Faculty 121Abdulghani Ali Al-Hattami and Arif Ahmet Mohamed Hassan Al-Ahdal

7. A Synthesized Model of Faculty Motivation in Saudi Arabia’s Higher Education Sector 129Akram AbdulCalder

8. Curriculum Design Quality Assurance of Distance Education in Saudi Electronic University: A Case Study 145Tariq Elyas and Abdullah Al-Garni

Table of ConTenTs

viii

9. EAP as an Index of Academic Excellence in Medical Studies at Majma’ah University 175El-Sadig Yahya Ezza and Nasser Al-Jarallah

10. Best Practices in English Language Testing at the University Preparatory Year Programs 185Mubina Rauf

11. EFL Faculty Perspectives on Technology-integration Strategies: A Case Study at Jazan University 207Osama Mudawe Nurain, Ahmet T. Braima and Barakat H. Makrami

Contributors 221

ix

PRefACe

The idea of editing a collection of research papers about higher education in Saudi Arabia has been a long-standing goal of mine. When I was a graduate student in humanities – in education, writing my Master’s thesis and then my doctoral dissertation in Philosophy of Education in Canada – there were no books in English, to my knowledge, that discussed the Saudi education system. I depended mostly on the unpublished research of Saudi graduate students studying in the United States, whose research was compiled in the Directory of the Doctoral Dissertations of Saudi Graduates from US Universities (1964–2005) by the Saudi Arabian Cultural Mission to the United States.

This book is the first anthology about higher education in Saudi Arabia, written by scholars from seven universities in Saudi Arabia. Some contributors are Saudi national professors and some are scholars and faculty members who work in higher-education institutions in Saudi Arabia but are originally from India, the United States (of Arab and non-Arab origin), Sudan and Yemen.

I would like to extend my thanks to the chapter authors for their tireless efforts. They have made this book available to provide a reliable and useful source of scientific information to higher-education researchers, advanced graduate students and practitioners in the field of education and development in Saudi Arabia, and elsewhere.

Dhahran, August 2015

xi

ACKnowLedgmenTS

The creation of this book was a large endeavour and I am grateful to the contributors for their enthusiasm, for taking part and participating in this book, thereby forming the first anthology written in English and edited by a Saudi academic about teaching and learning in Saudi Arabia. I would like to thank Dawn Martin for reviewing and editing the last version of the book and persistently taking over contacting the authors to clarify the small details to have the book in the best shape possible. I also thank my colleagues Dr. Barb Toth, Professor Reima Al-Jarf, and Dr. Nina AbdulRazzak for reviewing parts of this book. We all share a common purpose and hope that this book will enable educators, and all those interested in rethinking teaching and learning in Saudi Arabian higher education, to reflect on the diverse perspectives of educators who work in Saudi Arabia and whose research sheds light on its education.

I would like to express my gratitude and thanks to various copy editors who helped with the editing and formatting of the chapters.

This book would not be possible without the support of Sense Publishers, who gave me the initial encouragement to publish the book as an anthology. It is my great pleasure to have edited this first volume of research on teaching and learning in Saudi Arabia from the perspective of higher education.

xiii

InTRoduCTIon

Higher education in Saudi Arabia is undergoing unprecedented changes (Al-Anqari, 2014).The government’s current effort in developing its higher education system is moving into the direction of worldwide recognition. From expanding the number of institutions over the last decade to accrediting the highest number of students on scholarships across 75 countries abroad – east and west – all efforts are aimed at making education in Saudi Arabia world-class. Saudi Arabia now has 28 public universities and 30 private higher-education institutions. Transformations in the higher-education system have been influenced by an increasing student population, shifting demands of the job market, and international higher education (Al-Anqari, 2014). These changes in the education system primarily serve to advance the country and its citizens.

The expansion of higher education and the expected growth in the coming years should meet proper documentation of research efforts about education and teaching and learning in Saudi Arabia.

The chapters in this anthology provide an overview of the research on teaching and learning in Saudi Arabia. As the first book of its kind on the topic, it provides a unique contribution to the field of education and the current knowledge base in higher education reform and research in Saudi Arabia, spanning theoretical, historical, and domain-specific perspectives to provide readers with a broad scope of the field. The book reflects the methodologies that are used by researchers in Saudi Arabia, offers practical applications to researchers and educators, and provides valuable insight into what initiatives are needed to improve the higher-education system in Saudi Arabia.

The chapters focus on a variety of subjects, such as preparatory year in Saudi Arabia, best methods of statistical analysis used by faculty, cooperative learning, use of technology, the effect of intensive use of the Internet and Smartphones and analysis of university student writing using digital rubrics. This may encourage other scholars to document the research happening in Saudi Arabia and how scholars and researchers view the changing education system in one of the fastest growing systems of education, not only in the Middle East and North Africa, but also worldwide.

Chapter 1, by Professor Reima Al-Jarf of English language teaching at King Saud University, sets the scene for the book. Al-Jarf reflects on the traditional and new modalities of teaching and learning. In doing so, she highlights the use of digital rubrics – i.e., a scoring guide that consists of specific pre-established performance criteria used for evaluating students’ and teachers’ performance – to ensure the reliability of teacher-performance assessments. Digital rubrics have been created for evaluating English as Foreign Language college teachers’ linguistic and professional competencies using the iRubric building tool of the RCampus language management system.

xiv

INTRODUCTION

In Chapter 2, Khadijah Bawazeer introduces a new take on an established methodology of learning English based on innate learning processes and the intensive use of the Internet and Smartphones. Such innate language processes are used by children to learn their first language. This methodology is based on using the innate language-learning skills of the learner’s first language, with the added difference that the learner is older and more experienced with language acquisition because of their first language-learning experience and experiences accumulated over time. It is hoped that learning English adds to, rather than subtracts from, the fluency of either one’s native or second language. Plus, this innovative system can change the attitudes of learners from relying heavily on others to being more autonomous, and changing their study skills from rote memorization to incorporating memorization with the array of skills described by Bloom (Bloom’s Taxonomy). Furthermore, the researcher discovered that this system is compatible with the concepts of multiple intelligences, thinking-based learning, as well as with Smartphones used to create group follow-up. All four aspects – multiple intelligences, study skills, Smartphones, and Internet use – are essential for producing autonomous learners, an important goal for education in Saudi Arabia.

Chapter 3, by Philline Deraney, is an analysis of written texts from a university in Saudi Arabia using the paradigm of systemic functional linguistics (SFL) and genre-based writing pedagogy focusing on the textual meta function. The research methodology was mainly a description of the patterns that emerged in the female students’ writing related to features of coherence and cohesion with basic frequency tables to support the qualitative data. However, similar to several international and regional studies, elements of academic writing related to the textual metafunction were used inappropriately or lacked language complexity. The participants’ writing was limited and, at times, over-simplified, shown through the lack of accurate paragraphing, logical coherence, clear understanding of the genre requested, overuse of basic cohesive devices, and underuse of more complex ones. Implications for EFL writing education are discussed to assist students in creating more meaningful texts.

In Chapter 4, Barb Toth describes her teaching, learning, and administrative experiences in Saudi Arabia. More specifically, she narrates her experiences in building an English writing program over the last four years in the College of Languages and Translation at Princess Nourah bint Abdulrahman University for Women (PNU) in Riyadh. The students, however, are the main characters and the focus of her story.

In Chapter 5, Mona Al-Sheikh highlights the concept of cooperative learning as an innovative method of teaching that helps students work not only for their own benefit, but for others, too. The idea is that students cannot succeed alone; they need to cooperate with peers to achieve shared goals in learning and to excel in the classroom. They can excel in their respective fields, if trained in positive interdependence and individual accountability. The main objective of this project is to develop a cooperative learning model at the University of Dammam, which would include learning strategies to aid the university deliver its vision and mission as well

xv

INTRODUCTION

as apply cooperative learning within university departments. This project, designed to transform students from passive spectators in the classroom to partners in their own learning, will benefit higher education as well as broader society. Students will enter the world of work equipped with the essential skills of teamwork and accountability for their actions.

In Chapter 6, Abdulghani Ali Al-Hattami and Arif Ahmet Mohamed Hassan Al-Ahdal discuss how scientific research plays an important role in creating growth and progress in developing countries. Many Arab countries, including Saudi Arabia, are trying to encourage faculty members at all universities to write as many good-quality research papers as possible and thereby benefit from the results that can serve society better. This study examines the relationship between university faculty members’ competence to statistically analyze data and their production of research. Results explain why many academics revert to writing qualitative research. The authors recommend that faculty members in Saudi universities improve their knowledge about the use and implementation of statistical tests.

In Chapter 7, Akram AbdulCalder discusses how faculty motivation is a key factor in the success of academic programs in higher education. His study analyzed 34 years of research conducted on numerous variables affecting faculty motivation in Saudi Arabia’s higher education institutions. A meta-analysis of 1,560 journal articles resulted in 35 relevant studies that identified eight factors: 1) monetary incentives and wealth, 2) leadership and management, 3) recognition, 4) technology, 5) students, 6) local culture and language, 7) physical environment, and 8) nepotism. The majority of studies indicated that advances in technology, including distance learning, had an impact on motivation. The role of leadership in motivating faculty was highlighted in 71.42 percent of studies analyzed, indicating a need for those in academic management to more effectively motivate their faculty. A model was developed based on the synthesis of 35 articles used in this meta-analysis study.

In Chapter 8, Tariq Elyas and Abdullah Al-Garni reflect on the ways integrating new technology in the Saudi education system has stirred a desire to move from traditional teacher-centered methods to participatory classrooms. Over the last decade, considerable progress has been made in the development of distance education across the world. Despite the numerous advantages of distance education, universities face several challenges, including the quality of distance education programs (Galusha, 2011; Stephens, 2007). For some students, the lack of a formal classroom setting, and the potential distractions that come with studying at home, may hamper their concentration and motivation to study (Galusha, 2011). In recent decades, Saudi Arabia has developed its human resources considerably in order to become less dependent on foreign labor. A critical aspect of this development has been the establishment of a strong higher-education infrastructure, including developing distance education programs, most notably the Saudi Electronic University (SEU), an institution accessible across the country. Given the recent establishment of the SEU and the dropout rate of students, there is a strong impetus to develop and

xvi

INTRODUCTION

design higher quality curricula for distance education programs at SEU. Further, in order to take measures to address the challenges faced by SEU, the Ministry of Higher Education (MoHE) needs to be able to identify these challenges. This chapter explores this gap in the literature in terms of theoretical and methodological frameworks.

In Chapter 9, El-Sadig Yahya Ezza and Nasser Al-Jarallah discuss how English for Academic Purposes (EAP) is thought to stem from foreign learners’ need to overcome academic problems caused by the use of English as a medium of instruction. The study explores EAP practice at Majma’ah University (MU), focusing on the current EAP syllabus and subject-specific stakeholders (administration, faculty and students). EAP faculty were interviewed to discover how MU approaches and practices EAP and how subject-specific faculty and students at Colleges of Applied Medical Sciences and Medicine perceive the need for EAP skills. Major steps have been taken to improve students’ EAP skills to be able to deal with specialist materials and follow classes conducted in English. On the other hand, interviews with students and subject-specific faculty identified that no EAP courses were offered despite the urgent need for advanced EAP skills. Also, researchers have recently extended the scope of EAP to include the enculturation of students into their respective academic fields. This approach introduces students to the academic discourse used by doctors, engineers, and programmers, for instance, so that they can succeed academically.

In Chapter 10, Mubina Rauf reflects on the Preparatory Year Programs (PYP) in Saudi Arabia, which are designed to equip pre-university students with 21st-century skills and to enable them to communicate effectively with the rest of the world. English language is the most significant part of these programs and is taught following the communicative method. This approach is also practiced in the University of Dammam PYP. This study explores the assessment practices in the English department at the PYP. The chapter discusses and compares current assessment trends to best practices in this field; it also reviews best practices in exam procedures starting from types of tests and their suitability to the current program at UoD, best practices in item writing, trialling, writing exam specifications and rubrics, and administration of exams. An assessment framework based on the test qualities, most importantly validity and reliability presented by Bachman and Palmer (1996) and adapted to the needs of Saudi students, has been suggested for use. Assessment based on time-tested theories is a strong indicator of the quality of a language-learning program like the PYP. Students should be exposed to various kinds of testing methods, as this will bring out their best performances, depending on their proclivities and needs.

In Chapter 11, Osama M. Nurain, Ahmed T. Braima, and Barakat H. Makrami discuss teaching operations management using hands-on educational tools and interactive games. Although technology has been widely used for enhancing language teaching and learning since the 1960s, some EFL faculties are still reluctant to use it on a large scale. In light of this, the present study attempts to determine the factors that cause EFL teachers not to use technology in their classes. The study

xvii

INTRODUCTION

used a quantitative method; 152 EFL instructors at Jazan University were given a questionnaire designed to measure their perception of technology-integration strategies. The study identifies a number of issues that discuss the scale of technology-integration strategies – among them, lack of experience with technology or, in some cases, resistance to technology. The most critical factor that hinders the use of technology is inadequate training programs. Therefore, the researchers suggest that EFL faculties be exposed to technology-implementation strategies, which will assist them to more successfully integrate these strategies into classroom instruction.

All papers went through double blind review anonymously by scholars in education.

A. K. Hamdan (Ed.), Teaching and Learning in Saudi Arabia, 1–30. © 2015 Sense Publishers. All rights reserved.

REIMA AL-JARF

1. ASSeSSIng efL CoLLege InSTRuCToRS’ PeRfoRmAnCe wITH dIgITAL RubRICS

INTRODUCTION

Teacher performance appraisal (TPA) is an indispensible part of educational systems around the world. It constitutes an important element of a university’s vision of achieving high-level student performance. For example, at King Saud University, in Riyadh, Saudi Arabia, TPA is an integral part of the teaching/learning process. Instructors of all levels are annually evaluated by college administrators. They are also evaluated by students enrolled in every course they teach. Standard evaluation forms are used in all subject areas throughout the university. Other institutions perform TPA through ratings made by the students, peers, principals, supervisors, and self-ratings made by the teachers themselves (Alicias, 2005). Many use rubrics that document teacher candidate accomplishments of basic teaching skills and content knowledge. The purpose of TPA is to produce high quality teaching, improve student-learning outcomes, bridge gaps in their achievement and provide instructors with meaningful evaluations that encourage professional learning and growth. It fosters their professional development and creates opportunities for additional professional support where needed. According to Yonghong and Chongde (2006), the purpose of TPA is to improve the quality of instruction, to prescribe professional development activities, and to develop a foundation for administrative decisions. Ideally, the evaluation process produces qualified teachers who meet the needs of their students (Adams, 2009).

Despite the importance of teacher performance appraisal, some experts consider it a challenge and a difficult issue in educational reform. Findings of prior research revealed some problems and abuses of TPA such as: subjectivity, and vulnerability to the “politics of teacher evaluation,” as well as professional incapacities of the raters (Alicias, 2005). Most teachers do not feel that they benefit professionally from TPAs, although they are in favor of being evaluated (Adams, 2009). Pimpa (2005) reported three problems related to the national teacher performance appraisal system in Thailand: mismanagement of the system; problems arising from the evaluators; and problems arising from the applicants.

For more objective and effective TPA schemes, several researchers proposed some guidelines such as: (i) mutual goal setting, use of objective criteria for classroom observation, alternative data collection, evaluator training, teacher involvement, peer

R. AL-JARF

2

observation, classroom improvement, in-service training, availability of resources, emphasizing the positive qualities of teacher evaluation, and use of a reward system (Whyte, 1986); (ii) training evaluators to become better evaluators of teachers’ performance (Weast et al., 1996); (iii) viewing videotaped lessons and applying evaluation standards to enable comparison of outcomes of evaluation (Kuligowski et al., 1993); (iv) implementing a teacher performance appraisal process that encourages self-directed learning, which teachers consider the most effective and meaningful (Rowe, 2000); (v) use of feedback from students, peers, self-evaluation, supervisors, parents, and student achievement (Manatt & Benway, 1998); (vi) use of constructs of teacher performance to diagnose problems and plan appropriate strategies for professional improvement (Yonghong & Chongde, 2006); (vii) using evaluation systems on the web such as the educational electronic performance support systems (EPSS) (Park, Baek, & An, 2001) and the Teacher Performance Appraisal System;1 and (viii) use of TPA rubrics.

To obtain objective, valid, reliable, and meaningful evaluations of instructor performance, rubrics can be of special importance. They are tools for assessing performance according to a set of predetermined scaled expectations and criteria. In a review of empirical research on the use of rubrics at the post-secondary level undertaken in a wide range of disciplines and for multiple purposes, such as student achievement, improving instruction and evaluating programs, Reddy and Andrade (2010) found that student perceptions of rubrics were generally positive. Teachers also reported positive responses to rubric use. Kearns, Sullivan, O’Loughlin, and Braun (2010) created a valid and reliable teaching statement scoring rubric to investigate and document the progression of graduate student instructors as scholarly teachers. The rubric was found to be a useful tool for faculty mentors and instructional consultants who wish to quantify the scholarly progression of writers and modify mentoring practice accordingly. In a third study, performance assessment tasks and rubrics were successfully used in assessing secondary school mathematics preservice teachers’ pedagogical content knowledge and skills (Koirala, Davis, & Johnson, 2008). Similarly, Clinical Experience Rubrics (CER) were found to be effective in rating preservice teachers’ professionalism, teaching quality and relationship with others (Flowers, 2006).

In English as a foreign language (EFL), there is a need for using rubrics in assessing teacher performance objectively and reliably. For example, instructors working at the College of Languages and Translation (COLT), King Saud University are evaluated by college administrators (program coordinators, department heads and vice-deans) at the end of the academic year and by the students at the end of each semester. The following types of TPA forms are used for these purposes:i. a student TPA form that consists of 27 statements with a 5-point scale “strongly

agree, agree, slightly agree, disagree and strongly disagree.”ii. an administrator TPA form for PhD holders which consists of ten statements

under 3 categories: teaching performance (60 points), publications (20 points),

ASSESSING EFL COLLEGE INSTRUCTORS’ PERFORMANCE WITH DIGITAL RUBRICS

3

and academic services (20 points). Each statement is allocated a maximum score of between 4 and 20 points.

iii. an administrator TPA form for teaching assistants and lecturers which consists of 20 statements under 3 categories: professional competence (72 points), personal qualities (16 points) and relationship with others (12 points). Each statement is allocated a maximum score of between 4 and 10 points.

NEED FOR STUDY

Despite the importance of TPA, an exploratory study with a sample of 25 instructors at COLT showed that the majority are dissatisfied with and feel underrated in the PTA reports prepared by the college administrators. Generally speaking, TPAs do not accurately and dispassionately assess instructors’ performance. There are discrepancies and biases in the TPA reports. Evaluations do not reflect their actual performance, as they are not based on classroom visits and observations, samples of teaching practices, course portfolio, course reports, exams and assignments. They are not based on caliber and competence but on favoritism and the instructor’s relationship with the evaluator. They are also affected by clashes and prior conflicts. The currently used TPA forms lack scales for describing excellent, average and poor performances. Statements in the form are not operationally defined and are subject to personal interpretation, especially because administrators are not trained to make the evaluations. Scores given for each statement are not based on criteria for discriminating excellent, average and poor performances. Instructors do not know on what basis they get a score of 4 or 5 out of 10 or 20 points for a particular item on the TPA from.

Another exploratory study with a sample of 50 students at COLT, in addition to a survey-report about students’ evaluations of instructor published in the King Saud University student newspaper Risalatul-Jami’aa,2 showed that 75% of the students participating in the survey do not take the end-of-course TPAs seriously, and do not respond to the items accurately. Some have a friend respond to it on their behalf; some just tick a rating randomly without reading the statements; and others tick the same rating for all the items on the TPA form, as evaluations are conducted during final exams, and students cannot view their course grade until they complete the TPA forms. They also believe that their evaluations have no real effect on teachers and are not taken into consideration in decision-making. They consider them “a routine procedure.” As a result, student TPAs do not reflect an instructor’s actual performance. Both good and poor instructors receive an overall average rating. The currently used TPA forms do not really discriminate an “excellent, average and poor” performance. Students in different sections of the same course taught by the same instructor give significantly different ratings of the instructor and ratings correlate with the grades the students get. The more the failures are in the course, the worse the ratings. Many students do not know how much of behavior marks the difference between “agree” and “slightly agree” or

R. AL-JARF

4

“disagree” and “slightly disagree.” As it is in the case with administrator TPAs, instructors are also dissatisfied with students’ ratings.

To help COLT administrators avoid biases, unfair ratings, personal interpretation of assessment statements, and follow reliable and valid criteria when evaluating teachers, this chapter proposes the use of digital rubrics to assess teachers’ linguistic and professional performance using the iRubric tool of RCampus (www.rcampus.org). It aims at converting the currently used TPA forms at COLT to digital rubrics. The proposed digital rubrics consist of the following versions: (i) a TPA digital rubric to be used by the students; and (ii) a TPA digital rubric to be used by administrators. These digital rubrics will help both administrators and instructors at COLT diagnose problems and plan appropriate strategies for professional improvement.

Digital rubrics are believed to have several advantages. According to RCampus, digital rubrics show levels of performance and what is expected from EFL instructors; serve as a guide for assessors; save time, and empower programs with an easy-to-use system for monitoring teaching performance and aligning with standards. They can be collaboratively assessed with colleagues. Their scores can be automatically adjusted to the teaching assessment scale.

In addition, use of rubrics in assessment has numerous benefits: they set the standards and help specify the criteria to be used in evaluating teachers’ performance. They constitute a purposeful and appropriate construct that articulates varying levels of proficiencies. They guide teachers in their work and help them evaluate its quality. They are easy to use and explain. They communicate department and college expectations clearly, provide instructors with constructive feedback, and support teaching, skill development and understanding (Andrade, 2000).

DEFINITION OF RUBRICS

A rubric is a scoring guide that consists of specific pre-established performance criteria, used for evaluating performance. It is a tool that formalizes the process of evaluation by explicitly stating the criteria and standards to be used for evaluating performance. It is a rating scale – as opposed to checklists – that consists of specific pre-established scaled performance criteria, used in assessing skill levels and/or performance. It also provides guidelines laid out for judging work on performance-based tasks (Luft, 1998; Kist, 2001; Arter, 2000).

Rubrics are generally divided into holistic, analytic, task-specific, general or developmental. A holistic rubric presents a set of generalized descriptions of what “above average, average, and below average” performances are in the developer’s mind; an analytic rubric allows one to measure something against several different criteria; in a developmental rubric, what is being evaluated is put on a continuum of stages; and a metarubric is a rubric for rubrics with four traits: content, clarity, practicality and technical soundness (Kist, 2001; Arter, 2000).


5

WHAT IS IRUBRIC?

iRubric is a comprehensive digital rubric development, assessment and sharing tool that shows the major competencies and sub-competencies to be achieved or skills and subskills to be mastered, the different competency levels and marks allocated to each level. It consists of the following:

i. rubric criteria, which show the characteristics of a performance; they are generally listed in rows

ii. criteria descriptors, which describe what is expected at each level of performanceiii. levels of performance, which describe the quality of work; they are generally

listed in columnsiv. a divider, which is a special row that divides a rubric into sections; it can be

simple, dividing the section without changing the levels, or weighted, allowing changes in level titles and weights for a rubric section

BUILDING THE TPA DIGITAL RUBRICS

According to Kist (2001), Moskal (2003a, 2003b), Marshall (2006), Marzano (2007) and Brown (2008), the process of developing a rubric consists of several steps:

• identifying the purpose or objectives of the activity in measurable outcomes; the scoring rubric should be clearly aligned with the requirements of the task and the program goals and objectives; they should be expressed in terms of observable behaviors or product characteristics, and should be written in specific and clear language;

• identifying the qualities that need to be displayed in a teacher’s work, i.e., expectations for both process and product;

• identifying which type of rubric should be used (analytic or holistic);• defining the performance levels (excellent, average, poor), using observable

behaviors and descriptions of the work (rather than judgments about the work) and marks (points) allocated to each level; scoring rubrics should be written in specific and clear language and a clear separation between score levels.

Based on the above criteria, the author converted (re-designed) the TPA forms currently used by COLT students and administrators to digital rubrics using the iRubric tool of RCampus, an Online Course Management System. The study does not aim to construct new TPA tools from scratch.

The author pooled the items of the administrator TPA form for PhD holders and the one for teaching assistants and lecturers into a single Administrator TPA Rubric, as there are overlapping items that measure the same skill or behavior. Duplicate statements were deleted. The new Administrator TPA Rubric consists of 26 statements selected from the original TPA form. The statements were grouped under six categories: linguistic competence, teaching performance, professional achievements, academic services, relationships with others and personal qualities. (See Rubric 1.)

R. AL-JARF

6

Similarly, the new Student TPA Rubric consists of the same 27 statements in the original TPA form. The items were grouped under six categories: teaching skills, assessment, feedback, punctuality, relationship with students and overall opinion. (See Rubric 2.)

Each rubric consists of a grid with rows for the teaching criteria (indicators or skills) and columns for the performance levels (poor, fair, very good, excellent). Each statement in the original forms was re-formulated and changed into a criterion that describes the instructor’s behavior in specific, observable and measurable behaviors either by giving an operational definition or examples. Four performance levels were added in the columns and each was defined by describing the excellent, very good, fair and poor performances related to the given criterion. Points were allocated to each performance level: 4 points to the “excellent,” 3 points to the “very good,” 2 points to the “fair,” and 1 point to the “poor” performance. A subsection entitled “Professional Achievements” in Rubric 1 was added for evaluating instructors with a PhD degree. Different points were allocated to each performance level: 8–10 points to the “excellent,” 5–7 points to the “very good,” 3–4 points to the “fair” and 0–2 point to the “poor” performance.

The TPA rubrics, especially the detailed performance criteria that describe the instructor’s behavior, as well as the detailed descriptions of the performance levels, were based on the author’s 26 years of experience in teaching EFL at the college level and her administrative experience at the department and college levels. In addition, several TPA frameworks in the literature, such as Marshall (2006), Marzano (2007), list of teacher and principal practice rubrics approved by the New York State Education Department (NYSED) (2011), and the New York State United Teachers’ (NYSUT) (2011) teacher standards, were reviewed.

Both criteria and performance levels were entered in the rows and columns of the iRubric tool of RCampus, to create the digital rubrics for both administrators and students. The detailed Teacher Performance Appraisal Rubric by College Administrators and the detailed Teacher Performance Appraisal Rubric by Students are shown in Rubric 1 and Rubric 2.

VALIDATING THE TPA IRUBRICS

Both TPA scoring rubrics were validated by having ten professors specialized in EFL and evaluation look at the criteria and performance levels in each rubric and make sure the behaviors, operational definitions, and examples given reflect the important and desired teaching competencies of EFL college instructors and that the descriptions and scales under each performance level are discriminating. Clarifications and modifications were made based on the comments received.

To ensure usability and practicality of the rubrics in evaluating EFL teaching performance, both TPA rubrics were tried out. A sample of five administrators and 20 students used both rubrics to evaluate the teaching performance of a sample of six instructors (two PhDs, two MAs and two TAs). Disagreements in evaluating the


7

Teac

her P

erfo

rman

ce A

ppra

isal

(TPA

) Rub

ric

by C

olle

ge A

dmin

istr

ator

s

Cat

egor

ies

Cri

teri

aPo

or1

Fair

2Ve

ry G

ood

3Ex

celle

nt4

Ling

uist

ic

Com

pete

nce

Cor

rect

use

of l

angu

age

Mak

es m

any

pron

unci

atio

n,

gram

mat

ical

, spe

lling

an

d/or

usa

ge m

ista

kes

Mak

es so

me

pron

unci

atio

n,

gram

mat

ical

, sp

ellin

g an

d/or

us

age

mis

take

s

Cor

rect

pro

nunc

iatio

n;

mak

es fe

w

gram

mat

ical

, spe

lling

an

d/or

usa

ge m

ista

kes

Nat

ive

or n

ear-n

ativ

e pr

onun

ciat

ion,

gra

mm

ar,

spel

ling,

usa

ge. M

akes

no

mis

take

s

Kno

wle

dge

of su

bjec

t mat

ter:

theo

ries

, spe

cific

lang

uage

skill

s, gr

amm

atic

al st

ruct

ures

, Eng

lish

phon

olog

y, an

d vo

cabu

lary

Littl

e kn

owle

dge

Kno

ws s

ome

as

pect

sVe

ry g

ood

know

ledg

e of

mos

t asp

ects

Exce

llent

kno

wle

dge

of a

ll as

pect

s

Teac

hing

Pe

rfor

man

ceC

ours

e go

als:

can

wri

te sp

ecifi

c,

mea

sura

ble,

ach

ieva

ble,

real

istic

, be

havi

oral

obj

ectiv

es; a

chie

ves p

re-

dete

rmin

ed o

bjec

tives

at t

he e

nd o

f cl

ass s

essi

on, u

nit a

nd c

ours

e

Few

Som

eM

ost

All

Con

tent

dis

tribu

tion

over

teac

hing

w

eeks

: bal

ance

d w

eekl

y co

vera

ge o

f sk

ills,

text

s, ex

erci

ses

Cov

ers l

ess t

han

2/3

of th

e ch

apte

rs a

nd

exer

cise

s

Cov

ers 2

/3 o

f the

ch

apte

rs, s

kills

and

ex

erci

ses

Cov

ers 3

/4 o

f the

ch

apte

rs a

nd e

xerc

ises

Cov

ers 9

0–10

0% o

f the

re

quire

d m

ater

ial a

nd

exer

cise

s

Rubr

ic 1

. Tea

cher

per

form

ance

app

rais

al ru

bric

by

colle

ge a

dmin

istr

ator

s

(Con

tinue

d)

R. AL-JARF

8

Pres

enta

tion

of m

ater

ial a

nd

skill

s: u

ses a

dvan

ced

orga

nize

rs,

expl

anat

ion,

lect

ure,

tran

slat

ion,

gr

aphi

c or

gani

zers

, tec

hnol

ogy,

PPT

pres

enta

tion;

giv

es re

al-li

fe, c

oncr

ete,

cl

ear,

fam

iliar

exa

mpl

es; g

ives

ext

ra

info

rmat

ion

not i

n te

xtbo

ok

Rar

ely.

Sam

e te

chni

que;

le

ctur

e; e

xpla

natio

n;

trans

latio

n

Hal

f of t

he

tech

niqu

esM

ost t

echn

ique

s m

entio

ned;

mos

t of

the

time

All

tech

niqu

es m

entio

ned;

al

way

s

Lead

s dis

cuss

ions

skill

fully

: cal

ls o

n al

l stu

dent

s; p

rom

pts a

nd m

otiv

ates

st

uden

ts to

par

ticip

ate

and

gene

rate

id

eas;

giv

es e

xtra

cre

dit;

acce

pts

disa

gree

men

ts, e

ncou

rage

s sel

f-ex

pres

sion

Rar

ely;

doe

s not

w

elco

me

ques

tions

fr

om st

uden

ts; d

oes n

ot

acce

pt a

rgum

ents

or

disa

gree

men

t; ca

lls o

n fe

w st

uden

ts

Som

e of

the

aspe

cts m

entio

ned;

so

met

imes

Mos

t of t

he a

spec

ts

men

tione

d; m

ost o

f th

e tim

e

All

aspe

cts m

entio

ned;

al

way

s

Acc

omm

odat

es in

divi

dual

diff

eren

ces

and

diffe

rent

lear

ning

styl

es: b

y us

ing

audi

tory

, visu

al a

nd ta

ctile

te

chni

ques

, sm

all g

roup

, pai

r wor

k,

oral

pre

sent

atio

n, va

riety

of a

ctiv

ities

; us

es re

med

ial a

nd en

richm

ent e

xerc

ises

for p

oor a

nd g

ood

stude

nts

1 te

chni

que;

som

etim

es;

no re

med

ial a

nd

enric

hmen

t mat

eria

l

2 te

chni

ques

; so

met

imes

3 te

chni

ques

; mos

t of

the

time

Mor

e th

an 4

tech

niqu

es;

alw

ays

Rubr

ic 1

. (C

ontin

ued)

Teac

her P

erfo

rman

ce A

ppra

isal

(TPA

) Rub

ric

by C

olle

ge A

dmin

istr

ator

s

Cat

egor

ies

Cri

teri

aPo

or1

Fair

2Ve

ry G

ood

3Ex

celle

nt4


9

(Con

tinue

d)

Use

s te

achi

ng a

ids:

whi

tebo

ard,

gr

aphi

c or

gani

zers

and

min

d-m

aps,

pict

ures

; aud

io, S

mar

tboa

rd,

LCD

pro

ject

or, P

PTs,

mob

ile,

podc

asts

, vid

eos,

onlin

e co

urse

s, bl

ogs,

foru

ms

Use

s whi

tebo

ard

only

Use

s whi

tebo

ard

and

1 te

achi

ng a

idU

ses w

hite

boar

d an

d 2

teac

hing

aid

sU

ses w

hite

boar

d an

d 3

or

mor

e te

achi

ng a

ids

Cre

ativ

ity in

teac

hing

tech

niqu

esU

ses t

each

er-c

ente

red

tech

niqu

es su

ch a

s le

ctur

ing

Som

etim

es u

ses

a ne

w te

achi

ng

tech

niqu

e; c

hang

es

Inte

rnet

Web

site

s

Use

s Sm

artb

oard

in

clas

s, PP

Ts, f

orum

; se

lect

s and

cha

nges

In

tern

et W

eb si

tes

Inte

grat

es a

nd e

xper

imen

ts

with

tech

nolo

gy e

very

se

mes

ter (

mob

ile, p

odca

sts,

vide

os, o

nlin

e co

urse

s, bl

ogs,

foru

ms)

; use

s Sm

artb

oard

in c

lass

; ro

le-p

lay

Ass

ignm

ents

: giv

es a

pplic

atio

n as

sign

men

ts; c

heck

s ass

ignm

ents

in

clas

s; g

ives

indi

vidu

aliz

ed fe

edba

ck,

show

s err

or lo

catio

n

Rar

ely;

giv

es c

opio

us

hom

ewor

kSo

met

imes

Mos

t of t

he ti

me

Alw

ays

Ass

essm

ent:

exam

s cov

er a

ll to

pics

an

d sk

ills;

use

s a v

arie

ty o

f que

stio

n fo

rmat

s; fo

cuse

s on

appl

icat

ion

ques

tions

; giv

es c

lear

and

spec

ific

inst

ruct

ions

Very

few

asp

ects

; man

y w

eakn

esse

sSo

me

aspe

cts;

nu

mer

ous

wea

knes

ses

Mos

t asp

ects

; few

w

eakn

esse

sA

ll as

pect

s

R. AL-JARF

10

Abi

lity

to im

prov

e co

urse

Just

follo

ws t

extb

ook

Follo

ws t

extb

ook;

bo

rrow

s su

pple

men

tary

m

ater

ial f

rom

co

lleag

ues

Rec

omm

ends

Web

si

tes;

pre

pare

s su

pple

men

tary

m

ater

ial

Dia

gnos

es w

eakn

esse

s in

cur

ricul

um, t

extb

ooks

, ex

ams,

stud

ents

skill

s;

adap

ts o

r sup

plem

ents

cu

rricu

lum

to a

chie

ve

cour

se g

oals

and

mee

t st

uden

ts’ l

ingu

istic

and

pr

ofes

sion

al n

eeds

Ove

rall

teac

hing

effi

cien

cy in

: pr

epar

ing,

pre

sent

ing

mat

eria

l in

clas

s, se

lect

ing

and

perf

orm

ing

lear

ning

task

s, im

prov

ing

stud

ents

’ lin

guis

tic a

bilit

y, gi

ving

feed

back

, co

nstr

uctin

g te

stin

g, a

ssig

nmen

ts,

focu

sing

on

high

er-le

vel s

kills

such

as

appl

icat

ion

Poor

in a

ll ar

ea; a

lway

sG

ood

in so

me

area

s;

som

etim

esVe

ry g

ood

in m

ost

area

s; m

ost o

f the

tim

eEx

celle

nt in

all

area

s;

alw

ays

Cla

ss m

anag

emen

t ski

lls: s

ets c

lass

ru

les a

t the

beg

inni

ng o

f sem

este

r for

m

obile

use

in c

lass

, mis

sing

cla

sses

an

d ex

ams,

talk

ing

in c

lass

, com

ing

late

; tea

ches

dis

cipl

ine,

pun

ctua

lity,

com

mitm

ent;

seri

ousn

ess a

nd

atte

ntiv

enes

s in

clas

s

Littl

e or

no

disc

iplin

e;

too

leni

ent;

no c

ontro

l ov

er a

spec

ts m

entio

ned

Som

e; so

met

imes

Mos

t; m

ost o

f the

tim

eA

ll; a

lway

s

Rubr

ic 1

. (C

ontin

ued)

Teac

her P

erfo

rman

ce A

ppra

isal

(TPA

) Rub

ric

by C

olle

ge A

dmin

istr

ator

s

Cat

egor

ies

Cri

teri

aPo

or1

Fair

2Ve

ry G

ood

3Ex

celle

nt4


11

(Con

tinue

d)

Punc

tual

ityG

oes t

o cl

ass l

ate

and/

or

dism

isse

s stu

dent

s ear

ly

(mor

e th

an 1

0 m

inut

es);

canc

els c

lass

es; d

oes

not m

ake

up fo

r mis

sed

clas

ses;

mis

ses m

ore

than

6 d

ays p

er se

mes

ter

Goe

s to

clas

s lat

e or

le

aves

ear

ly (5

–10

min

utes

) onc

e a

wee

k; m

isse

s 4–5

da

ys p

er se

mes

ter

Goe

s to

clas

s lat

e or

leav

es e

arly

(5

min

utes

); fe

w ti

mes

; m

isse

s 1–3

day

s per

se

mes

ter

Alw

ays g

oes t

o cl

ass o

n tim

e, le

aves

on

time.

No

abse

nces

. Nev

er c

ance

ls

clas

ses.

Mak

es u

p fo

r m

isse

d cl

asse

s

Offi

ce h

ours

and

aca

dem

ic a

dvis

ing

Avai

labl

e oc

casi

onal

ly;

sign

s for

ms

Sign

s for

ms;

giv

es

acad

emic

adv

ice;

av

aila

ble

mos

t of

the

time

Alw

ays i

n of

fice;

si

gns f

orm

s; h

elps

in

regi

stra

tion;

ans

wer

s qu

estio

ns; l

iste

ns to

so

me

prob

lem

s

Alw

ays i

n of

fice;

sign

s fo

rms;

hel

ps in

regi

stra

tion;

an

swer

s que

stio

ns; l

iste

ns

to a

cade

mic

, soc

ial a

nd

psyc

holo

gica

l pro

blem

s and

gi

ves s

olut

ions

; wel

com

es

and

cont

ains

stud

ents

; re

quire

s adv

isee

s to

mee

t w

ith h

er

Rel

atio

nshi

p w

ith o

ther

sW

ith b

oss:

frien

dly;

has

a g

ood

sens

e of

hum

or; c

oope

rativ

e; o

ffers

hel

p;

com

forta

ble;

trus

twor

thy;

resp

ectfu

l; av

oids

conf

licts

Res

entfu

l; di

ssat

isfie

d;

avoi

ds in

tera

ctio

n; la

ck

of tr

ust;

in c

onfli

ct;

unco

oper

ativ

e

Som

e qu

aliti

esM

ost q

ualit

ies

All

qual

ities

With

col

leag

ues:

wor

ks in

a te

am;

shar

es k

now

ledg

e; m

ater

ial a

nd

reso

urce

s; a

dvis

es y

oung

col

leag

ues;

an

swer

s que

stio

ns; f

rien

dly;

has

a

good

sens

e of

hum

or; l

iste

ns, h

elps

w

ith p

robl

ems;

ope

n; c

over

s cla

sses

fo

r oth

ers w

hen

requ

este

d

Dis

play

s neg

ativ

e be

havi

ors w

ith

mos

t col

leag

ues;

in

conf

lict w

ith o

ther

s;

unco

oper

ativ

e

Dis

play

s som

e of

th

ese

beha

vior

s;

som

etim

es; w

ith

som

e co

lleag

ues

Dis

play

s mos

t be

havi

ors;

ver

y of

ten;

w

ith m

ost c

olle

ague

s

Dis

play

s all

beha

vior

s;

alw

ays;

with

mos

t co

lleag

ues

R. AL-JARF

12

With

stud

ents

and

par

ents

: fri

endl

y;

help

ful;

cour

teou

s; re

spec

tful;

enco

urag

ing;

list

ens t

o pr

oble

ms;

pr

ovid

es su

ppor

t and

hel

p

Rar

ely;

unfr

iend

ly; u

nhel

pful

; di

scou

rteou

s;

disc

oura

ging

; doe

s not

lis

ten

to p

robl

ems;

and

/or

uns

uppo

rtive

Som

e qu

aliti

es;

with

som

e st

uden

ts;

som

etim

es

Mos

t qua

litie

s; w

ith

mos

t stu

dent

s or m

ost

of th

e tim

e

All

qual

ities

; alw

ays;

with

al

l stu

dent

s

Pers

onal

Q

ualit

ies

Act

ing

as a

role

-mod

el: p

unct

ual,

does

not

mis

s cla

sses

; pro

fess

iona

l; pr

epar

es; p

rese

nts m

ater

ial i

n cl

ass;

se

lect

s and

exe

cute

s lea

rnin

g ta

sks

wel

l; im

prov

es st

uden

ts li

ngui

stic

ab

ility

; giv

es fe

edba

ck; g

ood

test

s an

d as

sign

men

ts; f

ocus

es o

n hi

gher

-le

vel s

kills

such

as a

pplic

atio

n

Rar

ely

Som

etim

esM

ost o

f the

tim

eA

lway

s

Und

erta

kes r

espo

nsib

ility

: tak

es c

are

of o

wn

exam

s, co

urse

spec

ifica

tions

an

d re

port

s and

oth

er p

aper

wor

k;

carr

ies o

ut d

utie

s in

time

and

as

requ

ired

Rar

ely

Som

etim

esM

ost o

f the

tim

eA

lway

s

Acc

epts

rem

arks

, gui

danc

e an

d fe

edba

ck fr

om su

perio

rsR

arel

ySo

met

imes

Mos

t of t

he ti

me

Alw

ays

Rubr

ic 1

. (C

ontin

ued)

Teac

her P

erfo

rman

ce A

ppra

isal

(TPA

) Rub

ric

by C

olle

ge A

dmin

istr

ator

s

Cat

egor

ies

Cri

teri

aPo

or1

Fair

2Ve

ry G

ood

3Ex

celle

nt4


13

Take

s pro

per a

ctio

n in

cris

es a

nd

solv

es p

robl

ems

Ref

ers t

o de

partm

ent

head

or c

oord

inat

or,

does

not

take

act

ion;

es

cape

s; ig

nore

s si

tuat

ion

Som

etim

esM

ost o

f the

tim

eA

lway

s

Crit

eria

Poor

0–2

Fair

3–4

Very

Goo

d5–

7Ex

celle

nt8–

10

Prof

essi

onal

ac

hiev

emen

tsPu

blic

atio

ns:*

boo

ks, r

esea

rch

artic

le

(pee

r-re

view

ed),

tran

slat

ed b

ook,

ar

ticle

, tra

nsla

ted

artic

le, r

evie

ws

and/

or a

udio

visu

al d

ocum

enta

ries

Non

eA

ny 1

Any

2A

ny 3

or m

ore

Prof

essi

onal

self-

deve

lopm

ent:

give

s co

nfer

ence

pre

sent

atio

ns, w

orks

hops

, pu

blic

lect

ures

, TV

and

radi

o in

terv

iew

s; su

bscr

ibes

to jo

urna

ls

Giv

es 1

or l

ess

Giv

es a

ny 2

Giv

es a

ny 3

Giv

es a

ny 4

or m

ore

Aca

dem

ic se

rvic

es: a

ttend

s de

part

men

t and

col

lege

mee

tings

; se

rves

on

com

mitt

ees;

atte

nds

wor

ksho

ps, c

onfe

renc

es, p

ublic

le

ctur

es, c

olle

ge e

vent

s; w

rite

s re

port

s

1 or

less

Any

2A

ny 3

Atte

nds a

ny 4

or m

ore

* Th

is se

ctio

n is

for e

valu

atin

g Ph

D h

olde

rs o

nly.

R. AL-JARF

14

Teac

her P

erfo

rman

ce a

ppra

isal

rubr

ic b

y st

uden

ts

Cat

egor

ies

Cri

teri

aPo

or 1

Fair

2Ve

ry G

ood

3Ex

celle

nt 4

Teac

hing

Ski

llsPr

ovid

es st

uden

ts w

ith c

ours

e

desc

riptio

nPr

ovid

es st

uden

ts w

ith

text

book

, cre

dit h

ours

, co

urse

des

crip

tion,

mar

ks,

num

ber o

f tes

ts, t

est d

ates

Prov

ides

stud

ents

w

ith c

ours

e tit

le

and

code

, tex

tboo

k,

cred

it ho

urs,

cour

se d

escr

iptio

n,

asse

ssm

ent,

mar

ks,

a sa

mpl

e te

st, t

est

date

s

Prov

ides

stud

ents

w

ith c

ours

e tit

le

and

code

, tex

tboo

k,

cred

it ho

urs,

leve

l, co

ntac

t inf

orm

atio

n,

offic

e ho

urs,

cour

se

desc

riptio

n, m

ater

ial

cove

rage

, ski

lls,

asse

ssm

ent,

mar

ks,

num

ber o

f tes

ts,

rela

ted

Web

site

s, a

sam

ple

test

, tes

t da

tes

Prov

ides

stud

ents

with

co

urse

title

and

cod

e,

text

book

, cre

dit h

ours

, le

vel,

pre-

requ

isite

s, co

-req

uisi

tes,

loca

tion,

co

ntac

t inf

orm

atio

n,

offic

e ho

urs,

teac

hing

ph

iloso

phy,

cou

rse

desc

riptio

n, a

udio

visu

als,

mat

eria

l cov

erag

e, sk

ills,

teac

hing

tech

niqu

es,

asse

ssm

ent,

mar

ks,

num

ber o

f tes

ts, s

tudy

sk

ills,

exte

nsio

n ac

tiviti

es, t

echn

olog

y,

rela

ted

Web

site

s, se

lf-im

prov

emen

t, a

sam

ple

test

, tes

t dat

es

Stat

es o

bjec

tives

and

skill

sR

arel

yFe

w ti

mes

per

se

mes

ter o

r onc

e a

mon

th

Som

e cl

ass s

essi

ons

per w

eek

(onc

e a

wee

k)

Ever

y cl

ass s

essi

on

Com

es to

cla

ss w

ell-p

repa

red:

read

y ha

ndou

ts; k

now

s tas

ks a

nd se

quen

ce;

mov

es fr

om o

ne st

ep/s

kill

to a

noth

er

smoo

thly

; dis

trib

utes

cla

ss ti

me

on ta

sk

Rar

ely

Occ

asio

nally

Mos

t of t

he ti

me

Alw

ays

Rubr

ic 2

. Tea

cher

per

form

ance

app

rais

al ru

bric

by

stud

ents


15

Kno

ws m

ater

ial v

ery

wel

l: gi

ves

back

grou

nd in

form

atio

n no

t in

text

book

; gi

ves t

heor

y, ru

les,

histo

ry; g

ives

add

ition

al

exam

ples

; writ

es su

pple

men

tary

mat

eria

l

Stic

ks to

mat

eria

l and

ex

erci

ses i

n te

xtbo

okO

ccas

iona

lly;

give

s som

e ex

erci

ses o

r tex

t re

late

d to

few

topi

cs

or c

hapt

ers

Mos

t of t

he ti

me;

m

ost t

opic

s in

text

book

Alw

ays;

all

topi

cs

cove

red

in te

xtbo

ok

Use

s a v

arie

ty o

f tea

chin

g te

chni

ques

: ex

plan

atio

n, le

ctur

e, tr

ansl

atio

n, g

raph

ic

orga

nize

rs a

nd m

ind-

map

s, pi

ctur

es,

onlin

e co

urse

, onl

ine

Web

site

s, pa

ir

wor

k, sm

all g

roup

s, st

uden

ts g

ive

oral

pre

sent

atio

n, P

PT p

rese

ntat

ion,

de

pend

ing

on c

onte

nt, s

kill

and

task

Sam

e te

achi

ng te

chni

que

alw

ays;

2 o

r les

s te

chni

ques

Use

s 3 te

chni

ques

Use

s 4 te

chni

ques

Use

s 5 o

r mor

e of

thos

e te

chni

ques

Con

nect

s mat

eria

l with

real

-life

: tel

ls u

s ho

w m

ater

ial a

pplie

s to

our s

peci

alty

and

fu

ture

car

eer a

nd w

ith o

ther

cou

rses

Rar

ely

Few

tim

es p

er

sem

este

rM

ost o

f the

tim

eA

lway

s

Giv

es c

lear

, con

cret

e, fa

mili

ar e

xam

ples

an

d an

alog

ies t

o cl

arify

and

sim

plify

m

ater

ial

Rar

ely

Occ

asio

nally

Mos

t of t

he ti

me

Alw

ays

Inst

ruct

or is

ent

husi

astic

and

inte

rest

ed

in w

hat s

he is

teac

hing

Alw

ays b

ored

; stu

dent

s fal

l as

leep

in c

lass

Occ

asio

nally

Mos

t of t

he ti

me

Alw

ays

Spea

ks a

nd re

ads c

lear

ly w

ith a

loud

vo

ice,

goo

d ex

pres

sion

, mod

erat

e sp

eed

and

a co

mpr

ehen

sibl

e ac

cent

Spea

ks w

ith a

low

m

onot

onou

s, vo

ice;

spea

ks

fast

; rea

ds fa

st a

nd/o

r has

a

diffi

cult

acce

nt

Has

num

erou

s w

eakn

esse

sH

as fe

w w

eakn

esse

s in

som

e as

pect

sPe

rfec

t in

all o

f the

se

aspe

cts

Enco

urag

es st

uden

ts to

par

ticip

ate

in

clas

s: c

alls

on

all s

tude

nts;

giv

es b

onus

m

arks

for p

artic

ipat

ing;

pro

mpt

s

Rar

ely;

focu

ses o

n fe

w

stud

ents

; ask

s tho

se w

ho

rais

e th

eir h

ands

Som

e of

thes

e sk

ills;

occ

asio

nally

Mos

t of t

hese

skill

s;

mos

t of t

he ti

me

Alw

ays

(Con

tinue

d)

R. AL-JARF

16

Enco

urag

es st

uden

ts to

read

from

m

ultip

le re

sour

ces:

boo

ks, e

-boo

ks,

onlin

e m

ater

ial,

mob

ile, n

ewsp

aper

s, m

agaz

ine

Focu

ses o

n te

xtbo

ok o

nly

Occ

asio

nally

; re

com

men

ds 1

so

urce

Mos

t of t

he ti

me;

re

com

men

ds 2

so

urce

s

Alw

ays;

reco

mm

ends

at

leas

t 3 so

urce

s

Enco

urag

es c

ritic

al a

nd c

reat

ive

thin

king

su

ch a

s giv

ing

reas

ons a

nd so

lutio

ns to

pr

oble

ms;

just

ifyin

g; w

ritin

g th

eir o

wn

poet

ry o

r sto

ries

Nev

erO

ccas

iona

llyM

ost o

f the

tim

eA

lway

s

Ass

essm

ent

Use

s a v

arie

ty o

f eva

luat

ion

tech

niqu

es:

long

test

, sho

rt te

sts,

pop-

quiz

zes,

shor

t an

swer

, ess

ay, p

artic

ipat

ion,

onl

ine

cour

se, p

rese

ntat

ion,

hom

ewor

k, c

lass

w

ork

and/

or p

aper

Use

s sam

e ev

alua

tion

tech

niqu

e; u

ses 2

in-te

rm

test

s onl

y

Use

s in-

term

test

s an

d qu

izze

sU

ses 3

–5 te

chni

ques

Use

s 6 o

r mor

e te

chni

ques

Exam

que

stio

ns a

re c

ompr

ehen

sive

Que

stio

ns c

over

less

than

50

% o

f ski

lls a

nd c

onte

nt

taug

ht

Que

stio

ns c

over

66

% o

f the

skill

s an

d co

nten

t tau

ght

Que

stio

ns c

over

75

% sk

ills a

nd

topi

cs ta

ught

Que

stio

ns c

over

mor

e th

an 9

0% o

f ski

lls a

nd

topi

c ta

ught

Exam

que

stio

ns a

re c

lear

and

co

mpr

ehen

sibl

eM

ost q

uest

ions

are

co

nfus

ing

Som

e qu

estio

ns;

man

y ar

e co

nfus

ing

Mos

t que

stio

nsA

ll qu

estio

ns

Bala

nced

dist

ribut

ion

of m

arks

ove

r cou

rse

requ

irem

ents:

in-te

rm te

sts, p

op q

uizz

es,

parti

cipa

tion,

onl

ine c

ours

e, pr

esen

tatio

n,

hom

ewor

k, cl

ass w

ork a

nd/o

r pap

er

Cou

rse

mar

ks c

over

onl

y in

-term

test

sC

ours

e m

arks

cov

er

in-te

rm te

st a

nd 1

re

quire

men

t

Cou

rse

mar

ks c

over

in

-term

test

s and

any

2

requ

irem

ents

Cou

rse

mar

ks c

over

in-

term

test

s and

3 o

r mor

e re

quire

men

ts

Teac

her P

erfo

rman

ce a

ppra

isal

rubr

ic b

y st

uden

ts

Cat

egor

ies

Cri

teri

aPo

or 1

Fair

2Ve

ry G

ood

3Ex

celle

nt 4

Rubr

ic 2

. (C

ontin

ued)


17

Fair

in as

sess

ing

stude

nts’

perfo

rman

ce:

uses

an

answ

er ke

y; sh

ows h

ow m

arks

and

fra

ctio

ns w

ere a

lloca

ted

to ea

ch a

nswe

r;

give

s sam

e mar

k to

sam

e ans

wer t

o al

l stu

dent

s, de

duct

s sam

e mar

ks fo

r sam

e er

ror t

o al

l stu

dent

s

Man

y in

cons

isten

cies

; doe

s no

t ret

urn

pape

rs to

stud

ents;

or

stud

ents

do n

ot k

now

why

m

arks

wer

e ded

ucte

d

Som

e in

cons

iste

ncie

sFe

w in

cons

iste

ncie

sA

ll of

thos

e; n

o in

cons

iste

ncie

s in

allo

catio

n of

mar

ks

Feed

back

Giv

es in

divi

dual

ized

feed

back

and

sh

ows e

rror

sO

ccas

iona

lly; s

ome

feed

back

to w

hole

cla

ssSo

met

imes

to so

me

stud

ents

Mos

t of t

he ti

me

to

mos

t stu

dent

sA

lway

s; to

eve

ry st

uden

t

Follo

ws u

p st

uden

ts’ p

rogr

ess:

giv

es

com

men

ts o

n ar

eas o

f im

prov

emen

t; pr

ovid

es se

lf-im

prov

emen

t tip

s

Nev

erC

omm

ents

on

few

st

uden

ts’ p

rogr

ess

or w

hen

aske

d by

st

uden

ts

Mos

t of t

he ti

me.

So

me

stud

ents

Alw

ays;

all

stud

ents

Prov

ides

stud

ents

with

thei

r gra

des;

sh

ows s

tude

nts t

heir

test

pap

ers

Doe

s not

retu

rn p

aper

s to

stud

ents

; pos

ts g

rade

s onl

yM

ore

than

10

days

With

in 7

–10

days

; al

way

s; a

ll qu

izze

sIn

less

than

a w

eek;

al

way

s; a

ll qu

izze

s

Punc

tual

ityC

omes

to c

lass

on

time

Alw

ays l

ate

to c

lass

or

leav

es c

lass

ear

ly (m

ore

than

10

min

utes

)

Com

es la

te o

r le

aves

ear

ly m

ost

of th

e tim

e (5

–10

min

utes

)

Com

es to

cla

ss o

n tim

e an

d le

aves

on

time

mos

t of t

he

time

or c

omes

late

or

leav

es e

arly

(5

min

utes

)

Alw

ays c

omes

to c

lass

on

time

and

leav

es o

n tim

e

Abi

des b

y cl

ass d

urat

ion

Teac

hes l

ess t

han

40

min

utes

mos

t of t

he ti

me

Teac

hes f

ull c

lass

so

met

imes

or l

ess

than

50

min

utes

m

any

times

Teac

hes f

ull c

lass

m

ost o

f the

tim

e or

le

ss th

an 5

0 m

inut

es

few

tim

es

Teac

hes f

ull c

lass

(50

min

utes

or m

ore) (Con

tinue

d)

R. AL-JARF

18

Rel

atio

nshi

p w

ith st

uden

tsR

espe

cts s

tude

nts;

use

s pol

ite

expr

essi

ons s

uch

as “

plea

se, t

hank

yo

u, e

xcus

e m

e, I

am so

rry”

list

ens t

o st

uden

ts’ p

robl

ems;

acc

epts

crit

icis

m

and

com

plai

nts

Rar

ely

Occ

asio

nally

Mos

t of t

he ti

me

Alw

ays

Wel

com

es q

uerie

s fro

m st

uden

tsR

arel

y; v

ery

few

que

ries

Occ

asio

nally

; som

e qu

erie

sM

ost o

f the

tim

eIn

and

out

of c

lass

; al

way

s; a

ll qu

erie

s:

whe

ther

rela

ted

or

unre

late

d to

cou

rse;

fu

lly a

nsw

ers s

tude

nts’

ques

tions

Inte

ract

s with

stud

ents

thro

ugh

emai

l, SM

S, a

Cha

tBox

, Mes

seng

er a

nd/o

r Fa

cebo

ok/T

witt

er

Rar

ely

Use

s 1 m

etho

d;

occa

sion

ally

Use

s 2 m

etho

ds;

very

ofte

nU

ses 3

or m

ore

met

hods

; al

way

s

Can

be

reac

hed

durin

g of

fice

hour

sR

arel

ySo

met

imes

Mos

t of t

he ti

me

Alw

ays

Ove

rall

opin

ion

I wou

ld li

ke to

take

oth

er c

ours

es

with

this

teac

her

Rar

ely

If I

have

toM

aybe

Def

inite

ly

Teac

her P

erfo

rman

ce a

ppra

isal

rubr

ic b

y st

uden

ts

Cat

egor

ies

Cri

teri

aPo

or 1

Fair

2Ve

ry G

ood

3Ex

celle

nt 4

Rubr

ic 2

. (C

ontin

ued)


19

same instructor’s performance were noted and necessary amendments were made on the criteria and descriptions of performance levels.

APPLYING THE TPA DIGITAL RUBRICS

The digital form of the TPA rubrics can be used in evaluating a teacher’s performance using the iRubric tool of RCampus. Steps in applying the digital rubric to an instructor’s performance, entering the scores, for viewing the instructor’s scores, sharing and discussing the rubric with other instructors, and categorizing the rubric are shown in Web pages 1–19 in Appendix A.

To report an instructor’s performance assessment summary, all of the scores given to criteria (indicators) according to the selected performance levels are added up. In addition to that, the percentage of students marking each performance level for each criterion (indicator) is given.

CONCLUSION

This chapter proposed the use of digital rubrics using the iRubric creator tool of RCampus for evaluating EFL college instructors’ linguistic and professional performance by administrators and students to ensure the reliability, accuracy and fairness of assessments. The proposed iRubrics consist of pre-established performance criteria, four performance levels and marks allocated to each. Performance criteria can be modified and performance levels can be adjusted easily. The effective use of iRubrics requires that they be explained to administrators, teachers and students ahead of time and that training be provided in their use. Guidance should be offered for using the ratings, in scoring, interpreting and using the results. With the rubric as a guide, teachers learn to monitor their own progress and make improvements in a timely manner. Involving teachers in creating rubrics encourages them to think about the criteria of quality work and promotes ownership of the assessment process. Revising and modifying EFL instructors’ evaluation policies, procedures and tools at COLT by using digital rubrics, as those devised in the present study, will lead to more satisfaction and better achievement of teaching and learning goals.

NOTES

1 http://www.edu.gov.on.ca/eng/teacher/appraise.html2 Risalatul-Jami’aa, issue#1078, p. 38, December 10, 2011.

REFERENCES

Adams, A. (2009). A study of the attitudes and opinions of southwest Missouri educators regarding the value and outcome of the performance based teacher evaluation process. (ERIC Document No. ED513646)

R. AL-JARF

20

Alicias, E. (2005). Toward an objective evaluation of teacher performance: The use of variance partitioning analysis, VPA. Education Policy Analysis Archives, 13(30).

Andrade, H. (2000). Using rubrics to promote thinking and learning. Educational Leadership, 57(5), 13–18.Arter, J. (2000). Rubrics, scoring guides, and performance criteria: Classroom tools for assessing and

improving student learning. (ERIC Document No. ED446100)Brown, C. (2008). Building rubrics: A step-by-step process. Library Media Connection, 26(4), 16–18.Flowers, C. (2006). Confirmatory factor analysis of scores on the clinical experience rubric: A measure

of dispositions for preservice teachers. Educational and Psychological Measurement, 66(3), 478–488.Kearns, K., Sullivan, C., O’Loughlin, V., & Braun, M. (2010). A scoring rubric for teaching statements:

A tool for inquiry into graduate student writing about teaching and learning. Journal on Excellence in College Teaching, 21(1), 73–96.

Kist, B. (2001). Using rubrics:Teacher to teacher. (ERIC Document No. ED458392)Koirala, H., Davis, M., & Johnson, P. (2008). Development of a performance assessment task and rubric

to measure prospective secondary school mathematics teachers’ pedagogical content knowledge and skills. Journal of Mathematics Teacher Education, 11(2), 127–138.

Kuligowski, B., Holdzkom, D., & French, R. (1993). Teacher performance evaluation in the southeastern states: Forms and functions. Journal of Personnel Evaluation in Education, 6(4), 335–358.

Luft, J. (1998). Rubrics: Design and use in science teacher education. (ERIC Document No. ED417145)Manatt, R., & Benway, M. (1998). Teacher and administrator performance evaluation: Benefits of

360-degree feedback. ERS Spectrum, 16(2), 18–23.Marshall, K. (2006, September-October). Teacher evaluation rubrics: The why and the how. Edge

Magazine, 1–25.Marzano, R. (2007). The art and science of teaching. Retrieved April 30, 2015, from

http://usny.nysed.gov/rttt/teachers-leaders/practicerubrics/Docs/LSI_Domains1234.pdfMoskal, B. (2003a). Developing classroom performance assessments and scoring rubrics – Part I. (ERIC

Document No. ED481714)Moskal, B. (2003b). Developing classroom performance assessments and scoring rubrics – Part II.

(ERIC Document No. ED481715)New York State Education Department (NYSED). (2011). Teacher and principal practice rubrics.

Retrieved April 30, 2015, from http://usny.nysed.gov/rttt/teachers-leaders/practicerubrics/home.htmlNYSUT. (2011). New York state teacher standards. Retrieved April 30, 2015, from http://www.nysut.org/

cps/rde/xchg/nysut/hs.xsl/research_16718.htmPark, S., Baek, E., & An, J. (2001). Usability evaluation of an educational electronic performance support

system (E-EPSS): Support for teacher enhancing performance in schools (STEPS). (ERIC Document No. ED470191)

Pimpa, N. (2005). Teacher performance appraisal in Thailand: Poison or panacea? Educational Research for Policy and Practice, 4(2–3), 115–127.

Reddy, Y., & Andrade, H. (2010). A review of rubric use in higher education. Assessment and Evaluation in Higher Education, 35(4), 435–448.

Rowe, B. (2000). The influence of teacher efficacy and readiness for self-directed learning on the implementation of a growth-oriented teacher performance appraisal process. (ERIC Document No. ED444942)

Weast, J., Wright, J. S., & Frye, S. (1996). Raising teacher performance by improving teacher evaluation: Guilford County’s project H.E.L.P. ERS Spectrum, 14(3), 3–8.

Whyte, J. (1986). Teacher assessment: A review of the performance appraisal literature with special reference to the implications for teacher appraisal. Research Papers in Education, 1(2), 137–163.

Yonghong, C., & Chongde, L. (2006). Theory and practice on teacher performance evaluation. Frontiers of Education in China, 1(1), 29–39.

Reima Al-JarfKing Saud UniversityRiyadh, Saudi Arabia


21

APPENDIx A

Steps of Creating and Applying Teacher Performance Appraisal (TPA) iRubrics

Web page 1: The RCampus iRubric Homepage

Web page 2: Go to the RCampus iRubric homepage (http://www.rcampus.com) and register for an account. All teachers must be enrolled in a group, like a class

R. AL-JARF

22

Web page 4: The rubric building page. Fill in the required information, the criteria and performance levels in the row

Web page 3: To build a new digital rubric, click “Build a rubric” and click start for Option A


23

Web page 5: These action buttons appear underneath the rubric after saving it. Click any of the links if you need to preview, edit, copy, print, categorize, bookmark, test-run, grade, collaborate, publish, email, or discuss the rubric that you have built. An explanation of each action button is given

R. AL-JARF

24

Web page 6: To apply the rubric to a teacher’s performance, click “apply to” in the blue horizontal menu above


25

Web page 7: Select ‘Apply the rubric to a coursework’ for assessment by an individual assessor or ‘Apply rubric to an object’ for assessment by a group

Web page 8: Before starting a teacher’s assessment by a single assessor, define the coursework properties. Select the assessment title and type of coursework to be evaluated, i.e., ‘teacher assessment’ from the drop-down menus. The rubric will be attached to assignment as in a regular course assignment

R. AL-JARF

26

Web page 9: For collaborative assessment, fill in the information

Web page 10: For collaborative assessment, fill in the information to select the assessment evaluators and ‘Building group’ for group of instructors to be evaluated


27

Web page 11: To enter a teacher’s assessment and score, click on the rubric icon adjacent to his/her name. This will transfer you to the rubric page below

Web page 12: To assess a teacher’s performance, go through the criteria/indicators row by row. Click on the relevant performance level per criterion/indicator to select it. Once finished, click on “show score” below the rubric. The teacher’s total score will be automatically calculated and entered in the gradebook

R. AL-JARF

28

Web page 13: Comments can be entered by clicking on a teacher’s name in Web page 14

Web page 14: To view all teachers’ assessments scores, click ‘gradebook’


29

Web page 15: For a single teacher’s assessments details, click on his/her name in the list

Web page 16: Discussing the rubric and evaluations with other administrators or teachers

R. AL-JARF

30

Web page 17: To share a rubric with other administrators or teachers, select one of the 4 options listed

Web page 18: To categorize the TPA rubric, select the subject area (foreign languages)and the type of task (assessment) to which the rubric will be applied

teaching and learning in saudi arabia perspectives from ... · pdf fileteaching and learning...

Documents