identity, positioning, and standardized language testing from an ielts examiner perspective: an...

Upload: louis-carter

Post on 14-Oct-2015

10 views

Category:

Documents


0 download

DESCRIPTION

Final paper for a Master's level Education course

TRANSCRIPT

Identity, Positioning, and Standardized Language Testing from an IELTS Examiner Perspective: an AutoethnographyThe narratorIn order for the following narrative to be meaningful, it is necessary that I situate myself within the context that is to be explored. Firstly, biologically, I identify myself as female and of white ancestry, and by nationhood as Canadian. Secondly, professionally, I reveal myself as having been engaged in the English language teaching profession for approximately 10 years as an instructor and 4 years as an examiner in Standardized English language testing. Thirdly, I perceive myself as educated in that I have had the privilege of completing an undergraduate degree in Biology and am currently in the process on obtaining a graduate degree in Teaching English as an Additional Language. Fourth, I would like to identify myself linguistically native-English speaking, and as having studied, to varying degrees of success French, Portuguese, and Spanish. Finally, I wish to include that I have had the opportunity to live, work, and travel in numerous countries throughout Asia and Latin America. These aforementioned aspects of my identity and socioculturally constructed history I feel are especially pertinent to the story at hand.The International English Language Testing System (IELTS)The IELTS exam is currently the most utilized standardized English language test conducted globally, and as such functions as a gatekeeper on an immense scale. It is accepted by over 7000 organisations and taken by more than 1.7 million candidates each year worldwide, and claims recognition as a secure, valid and reliable indicator of true-to-life ability to communicate in English for education, immigration and professional accreditation. It is offered in more than 130 countries and by more than 800 test centres (www.ielts.org, 4 Aug. 2012). The test is available in two versions, academic and general, and consists of four parts, which measure listening, reading, writing and speaking proficiency. All portions of the test are scored in bands ranging from 1 to 9 where 1 is a Non user of English and 9 is an Expert user. Organizations around the world set the scores that are required for acceptance to or application to their individual organization. In Canada, scores between 6 and 7 are the norm for admittance into universities and professional organizations (www.ielts.org, 4 Aug. 2012), and 6.5 to 7.5 earns one maximum points in the first official language category of the Ability in English or French factor of the Application for permanent residence: Federal skilled worker class. On the IELTS Score Scale, this translates to being a Competent to Good user of English. To put this into perspective, statistics for 2011 showed that on average women scored 6.0 and 6.1 on the academic and general tests respectively, and men 5.8 and 6.1 (www.ielts.org, 4 Aug. 2012). Thus, numerous candidates would have been denied application to organizations in the Canadian context, as well as others worldwide.The TheoryUnder the poststructural and sociocultural umbrella, identity and position hold vastly similar qualities and as such are often used interchangeably. These qualities fall into the following categories.(i) Identity and position are context dependant and context producing. In discussing the work in Hacking, Hawkins (2005, p. 62) explains that people come to fit into socially constructed categories, and that these categories in turn then act to define the positions or subjectivities available. In this way, identity operates recursively as it mutually constitutes and is constituted by socioculturally and historically constructed contexts, and to a large extent espouses their durability. (ii) Identity and position are constrained and augmented by social, cultural and situational processes of power. Power and therefore positions and identities are determined by the possession of cultural capital, which is situated in that it has different exchange value in varied fields (Norton, 2010, p. 3). Atkinson (2003) summarizes Pierre Bourdieus notion of cultural capital as a form of historically accumulated cultural advantage, and goes on to explain that like other forms of capital, it tends to be disproportionately concentrated in the hands of the relative minority, and to favour its own reproduction and further accumulation by that minority (p. 147). Although capital can be sought and gained, it is not equally available to all who need or desire it. (ii) Identity and position are (re/de)constructed through language. Language is inextricable from the social. According to Johnson, 2004, in her description of the tenets of Vygotskys sociocultural theory, higher mental functions originate on the social, historical, or institutional plane and it is while participating in these social interactions that patterns of social activities are internalized. This transfer from the intrapersonal to interpersonal is dependent on a mediated sign system in which language is crucial (p. 108). Thus, it is our internalization of socioculturally constructed engagement that determines the meaning made of subsequent social interactions, where engagement and interaction are language dependent, and from which identities construct. ii) Identity and position are self- and other-imposed and negotiated. Identity can be self-ascribed and negotiated with the self in internal struggle. Fernstens (2008) study of Mandy, clearly depicts a site of internal conflict in which Mandy ascribes herself alternately a subject position as a bad writer and a good writer, and struggles to negotiate how to identify herself to herself. In contrast, through interaction, individuals take part in both the conscious and unconscious positioning of others. (v) Identity and position are ephemeral and continually shifting. Davies and Harr explain that, persons as speakers acquire beliefs about themselves which do not necessarily form a unified coherent whole. They shift from one to another way of thinking about themselves as the discourse shifts and as their positions within varying story lines are taken up. Each of these possible selves can be internally contradictory or contradictory with other possible selves located in different story lines (1990, p.58-59). (vi) Identity and position are multidimensional and complex. With both internal and external forces at work, the complexity of it is overwhelming. Vick and Martinez (2011) paint a vivid picture.Such subject positions constitute and operate on a variety of dimensions: biologically/ naturally given (male/female; adult/child); biologically/naturally acquired (parent/ child); and socially acquired or attributed statuses and roles (teacher/student). These can be seen as essentially descriptive/constitutive, in that they simultaneously map the world and position the subject within that map, although each dimension is constituted in relation to a number of complexly defined personal (bright/dull), technical (skilful/ unskillful) and moral (good/bad) modalities. Multiple positions are occupied simultaneously (woman/adult/parent/mother/wife/teacher), and each occupation of position is inescapably normatively inflected in multiple ways (not just effeminate man, immature adult, loving father, lazy teacher, but also effeminate/foolish/self-indulgent/bright/ knowledgeable man) (p. 181). (vii) Identity and position are susceptible to compliance, resistance, and agentive measures. Individuals have the ability to make choices in how they position themselves and others, as well as to reflect on how the enactment of their identity impacts the greater whole. In a study that focuses on identity as pedagogy and the teacher as image-text, Morgan writes of the role of self-awareness in acting as an agent and also how compliance can lead to opportunity for agency. With each performance and the responses it engenders from students, I become aware of other ways to re-script myself; that is, I gain insights into ways of subverting or transforming the rules (e.g. educational practices) (2004, p. 173). It is important to keep in mind that the degree to which resistance and agency are possible, and compliance coerced depends greatly on numerous socioculturally imbedded variables and power relationships.(viii) Identity and position play crucial roles in both language teaching and testing. Tests act as mechanisms both for the definition of subject positions and the recognition of subject positions. Tests create the identities they measure. (McNamara & Roever, 2006, p. 196). Furthermore, tests have a washback effect on teaching practices.The storyChanges to the oral proficiency interview (OPI) of the IELTS test in 2001 aimed to thwart bias introduced into the test through variability in examiner/rater performance. The redesign was in response to studies determining that inconsistency in interviewer behaviour and interpretation and application of scales could either advantage or disadvantage certain candidates (Brown & Taylor, 2006, p. 14). Consequently, measures were taken to standardize interlocutors language use and interpretation of candidate achievement. McNamara and Roever explain that language testers have attempted to comprehend how the co-construction of performance can be incorporated into test design and candidate measurement, and that those of the psychometric tradition have responded by striving to further disengage the interlocutor from the testing procedure in order to better control him/her as a variable and return sole focus to the candidate (2006, p. 48). Hence, currently, parts one and two of the OPI are scripted in their entirety, with only part three enabling use of the interlocutors own language in the form of rephrasing supplied prompts. Assessment criteria are very explicitly defined into scales of Fluency and coherence, Lexical resource, Grammatical range and accuracy and Pronunciation, and contain descriptors in 9 bands. This leaves little room for personal interpretation. Following some criticism of the Pronunciation rating scale, in 2008, the descriptors were modified and there was a shift from global aspects of pronunciation (such as intelligibility, listener effort, and accentedness) to increased assessment of specific phonological features (segmental and prosodic) (Yates et al., 2011, p. 4). Although this shift was generally accepted positively, it was found that there was a tendency for examiners to score Pronunciation as a 6 when it should be a 7 (p. 33). In Canada, being that 6 to 7.5 is what most organisations are looking for in proven English ability, this could have severe consequences for certain individuals. In addition, examiners are routinely monitored, and retrained every two years to ensure compliance with the set standard. It is through this limiting rubric that I am compelled to perform my identity as an IELTS speaking examiner and impose positions on IELTS candidates. Moreover, it is by this scripted language and rating system that I recursively reconstruct and perpetuate examiner identity in all its authoritative glory, and maintain the status quo that serves to disadvantage the hitherto disenfranchised non-native English-speaker (NNES).The script If we take a Bahktinian view that language lies between oneself and the other, as half someone elses, and that it becomes ours only when we populate it with our own intentions, and that prior to its appropriation by an individual, it exists in other peoples mouths, in other peoples contexts, serving other peoples intentions (Bakhtin, 1981, p. 294), how much ownership of words and intention is mine when Im reading from a script? And, given that identity and position are constructed through and by language, would it be fair to say that an identity is imposed upon me by means of the Interlocutor Framework? These questions are inextricably tied to how my doing being in the context of the IELTS exam positions both others and myself. As an examiner, I hold the future of numerous individuals in my hands, ergo I am in a position of substantial power. This is not a power I revel in, yet language is not at my disposal as a means of renegotiating positionality. Unquestionably, language is not the only means by which positioning takes place. Although joint construction of performance remains of paramount concern to test validity, the social nature of the testing interaction has also been investigated from more macrosociological points of view wherein examiner and candidate gender and native/non-native status, as well as examiners professional identity and linguistic experience has been explored as having potential influence on outcomes (McNamara & Roever, 2006, p. 52). I have experienced this first hand. With regard to gender, age, physical appearance, and native-speakerness, I have observed responses to these features of my identity manifest themselves in degrees of politeness, levels of comfort, and observable resentment of and resistance to my position as a gatekeeper. All of which can not only determine candidates language use, but also influence affective filters that may alter their performance under the test conditions. These aspects of my identity are not malleable, and cannot be controlled for. My professional and linguistic self also come into play, which I will speak to in the next section.The very nature of the IELTS exam and its uses serve to position candidates as others. On the surface, this could be viewed simply as not of the country into which they strive to immigrate or not a native English-speaker, but at a deeper level the other could be understood as not white, not a culturally/linguistically legitimate English user, not privileged, deemed deficient, in need of authoritative validation, and forced to assimilate to authoritative discourse. Candidates are lumped together into the them category of the us and them dichotomy. This is done with little consideration of the vast uniqueness of the individual, and at the same time without acknowledgement of the innate sameness that exists among and between individuals of diverse histories. By design, the test serves to further compound this already inherent quality of the Standardized English Language Test with its gatekeeping functions.It is not uncommon for questions in part one of the speaking test to include either the phrase in your country or where you live. As these scripted words leave my lips, further imposition is places on the identity of the candidate. I set myself in opposition to whom I am speaking. Language as a sociocultural meditational system or tool conveys meaning as such, and positionality determines who has the right to use and give meaning to language. It is my position as a white Canadian native English-speaker in Canada that lends credence to the aforementioned phrases and serves to accentuate the other status of the hearer. Furthermore, by employing these phrases, I may be invalidating already exercised degrees of devotion to Canadianness or presuming Canada is not in fact home. It is not uncommon for me to interview candidates who have grown up in Canada or lived here for many years. In these circumstances, such utterances are sometimes met with confusion and often a reply requesting clarification of where I mean. These are questions I am unable to answer as this would lead to deviation from the script. Candidates are left to make meaning of the utterance on their own without confirmation as to what type of response is expected of them. As a consequence, the candidate is positioned as incompetent in understanding a rather simple utterance, from a semantic standpoint, but which was laden with unclarified pragmatic implication. Despite candidates attempt to negotiate meaning, as would likely be considered pragmatically appropriate were the context an actual conversation and not the IELTS test cloaked as such, they are compelled to soldier on unenlightened. Such instances seem bound to affect subsequent language production. This is one repercussion of the nonconversational nature of the face-to-face based interview in which meaningful interaction between examiner and candidate is constrained in an attempt to negate introduction of interlocutor bias, and like examiners have commented, in which the wording of script language is sometimes considered awkward, over-lengthy, unclear, and colloquial or culture specific (Brown & Taylor, 2006, p. 15). In their discussion on research into the social implicit in Conversation Analysis, McNamara and Roever note that despite claims that the interview interaction is fundamentally conversational in nature, research clearly shows that it deviates from conversation significantly with unequal distribution of opportunities to initiate topic change, ask questions, and so on (2006, p. 47).While highlighting otherness, the phrase in your country also serves to blatantly ignore the uniqueness of individuals. Norton (2006) describes that in her earlier work, she chose not to examine identity as solely a cultural construct because she was unsure that theories of cultural identity could adequately consider the homogeneity within groups of individuals and found that such theories tended to essentialize and reify identity (p. 23). Such phrases as in your country are similarly inconsiderate of identity in its multiplicity. I routinely examine individuals from various backgrounds. Being especially cognizant of the implications of the aforementioned phrase, I was recently starkly aware of my employment of it with first a Chinese candidate, followed by a Russian candidate, and then an Indian candidate. To put the absurdity of such an utterance into perspective, Russia and China are the first and third largest countries on the planet in respect to land mass, and China and India the most populated at 1,339,724,852 (November, 2010) and 1,210,193,422 (February 2011) respectively (United Nations, 2012, p. 8). Not surprisingly, questions incorporating this phrase are often replied to with the lead in of Well, it depends on the person. What could be interpreted as disturbing, though not surprising, here is also the fact that never have I detected resistance to the exoticizing and stereotyping of individuals with such a sweeping generalization. Like Millers (2009) investigation into the (re)construction of hegemonic ideologies by orienting to speaking English as being ordinary, I see a parallel in which exoticizing is doing being ordinary, and until the repercussions of describing identities in terms of nationhood is made more lucid, the status quo will continue to go unchallenged.The assessmentThe goal of the IELTS band descriptors and rater training in using these scales is to achieve an accurate and standardized judgment of English-speaking proficiency, or rather proficiency in Standard English. In order to do this, I am expected to perform the role of one who is less proficient in the English language, as it is viewed in the era of postmodern globalization. Canagarajah explains that in the postmodern world, a proficient speaker of English needs to be aware of both native and new English norms, which are relative, variable, heterogeneous, and changing. Moreover, a proficient speaker needs to be able to shuttle between different norms, recognizing the systemic and legitimate status of different varieties of English in this diverse family of languages (2006, p. 234). By dint of my profession, as well as my propensity for travel, I have spent the past ten years of my life engaging in conversation with speakers of various Englishes. Through this experience, I have become attune to numerous non-standard English forms, am able to make meaning where other proficient English-speakers may not, and can easily make use of accommodation strategies that facilitate communication. In my view, this is a valuable skill and facet of my identity of which I take pride. Yet, when performing my IELTS position, I have to take on a role in which I do not have this competence. Jenkins states that testers need to take into account that non-native speaker English variants have legitimacy and hence the right not to be relegated to the status of error, and in the very least not penalize candidates for employing linguistic creativity with communicative success (2006, p. 45). I certainly agree that this would be ideal, but current test design prevents this. Hence, individuals who are highly communicative in English will continue to be positioned as deficient in English language skills by means of current Standardized English Testing.IELTS examiners are trained not to bring their postmodern English proficiency into the exam. Luckily, I am able to imagine Standard English competence because I have spent most of my life negotiating my identity by means of it, but not all examiners are NESs. In a survey of IELTS examiners, with regards to pronunciation, there were a number of comments related to the question of what native speaker speech was and comments that it is difficult for a non-native speaker of English to know what native speaker errors are (Brown & Taylor, 2006, p. 17). In a study on whether familiarity with candidates pronunciation affected rating, Carey et al. found that a significant portion of examiners awarded higher pronunciation marks when they possessed a greater level of interlanguage phonology familiarity. They also found that test centre location had a noticeable effect on rating (independent of the familiarity variable) with NNS raters awarding higher scores to candidates from their home country (2010, p. 201). The identity of the examiner cannot be completely removed from the OPI. While some examiners are familiar with numerous Englishes, others are familiar with none. Furthermore, it seems unpreventable for comprehension to affect rating along the other scales as well. Future directions and considerationsGiven the current status of the English language, the connection between language and identity, and how identity serves to marginalize some while privileging others, I think it is essential that Standardized English Tests change to reflect the current status of English as a diverse language, language as a means of communication, and communication as more relevant that accuracy. In order for this to be achieved, numerous changes need to be made to the IELTS OPI.By allowing examiners more freedom in which language they use to administer the IELTS test, individuals could be able to create a less threatening atmosphere in which the power dynamics were less palpable and the candidate language produced more representative of their true linguistic ability. Also, this would give examiners the ability to rephrase or avoid language that they deemed inappropriate and practice some discretion in imposing identities on IELTS candidates and themselves. An IELTS research report in 2006 on the impact of examiner deviation from the Interlocutor Frame concluded that deviations had negligible impact in practical terms and stated that the most relevant implication of their findings was that it may be possible to allow for some flexibility in the Interlocutor Frame (p. 22). I believe such an allowance would certainly improve the testing dynamics.Canagarajah expresses that if we are going to have a general proficiency test at all, that we should move towards a multitask, multirater, and multicandidate test in which having multiple raters would lead to better assessment of both holistic and discrete-item criteria. (2006, p. 238). I agree on the first two proposals, but feel that, as she points out, there are interpersonal dynamics between two paired candidates that have implications for their performance (p. 240). I would like to see a move towards multiple raters, in which one rater also played the role of a candidate, and at least one other playing the role of interlocutor. Ideally, these raters would come from a variety of linguistic backgrounds with the candidate rater not needing to be considered an Expert user of English. This could provide scores that are much more accurate and a better assessment of English use in the postmodern era.In order for better assessment to take place, it is also necessary to design test tasks and assessment scales that gauge pragmatic competency. The current focus in assessment is on native speaker-like accuracy and lexis, and does not reflect the purpose of language for communication or the relevant ways in which the English language is being used. According to Jenkins testing at present discourages the development of vital accommodation skills by, on the one hand, penalizing their use whenever the result is not native-like production regardless of any beneficial effect on communication and, on the other hand, rewarding their use whenever the result is native-like production, regardless of any negative effect on communication (2006, p. 48). By changing what is assesses, the real-life applicability of tests can be enhanced and communicative competence taken into account. Fulcher et al. (2011) argued that an analysis of how people use language in actual communicative contexts can form the basis for more dynamic and contextually sensitive approaches to rating that help to define the nature of interactional competence in context. Performance Decision Trees are more flexible and do not assume a linear, unidimensional, reified view of how second language learners communicate. They are also pragamatic, focusing as they do upon observable action and performance, while attempting to relate actual performance to communicative competence (p. 23). Their Performance Decision Tree notion could certainly be a step in the right direction both for assessment and task design. For rating of candidates on the basis of communicative competence for English use in settings around the world to be possible, new test tasks need to be introduced into testing. These will need to be context specific and to include real-time measures of learners interactional abilities to allow extrapolation to a target domain of social use (Roever, 2011, p. 470).ConclusionAs an IELTS speaking examiner, I aid the forces that struggle to overcome the heteroglossia of the English language and centralize verbal ideological thought, which develop in vital connection with the processes of sociopolitical and cultural centralization (Bakhtin, 1981, p. 271). By performing this identity, I produce the context that serves to further marginalize non-native English speakers. From this, arises an internal struggle. My position in the IELTS OPI lies in opposition to the identity I have ascribed myself as an aware individual who desires to challenge the status quo. Davies and Harr state that the possibility of choice in a situation in which there are contradictory requirements provides people with the possibility of acting agentically (190, p. 59). However, I feel that I have little possibility of choice in how I enact my examiner identity. Thus, in response to this struggle, I imagine myself working to make future changes in Standardized English Language Testing, and it is by continuing to perform my examiner role that I may be able to later gain a position of power in which greater agency is possible. By educating myself, I work towards this goal and further enhance my awareness. After all, we need critical self-awareness in order for us too recognize and then to decide whether to accept or resist our own subject positions in the system of social control in which tests play a part (McNamara & Roever, 2006, p. 198). In future, I hope to contribute to the positioning of all English speakers as legitimate in their English language use in the myriad context available for its use in the postmodern era. Like Jenkins, I proclaim that an English as an International Language approach to testing should be a pluricentric approachone which recognizes that while speakers of English around the world need sufficient in common to enable them to communicate, they are also entitled to use English varieties which project their identities and protect their language rights in international communication. (2006, p. 49). It is from an understanding of identity and my identity that I will bring about change.

ReferencesAtkinson, D. (2005). Language socialization and dys-socialization in a South India college. In E. Hinkel (Ed.). Language socialization in bilingual and multilingual societies (pp. 147-162). Hillsdale, NJ: L. Erlbaum Associations.Bakhtin, M. (1981). Discourse in the novel in The dialogic imagination: Four essays. Holquist, M. (Ed.). Austin: University of Texas Press.Brown, A. & Taylor, L. (2006). A worldwide survey of examiners views and experience of the revised IELTS Speaking Test. Research Notes, 26, 14-18.Canagarajah, S. (2006). Changing communicative needs, revised assessment objectives: testing English as an international language. Language Assessment Quarterly, 3(3), 229-242.Carey, M. D. (2010). Does a raters familiarity with a candidates pronunciation affect the rating in oral proficiency interviews? Language Testing, 28(2), 201-219.Citizenship and Immigration Canada. Application for permanent residence: Federal skilled worker class (IMM 7000). Web. 4 Aug. 2012. www.cic.gc.ca/english/information/applications/guides/EG7TOC.asp#eg72Davies, B. (2006). Subjectification: the relevance of Butlers analysis for education. British Journal of Sociology of Education, 27(4), 425-438.Davies, B. & Harr, R. (1990). Positioning: the discursive production of selves. Journal for the Theory of Social Behaviour. 20(1), 43-63.Department or Economics and Social Affairs: Statistics Division (2012). Population and vital statistics report: January 2012, Statistical Papers, Series A Vol. LXIV. New York: United Nations.

Hawkins, M. R. (2005). Becoming a student: Identity work and academic literacies in early schooling.TESOL Quarterly, 39(1), 59-82.Fernsten, L. A. (2008). Writer identity and ESL learners. Journal of Adolescent & Adult Literacy, 52(1), 44-52.Fulcher, G., Davidson, F. and Kemp, J. (2011). Effective rating scale development for speaking tests: Performance decision trees. Language Testing, 2011 28(1), 5-29.Johnson, M. (2004). Philosophy of Second Language Acquisition. New Haven, CT: Yale University Press.Jenkins, J. (2006). The spread of EIL: a testing time for testers. ELT Journal, 60(1), 42-50.McNamara, T. & Roever, C. (2006). Language testing: the social dimension. Malden, MA: Blackwell Publishing.Miller, E. R. (2009). Orienting to being ordinary: The (re)construction of hegemonic ideologies in interactions among adult immigrant learners of English.Critical Inquiry in Language Studies, 6(4), 315-344.Morgan, B. (2004). Teacher identity as pedagogy: towards a field-internal conceptualization in bilingual and second language education. Bilingual Education and Bilingualism, 7(2&3), 172-188.Norton, B. (2006). Identity as a sociocultural construct in second language research. In K. Cadman & K. ORegan (Eds.), Tales out of school: Identity and English language teaching.Norton, B. (2010). Identity, literacy, and English-language teaching.TESL Canada Journal, 28(1), 1-13.OSullivan, B. & Yates, L. (2006). 4. The impact of candidate language of examiner deviation from a set interlocutor frame. IELTS Research Notes, 6, 1-27.Roever, C. (2011). Testing of second language pragmatics. Language Testing. 28(4), 463-481.The official IELTS website. Web. 4 Aug. 2012. www.ielts.org.Vick, M. J. & Martinez, C. (2011). Teachers and teaching: subjectivity, performativity and the body. Educational Philosophy and Theory, 43(2), 178-192.Yates, L., Zielinski, B. & Pryor, E. (2011). The assessment of pronunciation and the new IELTS Pronunciation scale. IELTS Research Reports, 12, 1-46.