1. introductions 2. testing & assessment – friend or foe? 3. speaking t & a 4. break

1. Introductions2. Testing & Assessment – Friend or Foe?3. Speaking T & A4. Break5. Listening T & A6. A bit about TOEFL iBT Speaking

IntroductionsPart I: Find your IPA partner

Match IPA character(s)on RED cards to words on GREEN cards RED circulate saying the sound of your IPA

character(s) GREEN say nothing – just nod when you

think your IPA character has found you When you think you have a match, step back

and wait.

Hull - L&S T&A - Da Nang - November 2008 2

For Example:

/z/ FLIMSY


Now that you’re matched: If you already know each other, share the

following:▪ one new thing the other person doesn’t know

about you; one thing you love about teaching; one thing you really don’t like about teaching

If you don’t know each other, share the following:▪ your name and place of work; one thing you

love about teaching; one thing you really don’t like about teaching


T & A – Friend or Foe?

QUESTION: Did any of you name Testing and Assessment (T&A) as something you really don’t like about teaching? Why?


Why do we T&A?

DISCUSSION QUESTION: What’s the difference between Testing and Assessment?


T&A: a method of measuring a person’s ability, knowledge, or performance in a given domain Test

prepared administrative procedures that occur at set times in a curriculum when learners prepare and aim for top performance knowing results are being evaluated, e.g. end-of-unit achievement test TOEFL and IELTS

Assessment an umbrella term that includes Tests – but also

encompasses a wider domain and range of conditions and objectives, e.g. student in-class response to teacher question writing journals (read but not graded by teacher)parts of this presentation are adapted from Brown, H.D.2004. Language Assessment: Principles and Classroom Practices


Key Terminology Review:How do these terms relate to T&A?Informal and Formal

Formative and Summative

Norm-referenced and Criterion Referenced

Test = prepared administrative procedures that occur at set times in a curriculum when learners prepare and aim for top performance knowing results are being evaluated

Assessment = an umbrella that includes Tests – but with wider range of conditions and objectives


Informal assessment/not a test: e.g. impromptu

feedback in class; marginal comments on writing drafts

Formal assessment/not a test: e.g. review student’s

portfolio, possibly graded (not a test because it includes open domain and takes place over extended period of time)

test: e.g. graded final draft of an essay, scheduled vocabulary quiz


Formative assessment aims to evaluate students in the process of

“forming” their competencies and skills with the goal of helping them to continue that growth process.

Summative assessment aims to measure, or summarize, what a student

has grasped, and typically occurs at the end of a course or unit of instruction (typically a test)


Norm-referenced: each score interpreted in relation to a mean

(average score), median (middle score), standard deviation (extent of variance in scores), and/or percentile rank, i.e. each student in relation to other students Especially standardized tests, e.g. TOEFL

Criterion-referenced: designed to measure performance in relation to

specific course or lesson objectives and to give test-takers feedback Especially classroom based tests


5 Key Principles of T&APracticality

not expensive to make or administer; time efficient; reasonably easy to administer and score

Reliability consistent and dependable

Validity represents objectives of the domain being tested

Authenticity natural, contextualized, meaningful, thematic, real-

worldWashback

effect of test on teaching & learning


Speaking Assessment: Question 1

◦ Most of the time an interlocutor (another speaker) is involved – that is, aural intake is involved

◦ Therefore, a speaker’s listening skill affects his/her productive speaking performance

◦ Therefore, how can we be sure a test is a valid test of speaking – and not really more about listening?


Speaking Assessment: Questions 2 & 3

◦ Authentic speech is mostly open-ended and creative – so how do we design test tasks for speaking that guide a test-taker to produce specific target forms?

◦ Alternatively, if we allow for S freedom of choice, how do we evaluate the product?


Taxonomy of Speaking Skills


4. Interactive – longer, more complex, multiple exchanges, with multiple participants

a. Transactional (exchanging information)

b. Interpersonal (maintaining social relationships)

5. Extensive – monologue: speeches, oral presentations, story-telling, etc.


1. Imitative SpeakingExample:S hears: Repeat after me.

beat (S repeats) bit (S repeats)OR

I bought a boat yesterday. (S repeats)

The glow of the candle is growing. (S repeats)


ISSUE: The Rubric, Part IDISCUSS:

What are the features of an effective and fair rubric (scoring system) for Speaking Assessment? consider issues of practicality, reliability, and validity

DESIGN:Design a 2-3-point rubric for an Imitative task

such as: (T says) Repeat after me:

The glow of the candle is growing. (S repeats)


2. Intensive SpeakingDirected Response TasksRead-Aloud TasksSentence/Dialogue Completion TasksOral QuestionnairesPicture-Cued TasksTranslation


Using Picture Cues


What’s being tested?


Could you use visuals like these?


Working alone: Choose one of the Bubble Dialogues given you and design a Speaking Test Task (draft instructions; consider what language you expect to elicit)

Then “test” it on a “student”

Did you get the language you expected?

Dialogue Completion

S reads dialogue in which one speaker’s lines have been omitted

S is given time to read through to get gist and think of appropriate lines to fill in

Then T reads written lines and S responds orally.


Example: (in a department store)Salesperson: May I help you?Customer: _______________________Salesperson: Okay, what size do you wear?Customer: ______________________Salesperson: Hmm. How about this green sweater here?Customer: _______________________Salesperson: Oh, well, if you don’t like green, what color

would you like?Customer: _______________________Salesperson: How about this one?(etc.)


Possible answers: (in a department store)Salesperson: May I help you?Customer: Yes, thank you. I’m looking for a sweater.Salesperson: Okay, what size do you wear?Customer: Size 10.Salesperson: Hmm. How about this green sweater here?Customer: I don’t like green.Salesperson: Oh, well, if you don’t like green, what color

would you like?Customer: Red.Salesperson: How about this one?(etc.)


TASK:In pairs, think of a language function or discourse

situation you would like your Ss to be tested on

Design a short Dialogue Completion Task (5 to 7 exchanges)

Practice it together to check validity (is it producing the target language performance you want?)

Write out 2 copies of the “test”

Administer it to two other peopleHull - L&S T&A - Da Nang - November 2008 29

(location/situation)Person 1: xxxx xxxx xxxx xxxx xxxxPerson 2: _________________________.Person 1:: xxxx xxxx xxxx xxxx xxxxPerson 2: ________________________.Person 1: xxxx xxxx xxxx xxxx xxxxPerson 2: _________________________.Person 1: xxxx xxxx xxxx xxxx xxxxPerson 2: _________________________.Person 1: xxxx xxxx xxxx xxxx xxxx(etc.)


3. ResponsiveQuestion and Answer

Giving Instructions and Directions

TASK: Brainstorm ideas for “giving instructions” (i.e. to do what?)


Paraphrasing

TASK: Practice being a student! Listen to the following passage. It will be read twice. When instructed to do so, “under your breath,” paraphrase what you heard in 2-3 sentences.

DISCUSSION: How would you judge this task in terms of reliability and validity?


4. InteractiveInterview

Discussions / Conversations

Role-plays


The Rubric – Part II: Interactive and Extensive SpeakingLook at the Rubric Handout.

DISCUSS: Discuss the Rubric’s strengths and

weaknesses.Do you think it meets your standards for

reliability and validity?Would you use it in your classes?


TASK: Role-Play for pairsDiscuss and then decide on a language point or

points (grammar, function, vocabulary, etc.) you want to create a T&A task for

Create a Role-Play Task that would elicit the linguistic behavior you want to test (4-8 exchanges)

Practice a performance of your T&A Task (quietly!)

Perform your task role-play (do not name the language point being tested!)

Group guesses what language point is being testedHull - L&S T&A - Da Nang - November 2008 35

5. ExtensiveOral presentationsPicture-cued story tellingRetelling a story or news articleTranslation


Picture-cuedStory-telling


Retelling a news storyDISCUSS issues regarding:

reliability and validityusing the rubricsource materialnote-taking

Breaking News English – Men more attracted to women in red


Taxonomy of Listening Skills1. Intensive – for perception of components

2. Responsive – listening to short stretches to make equally short responses

3. Selective – short monologues, scanning for info, testing for ability to comprehend designated info

4. Extensive – listening to develop top-down, global understanding


1. IntensivePhonological / Morphological Elements

Phonemic pair, vowels S hears: Is he living?

S reads: a. Is he leaving?b. Is he living?

Stress pattern in can’t S hears: My girlfriend can’t go to the party.

S reads: a. My girlfriend can’t go to the party.

b. My girlfriend can go to the party.


Paraphrase Recognition

Dialogue paraphrase

S hears: Man: Hi, Maria, my name’s George.

Woman: Nice to meet you, George. Are you American?

Man: No, I’m Canadian.

S reads:a. George lives in the United States.b. George is American.c. George comes form Canada.d. Maria is Canadian.


2. ResponsiveAppropriate response to a question

S hears: How much time did you take to do your homework?S reads: a. in about an hour c. about $10

b. about an hour d. Yes, I did.Open-ended response to a question

S hears: How much time did you take to do your homework?

S writes or speaks: __________________________


3. SelectiveListening Cloze

Autumn Songs on VOA Special English what language point(s) could you highlight for a selected-

cloze?

choose 4-6 words to leave blank Autumn Songs on VOA


People have written and recorded hundreds of songs about autumn. Many of these songs express sadness that summer is over. The days are shorter. It is getting darker earlier each day. The weather is cooler. The skies are gray. Birds fly south because they know winter is coming. The leaves turn colors of red and gold and then die, falling to the ground. Some songs about autumn also express the sadness of lost love.

What do you think of this?VOICE TWO:And I'm Barbara Klein. It is autumn in the _______________ part of

the world. So it is time to play some of our favorite songs about this season.

VOICE ONE:People have written and recorded hundreds of songs about

autumn. Many of these songs express sadness that summer is over. The days are ______________. It is getting darker earlier each day. The weather is _____________. The skies are gray. Birds fly south because they know winter is coming. The leaves turn colors of red and _____________ and then die, falling to the ground. Some songs about autumn also express the sadness of ___________ love.

Mary Dawson, in her Internet Writing Journal, writes that this season influenced songwriters to write some of the ________________ songs of all time. Here are some of our favorite songs about autumn.


Information transferThe Bird-Feeder

what language point(s) do you think is (are) going to being tested?


4. Extensive ListeningDictation – tried-and-still-true

Stimulus-Response TasksExample: Checking into a Hotel

QUESTION: What kind of students need this language?

(ESLPodcast – checking into a hotel)Hull - L&S T&A - Da Nang - November 2008 48

Multiple-choice questions Open questions

What kind of room did the man reserve? a. non-smoking standard b. smoking suite c. non-smoking suite d. smoking standard

What kind of room did the man reserve?

______________________________

Towards Greater Authenticitynote-taking (important for iBT TOEFL)

DISCUSS: how can we “grade” note-taking?

editing (noting difference between written and spoken versions of text)

TASK: what words would you change and why?

interpretive tasks (responding to songs, poetry, etc.)DISCUSS: what are the issues regarding reliability and

validity?

retelling (orally or in writing)DISCUSS: what is authentic?


http://eolf.univ-fcomte.fr/uploads/ressources/listening/real_english/41a/01.htm

Interviewer: What're you doing right now? Messenger: Working. Interviewer: What's your job. Messenger: Messenger. Interviewer: You're a messenger. Messenger: I deliver mail. Interviewer: Where do you deliver mail to? Messenger: Park Avenue South. Oh, downtown Manhattan. Uh ... Uptown. All over. All over the five boroughs. Interviewer: How long have you been a messenger? Messenger: About a year and a half. Interviewer: Do you like your job?

Messenger: Yeah. It pays the bills. It pays the bills. Interviewer: What do you hope to do ... uh? Messenger: I want to get on the radio. I want to be a radio broadcaster if somebody'd give me a break. That's what I'd like to do. Interviewer: Do you sing or play a musical instrument? Messenger: No. Interviewer: No. OK. Well, thanks a lot. Messenger: Bye. You take care. Peace!


Retelling from authentic sources


Jocelyn Chng,Sin Hwa Dee, Singapore

http://edition.cnn.com/video/#/video/business/2008/05/25/stevens.boardroom.chng.cnn

Online listening resources:http://www.voanews.com/specialenglishhttp://www.eslpod.com/http://www.breakingnewsenglish.com/http://eolf.univ-fcomte.fr/index.php?page=rea

l-englishhttp://edition.cnn.com/CNNI/Programs/board

room/your suggestions?


INDEPENDENT TASKS

Speaking Task 1: Familiar Topic You have 15 seconds to prepare and 45 seconds to

speak:

Prompt: You have the opportunity to study something you’ve never studied before. Explain what subject you would choose. Include reasons and examples in your response.

Speaking Task 2: Stating Your Opinion You have 15 seconds to prepare and 45 seconds to

speak:

Prompt: Do you agree or disagree with the following statement? Having a lot of money is very important. Use reasons and examples to support your response


INTEGRATED TASKS:

Speaking Task 3: Campus ConversationRead + Listen: 30 prep / 60 speak

Speaking Task 4: Academic Lecture Read + Listen: 30 prep / 60 speak

Speaking Task 5: Campus ConversationListen: 20 prep / 60 speak

Speaking Task 6: LectureListen: 20 prep / 60 speak


Speaking Task 4: Academic Lecture Read + Listen: 30 prep / 60 speakNew Scientific Ideas

In the world of science, it can be a difficult proposition to introduce new ideas to the scientific community. After a theory or method has become established, the scientific world tends to view it as the norm. Any attempt to propose a different theory or method is viewed with suspicion.There are many accepted scientists today who in the past were out of the mainstream of science at some point because they held views that did not follow accepted theory or methodology. There are also many scientists today who are out of the mainstream now but who may become accepted in the future.


Now listen and take notes!


And now prepare your response:

Prompt: The professor describes some accomplishments of Darwin and Einstein. Explain how this information about Darwin and Einstein is related to the information about establishing new ideas.

30 seconds to prepare60 seconds to speak


Speaking and ListeningTesting and Assessment

November 7, 2008

1. introductions 2. testing & assessment – friend or foe? 3. speaking t & a 4. break

Documents

objectiveshull ls ta

teachinghull ls ta

zflimsyhull ls ta

assessment ta

toefl ibt speakinghull

testing assessment friend

new thing

language assessment