integrated reading and writing tests: assessing the academic writing construct · integrated...
TRANSCRIPT
![Page 1: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/1.jpg)
Integrated reading and writing tests:
assessing the academic writing construct
Christopher Smith
University of Sheffield ELTC
![Page 2: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/2.jpg)
Overview
• Academic writing • Reading-to-write tests • The new reading-to-write test • The ratings scale • Discussion
![Page 3: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/3.jpg)
The Domain: Academic Writing
![Page 4: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/4.jpg)
Academic writing
Must be ‘text-responsible’
(Leki and Carson, 1997)
![Page 5: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/5.jpg)
Academic writing
• Relies on sources, not anecdotes.
(Moore and Morton 2005)
![Page 6: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/6.jpg)
Academic writing
• Relies on sources, not anecdotes. • Citations and references
(Moore and Morton 2005)
![Page 7: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/7.jpg)
Academic writing
• Relies on sources, not anecdotes. • Citations and references • Summarisation and description
(Moore and Morton 2005)
![Page 8: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/8.jpg)
Academic writing
• Informational • Impersonal / abstract • Non-narrative • Not generally persuasive • Not situation dependent
(Nesi and Gardner 2012)
![Page 9: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/9.jpg)
Academic writing
• Knowledge demonstration • Logical reasoning • Knowledge translation • Developing arguments • Critical evaluation
(Nesi and Gardner 2012)
![Page 10: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/10.jpg)
Testing academic writing
Why is this important?
![Page 11: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/11.jpg)
Testing academic writing
Why is this important? • Validity of score interpretation and use
![Page 12: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/12.jpg)
Testing academic writing
Academic writing domain
Test content
![Page 13: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/13.jpg)
Testing academic writing
Why is this important? • Validity of score interpretation and use • Positive washback
![Page 14: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/14.jpg)
Testing academic writing
Why have an exam?
![Page 15: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/15.jpg)
Testing academic writing
Why have an exam? Why design a new test?
![Page 16: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/16.jpg)
Current problems with Reading-to-write tests
![Page 17: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/17.jpg)
Integrated Tasks Overview
1. Mining texts 2. Synthesising ideas 3. Language transformation 4. Organisational structure 5. Connecting ideas
(Knoch and Sitajalabhorn, 2013)
![Page 18: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/18.jpg)
Integrated Tasks Overview
1. Mining texts 2. Synthesising ideas 3. Language transformation 4. Organisational structure 5. Connecting ideas 6. Criticality – commenting on texts
![Page 19: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/19.jpg)
Reading-to-write test task 1
1. Summary of a text 2. Opinion essay on the topic of the text
(Asención Delaney, 2008; Wu, 2013)
![Page 20: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/20.jpg)
Reading-to-write test task 1
• Summary task – No critical evaluation
![Page 21: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/21.jpg)
Reading-to-write test task 1
• Summary task – No critical evaluation
• Opinion essay – Not text-responsible
![Page 22: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/22.jpg)
Reading-to-write test task 1
• Summary task – No critical evaluation
• Opinion essay – Not text-responsible
• Both tasks on the same topic – Risk of topic bias
![Page 23: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/23.jpg)
Reading-to-write test task 2
• Read 2-3 texts, • Write essay on position • Elicits discourse synthesis (Gebril & Plakans, 2009, 2013; Plakans &
Gebril, 2012; Plakans, 2008, 2009a, 2009b; Wolfersberger, 2013)
![Page 24: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/24.jpg)
Reading-to-write test task 2
(Gebril & Plakans, 2009)
Some people believe that global warming is damaging our
planet. Others believe that global warming is not a serious
problem.
Which point of view do you agree with? Why?
Give reasons and support your writing with examples.
![Page 25: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/25.jpg)
Reading-to-write test task 2
• Test-wiseness strategy
![Page 26: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/26.jpg)
Reading-to-write test task 2
• Test-wiseness strategy • Undermines content validity
![Page 27: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/27.jpg)
Reading-to-write test task 2
• Logical reasoning • Evidence → position
![Page 28: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/28.jpg)
Reading-to-write test task 2
• Logical reasoning • Evidence → position • Position → find evidence in support
![Page 29: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/29.jpg)
Reading-to-write test task 2
• Logical reasoning • Evidence → position • Position → find evidence in support • Confirmation bias
![Page 30: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/30.jpg)
Reading-to-write test task 2
Fundamental problem: Asks for personal response to topic
![Page 31: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/31.jpg)
Reading-to-write test task 3
• TOEFL iBT • Integrated task
– Reading then listening – Then write about differences
• Independent task – Opinion essay on different topic
![Page 32: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/32.jpg)
Reading-to-write test task 3
ETS Website; http://www.ets.org/Media/ Tests/TOEFL/pdf/SampleQuestions.pdf;
date accessed: 31/01/2014
Summarize the main points made in the lecture, being sure to explain how they oppose specific points made in the reading passage.
![Page 33: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/33.jpg)
Reading-to-write test task 3
• Multiple tasks better than one (Weir, 2005; Weigle, 2002)
![Page 34: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/34.jpg)
Reading-to-write test task 3
• Multiple tasks better than one (Weir, 2005; Weigle, 2002)
• Integrated task: no personal response
![Page 35: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/35.jpg)
Reading-to-write test task 3
• Multiple tasks better than one (Weir, 2005; Weigle, 2002)
• Integrated task: no personal response • Independent task: different topic
![Page 36: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/36.jpg)
Reading-to-write test task 3
• Multiple tasks better than one (Weir, 2005; Weigle, 2002)
• Integrated task: no personal response • Independent task: different topic
• Lack of critical evaluation
![Page 37: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/37.jpg)
The New Reading-to-Write Test
![Page 38: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/38.jpg)
The New Reading-to-Write Test
• Two texts – Simplified language – Opposing viewpoints
![Page 39: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/39.jpg)
The New Reading-to-Write Test
What are the differences between the arguments in the two texts? Which do you think is stronger, and why?
![Page 40: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/40.jpg)
The New Reading-to-Write Test
1. Summary – Yes 2. Paraphrase – Yes
![Page 41: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/41.jpg)
The New Reading-to-Write Test
1. Summary – Yes 2. Paraphrase – Yes 3. Referencing – Sometimes 4. Critical thinking – Usually
![Page 42: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/42.jpg)
The New Reading-to-Write Test
1. Mining texts – Yes 2. Synthesising ideas – Yes 3. Language transformation – Yes 4. Organisational structure – Yes
![Page 43: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/43.jpg)
The New Reading-to-Write Test
1. Mining texts – Yes 2. Synthesising ideas – Yes 3. Language transformation – Yes 4. Organisational structure – Yes 5. Connecting ideas – Not so much 6. Criticality – Not always
![Page 44: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/44.jpg)
The New Reading-to-Write Test
• (Introduction) • Summary 1 • Summary 2 • Judgement
![Page 45: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/45.jpg)
The New Reading-to-Write Test
• What are the differences between the arguments in the two texts?
![Page 46: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/46.jpg)
The New Reading-to-Write Test
• What are the differences between the arguments in the two texts?
• Summarise the arguments in the two texts.
![Page 47: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/47.jpg)
The New Reading-to-Write Test
• Which do you think is stronger, and why?
![Page 48: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/48.jpg)
The New Reading-to-Write Test
• Which do you think is stronger, and why?
• Using evidence from the texts, show which argument is better and why.
![Page 49: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/49.jpg)
The marking criteria
![Page 50: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/50.jpg)
The marking criteria
1. Used exact words from CEFR “Can do” changed to “has done”
![Page 51: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/51.jpg)
The marking criteria
1. Used exact words from CEFR “Can do” changed to “has done”
2. 4 criteria; 8 levels; Criteria not analytic Too many levels
![Page 52: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/52.jpg)
The marking criteria
1. Used exact words from CEFR “Can do” changed to “has done”
2. 4 criteria; 8 levels; Criteria not analytic Too many levels
3. 8 criteria; 4 levels Yet to be trialled
![Page 53: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/53.jpg)
Discussion
• The new test • The marking criteria
![Page 54: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/54.jpg)
Thank you
![Page 55: Integrated reading and writing tests: assessing the academic writing construct · Integrated reading and writing tests: assessing the academic writing construct Christopher Smith](https://reader033.vdocument.in/reader033/viewer/2022052300/5ad0f7467f8b9a71028e7f21/html5/thumbnails/55.jpg)
References
Asención Delaney, Y. (2008). Investigating the reading-to-write construct. Journal of English for Academic Purposes, 7(3), 140–150. doi:10.1016/j.jeap.2008.04.001
Gebril, A., & Plakans, L. (2009). Investigating source use, discourse features, and process in integrated writing tests. SPAAN FELLOW, 1001, 47, 7.
Knoch, U., & Sitajalabhorn, W. (2013). A closer look at integrated writing tasks: Towards a more focussed definition for assessment purposes. Assessing Writing, 18(4), 300–308. doi:10.1016/j.asw.2013.09.003
Leki, I., & Carson, J. (1997). “Completely Different Worlds”: EAP and the Writing Experiences of ESL Students in University Courses. TESOL Quarterly, 31(1), 39. doi:10.2307/3587974
Moore, T., & Morton, J. (2005). Dimensions of difference: a comparison of university writing and IELTS writing. Journal of English for Academic Purposes, 4(1), 43–66. doi:10.1016/j.jeap.2004.02.001
Nesi, H., & Gardner, S. (2012). Genres across the Disciplines: Student writing in higher education. Cambridge: Cambridge University Press.
Plakans, L. (2008). Comparing composing processes in writing-only and reading-to-write test tasks. Assessing Writing, 13(2), 111–129. doi:10.1016/j.asw.2008.07.001
Plakans, L. (2009). Discourse synthesis in integrated second language writing assessment. Language Testing, 26(4), 561–587. doi:10.1177/0265532209340192
Weigle, S. C. (2002). Assessing Writing. Cambridge: Cambridge University Press. Weir, C. J. (2005). Language Testing and Validation: An Evidence-Based Approach. Basingstoke: Palgrave Macmillan.
Wolfersberger, M. (2013). Refining the Construct of Classroom-Based Writing-From-Readings Assessment: The Role of Task Representation. Language Assessment Quarterly, 10(1), 49–72. doi:10.1080/15434303.2012.750661
Wu, R.-J. R. (2013). Native and non-native students’ interaction with a text-based prompt. Assessing Writing, 18(3), 202–217. doi:10.1016/j.asw.2013.01.001