Assessment practice to complement use of text-matching software

Faculty of Education Humanities and LawFaculty Teaching and Learning Forum

November, 2015

David D Curtis

Outline

• Background• Two sets of questions about assessment

Questions to enhance learning through assessment design

Questions to minimise plagiarism in assessment

• Assessment methods• Discussion

Background

• Plagiarism is surprisingly commonMore than 75% of students self-report (anonymously) breaches of academic integrity

The incidence of these practices has grown

Students are increasingly likely to believe that using others’ work without acknowledgement, but in one’s own words, is not plagiarism

Text-matching software is a deterrent, but does not prevent plagiarism

Common forms of academic dishonesty includeUnacknowledged copying from other sources (publications, other students)

Commissioning ghost-writers of papers

It would be unwise to believe the problem does not exist at Flinders and to ignore it

• Responsibility to professions to ensure integrity of qualifications• Protection of the Flinders brand

(McCabe, et al., 2001; Miller, et al., 2011)

(Stapleton, 2012)

(Busch, 2014)

Enhancing learning through assessment design

Key questions

• Why assess?

• What to assess?

• How to assess?

• How to interpret?

• How to respond?(Rowntree, 1987, p.11)

Why assess?

• Deciding why assessment is to be carried out; what effects or outcomes it is expected to produce.

• Broadly, we identify at least two sets of reasonsWe assess in order to certify learning

We assess in order to enhance learning

• We expect several outcomesAssessment tasks should motivate learning

Assessment tasks should guide or direct learning

Assessment tasks should lead to high levels of knowledge and skill

• But, assessment motivates a range of unintended behavioursOne of these is plagiarism

How to assess?

• Selecting, from among all the means we have at our disposal for learning about people, those we regard as being most truthful and fair for various sorts of valued knowledge.

• We seek alignment between assessment purposes and methods• What methods are available to us?

[See below]

• By what criteria do we judge the suitability of assessment types?Validity Reliability

Fairness Feasibility

Objectivity Authenticity

Washback

How to respond?

• Finding appropriate ways of expressing our response to whatever has been assessed and of communicating it to the person concerned (or other people).

• Feedback is fundamental to effective learningHo: Engaging students in dialogue is likely to enhance personal commitment to the activities and to reduce AI breachesConsider assessment tasks that elicit discursive responses

Generate dialogue with and among students following their responses

Hattie, 2009

Hattie & Timperley, 2007; Nicol, 2010; Nicol & Macfarlane-Dick, 2006

Assessment to thwart plagiarism

Key question

• What methods of assessment will prevent plagiarism?And if it does occur, can you detect it?

Assessment methods

• ExaminationsClosed-book or open-book

Extended essays, short constructed response, multiple-choice (or similar)

• Take-home papersExtended essays, reports

Portfolios, wikis

Case studies

Literature reviews, annotated bibliographies

Artefacts

• PresentationsSeminars/tutorials

Viva voce

OSCEs (Objective Structured Clinical Examinations) (Brown & Race, 2013)

Discussion

Which of these methods:Enhances learning that isvalid, fair, authentic, feasible, …

We should pay particular attention to washback.How does assessment influence how students learn and how they respond?

Which of these methods:Prevents plagiarism

Assessment to curtail plagiarism

• Individualised assessment tasksIn statistics topics, I provide each student with a unique data set

Each data file is for a particular country, but all files have a common structure

Each has numerous variables reflecting diverse constructs

Students decide on research questions and choose the constructs to use, so each student’s assignment is unique

I did this, not primarily to counter plagiarism, rather to enhance authenticityCurtailing plagiarism was a consequence

• Incremental assessment tasksMajor assignment asks students to explain how they would implement an educational initiative

Prior to that, students submit a ‘readings summary’ and are advised to focus on the initiative they will investigate in the major assignment

Not designed to combat plagiarism, but to enhance engagement with literature

ReferencesBrown, S., & Race, P. (2013). Using effective assessment to promote learning. In L. Hunt & D. Chalmers (Eds.), University teaching in focus. A learning-centred approach (pp. 74-91). London: Routledge.Busch, P., & Bilgin, A. (2014). Student and staff understanding and reaction: academic integrity in an Australian university. Journal of Academic Ethics, 12, 227-243.Hattie, J. (2009). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. London: Routledge.Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. McCabe, D. L., Trevino, L. K., & Butterfield, K. D. (2001). Cheating in academic institutions: a decade of research. Ethics & Behavior, 11(3), 219-232.Nicol, D. J. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5), 501-517. Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. Rowntree, D. (1987). Assessing students: how shall we know them? (2nd ed.). New York: Harper and Row.Stapleton, P. (2012). Gauging the effectiveness of anti-plagiarism software: an empirical study of second language graduate writers. Journal of English for Academic Purposes, 11, 125-133.