LYNDA MILNEHTTP: / /WWW.CTL .MNSCU.EDU

MINNESOTA COLLEGES AND UNIVERSITIES ENGLISH AND WRITING CONFERENCE

MINNEAPOLISAPRIL 4 , 2009

How We Talk About and DoAssessment

Changes Everything

Ah, the Good Old Days

Before the curse of [assessment] fell upon mankind we lived a happy, innocent life, full of merriment and go and informed by fairly good judgment.

Hilaire Belloc

What Graff said.

“I've become a believer in the potential of learning outcomes assessment, which challenges…us to articulate what we expect our students to learn—all of them, not just the high-achieving few—and then holds us accountable for helping them learn it.”

What he said.

Colleges, and faculty themselves, are more interested in attracting and teaching “The best students” vs. “Students we have actually taught something.”

What he said.

“…In the hundreds of faculty meetings I must have attended in my 40-plus years of teaching, I have never heard anyone ask how our department or college was doing at educating all its students.

What he said.

“Outcomes assessment changes the question to what students can do as a result of seeing [being taught by] us.”

What he said.

“Once we start asking whether our students are learning what we want them to learn, we realize pretty quickly that making this happen is necessarily a team effort, requiring us to think about our teaching not in isolation but in relation to that of our colleagues.

What he said.

“The problem is not that we don't value good teaching, as our critics still often charge, but that we often share our culture's romanticized picture of teaching as a virtuoso performance by soloists…  the Great-Teacher Fetish, the counterpart of the Best-Student Fetish.”

What he said.

“For all its obvious value, excellent teaching in itself doesn't guarantee good education…. Outcomes assessment holds us to that obligation by making us operate not as classroom divas and prima donnas but as team players who collaborate with our colleagues to produce a genuine program.” 

What he said.

“To see outcomes assessment as merely a conservative dodge designed to distract everyone from structural inequality ignores the ways our own pedagogical and curricular practices contribute to the achievement gap.”

What he said.

“The original motivations of assessment lie in legitimate progressive efforts to reform higher education from within, by judging colleges according to what their students learn rather than by their elite pedigrees.”

What he said.

“Rather than reject assessment and circle the wagons, however, we should actively involve ourselves in the process, not only to shape and direct it as much as possible but to avoid ceding it by default to those who would misuse it.”

(Quoting David Bartholomae): ‘We make a huge mistake if we don't try to articulate more publicly what it is we value in intellectual work. We do this routinely for our students—so it should not be difficult to find the language we need to speak to parents and legislators.’

What they said!

First, Prof. Graff's article makes no mention of the faculty labor involved in assessment--and it is considerable. 

What they said!

The reservation I have about outcomes assessment is that it will inevitably "assess" those things that are most easily assessed . . . which often means things that are trivial. For a writing course, it is fairly easy to assess grammar errors or spelling or punctuation or T-units or sentence variety. It is difficult (and expensive) to assess the quality of thinking, the effectiveness of language, the sophistication of argument. I fear an increased emphasis on assessment will mean an increased emphasis on the trivial over the profound.

What they said!

Finally, anybody is welcome to assess how I teach by asking anybody anything they want--students, ex-students, colleagues, superiors, professionals out in the field where my students have gone. I just do not wish to have to check little boxes and line up long vertical columns of approved verbs relative to "strategies," "goals," "objectives," and other educratic fetishes.

What they said!

Faculty are concerned that outcomes assessment will end up like student course evaluations in many schools. Feedback comes too late to help the faculty member and the administration uses what is frequently a popularity contest to punish faculty.

What they said!

Many humanists resist assessment because they assume (or are told) that assessment begins with the description of learning goals, and the more precise their goals the better. In contrast, their idea of good teaching may be more like that of a colleague of mine [who] didn't mind if his students became taxi drivers, if they were better, more humane, engaged human beings. How does one state that goal as a learning outcome? How measure it with a test? Confronted with that definition of assessment, many humanists resist.

What they said!

“Outcomes Assessment is basically No Child Left Behind as applied to higher education. Its implications are profoundly anti-intellectual, conformist, and conservative”

What they also said…

Most of us are part of a noble enterprise, a university or a college. What is the purpose of that enterprise? What is its product? I would argue that it is simply "learning." That is what students do in our classrooms and that is what our faculties do in their scholarship and research. If one accepts that idea, then is it not reasonable to expect that an education institution should attempt to ascertain whether its product actually exists, and, if it does, to discover something about its quality and quantity? I think the answer is obvious.

And one said this.

In our history department at a large Western land-grant university, we were dragged kicking and screaming into doing outcomes assessment. We started as simply as possible, assessing just two learning outcomes using two essay-exam responses as our instruments. What we found surprised us. No, it didn't surprise us that our students performed rather badly at some of our outcomes.

It did surprise us that the entire assessment process (especially the measuring) led us to the richest, most intellectually engaging, and most useful faculty discussions we've ever had about teaching and student learning. I actually look forward to our assessment measurement day (it takes six of us faculty about 5 hours) each semester and the talk about what we might do to improve. Each of us has changed the way she/he teaches, and we will probably change our major in response to what we've found in assessment.

What they also said!

[Our department decided that it wanted] “students to be able to fairly summarize someone else’s argument and critically evaluate it without making ad hominem attacks . . . [Learning Outcomes Asssessment] doesn’t force a teacher to teach one kind of content vs. another, but it does identify certain skills that are important.” For example, it forces “a teacher to actually help the students correctly read an argument and learn how to compare and contrast it with other arguments.”

Assessment of…?

Student skills and knowledge at entrance to an institution—e.g., placement exams

Student learning in a course—for example, using tests or other evaluative procedures

Student knowledge, abilities at the end of a courseStudent learning at the completion of an academic

programMany or all students’ learning in an academic

programStudent learning outcomes across academic areas at

the conclusion of a major milestone of education—e.g., graduation from college.

Student gains in knowledge or skills, comparing entry and exit points.

Ok, Some Definitions

Assessment involves collecting information about the quality or quantity of a change in a student or group.

Evaluation may be defined as judging the merit, value, or desirability of a measured performance.

You can assess without evaluation, but you cannot evaluate without assessment.”

Roger T. Johnson, 2003

ENGLISH TEACHERS CAN CHANGE ASSESSMENT TALK & PRACTICE

Know enough to learn from us?

The University assumes that you are proficient in English and in writing about academic topics. Fulfillment of the University of California Entry-Level Writing requirement (formerly known as the Subject A requirement) is a prerequisite to enrollment in all reading and composition courses. If you have not passed the Analytical Writing Placement Examination (AWPE—formerly known as the Subject A Examination) or otherwise fulfilled the requirement by the time you enter the University, you should enroll in College Writing R1A during your first semester. College Writing R1A is a 6-unit course that satisfies the Entry-Level Writing requirement and the first half of the Reading and Composition requirement. (University of California at Berkeley, English Department entrance

requirements)

Are you sure?

In addition to a passing score on the AWPE, the Office of Undergraduate Admissions accepts the following means of fulfilling the Entry-Level Writing requirement before you enter the University:

A minimum score of 680 on the SAT Reasoning Test, Writing Section A minimum score of 680 on the SAT II: Subject Test in Writing, taken since May 1998 A minimum score of 660 on the SAT II: Subject Test in Writing, taken May 1995 through April

1998 A minimum score of 600 on the SAT II: Subject Test in Writing, taken May 1994 through April

1995 A minimum score of 30 on the ACT combined English/Writing Test A minimum score of 600 on either form of the College Board Achievement Test in English

Composition—“with essay” or “all multiple choice,” taken before May 1994 SAT Advanced Placement: A minimum score of 3 on the Advanced Placement Test in English

Composition and Literature or in English Language and Composition A minimum score of 5 on the International Baccalaureate Higher Level Examination in English

(Language A only) A minimum score of 6 on the International Baccalaureate Standard Level English and Exam A score of “Pass for Credit” on the California State University and Colleges English Equivalency

Examination (discontinued 1993) A minimum grade of C in a transferable college-level English composition course completed at an

accredited college or university and accepted by the Office of Undergraduate Admissions at Berkeley.

A minimum grade of C in a transferable college-level English composition course completed at an accredited college or university and accepted by the Office of Undergraduate Admissions at Berkeley.

Sharon Hamilton, IUPUI

Principles of Undergraduate Learning

Chancellor's Professor Emerita of EnglishIndiana University, IndianapolisOffice: (317) 278-1846E-mail: shamilto@iupui.edu

Barbara Walvoord at Notre Dame

Effective Grading/A Tool for Learning and Assessment

Professor Emerita of English334 Decio Faculty Hall574-631-0101Walvoord@nd.edu

Grade the Title: up to 5 points each

A. A Comparison of Prell and Suave Shampoo

B. The Battle of the Suds: Budweiser and Weiderman Beer

C. Would You Eat Machine-Made or Homemade Cookies?

D. A Comparison of Arizona and Snapple Ice Tea for pH, Residue, Light Absorbency, and Taste

E. Research to Determine the Better Paper Towel

F. A Comparison of Amway Laundry Detergent and Tide Laundry Detergent for characteristics of Stain Removal, Fading, Freshness, and Cloth Strength

Anderson’s Grades

A. A Comparison of Prell and Suave Shampoo 3

B. The Battle of the Suds: Budweiser and Weiderman Beer 2

C. Would You Eat Machine-Made or Homemade Cookies? 1

D. A Comparison of Arizona and Snapple Ice Tea for pH, Residue, Light Absorbency, and Taste 5

E. Research to Determine the Better Paper Towel 2

F. A Comparison of Amway Laundry Detergent and Tide Laundry Detergent for characteristics of Stain Removal, Fading, Freshness, and Cloth Strength 4

Anderson’s Criteria

1 Is patterned after another discipline or is missing.

2 Identifies function or brand name, but not both; lacks design information or is misleading.

3 Identifies function and brand name but does not allow reader to anticipate design

4 Is appropriate in tone and structure to science journal; most descriptors present; identifies function of experimentation, suggest design, but lacks brand names.

5 Is appropriate in tone and structure to a science journal; contains necessary descriptors, brand names, and allows reader to anticipate design.

Meaningful grades vs.

Description of a grade: An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material.

P. Dressel (1983). Grades: One more tilt at the windmill. In A.W. Chickering (Ed.), Bulletin. Memphis: Memphis State U. Center for the Study of Higher Education, Dec. 1983, p. 12

Joseph Eng at Eastern Washington University

Embracing the Exit: Assessment, Trust, and the Teaching of Writing

Professor of English and RhetoricDirector of the University Writing ProgramCalifornia State University Monterey Bay 831-582-4721 joseph_eng@csumb.edu

EWU’s Composition Program Exit Portfolio

A reflection essayA major paper chosen and revised by the

student (Shared Criteria scored)An in-class, timed essay on an assigned

Program prompt (Shared Criteria scored)

Paul Carney at Minnesota State Community & Technical College

Bridging the Gap: Identifying and Supporting College-Ready Writing Skills Among High School Students

Instructor, EnglishMSCTC, Fergus Falls218-736-1614Paul.Carney@minnesota.edu

Teacher : Teacher

Significant differences between high-school teachers and college instructors on “college-ready” rankings.

Notable inverse correlation between high-school grades and “college-ready” evaluation.

Frank talk from high-school teachers about lack of understanding of what college-ready meant.

Increased interest among college faculty in collaborating more with high schools (e.g., helping high-school teachers make better writing assignments)

Berkeley in 2009: Learning Outcomes?

The major in English is designed to introduce students to the history of literature written in English, to acquaint them with a variety of historical periods and geographical and cultural regions of English language and writing, to create an awareness of methods and theories of literary and cultural analysis, and to provide continued training in critical writing.

University of Illinois at Chicago says…

The English major curriculum provides for a significant broad-based knowledge as well as a degree of independent choice and specialization for each undergraduate major. It is designed to ensure a dynamic and coherent intellectual experience, to train students for further work in the discipline, and to draw on the diverse strengths of the English faculty.

Life After the Major

A major in English can open a world of opportunity for students. The analytical work we do in the English department improves students' ability to think critically-a skill that will be useful in any future endeavor. Additionally, students in the English department learn to produce precise, subtle, and well-crafted pieces of writing. All in all, a student who graduates with an English major is well-prepared to succeed in the world.

To prepare for life after the major, English majors are encouraged to complete a writing internship at a newspaper, magazine, public relations firm, non-profit organization or any place that relies on good writing skills.

Iowa State

Bachelor's graduates in Literary Studies will be able to Demonstrate knowledge of the nature of literature and the roles it plays in

culture and the expression of culture.

Demonstrate knowledge of the relevant working language of the discipline of literary study and the ways literature is defined, described, and classified.

Analyze and interpret important literary texts written in English, particularly British and American literature

Demonstrate knowledge of literary study as a discipline that makes use of specialized terminology and involves specific multiple intellectual perspectives, various analytical strategies, research, and writing.

Situate literature in historical, theoretical, aesthetic, social/political, ethical, and other contexts.

Demonstrate knowledge of skills in reading, writing, speaking, and research that are fundamental to the disciplined study of literature

Demonstrate knowledge of language as constantly changing and fundamental to cultural expression.

Thanks!

Hope your conference has been a great one.

Lynda Milnelynda.milne@so.mnscu.edu651-649-5741

The text and PPT will be up next week on SlideShare:

http://www.slideshare.net/lynda.milne