SHOCK EFFECT 2001
Moving From Data to Goals
Dr. Gary Steward, Associate Dean & Professor of Sociology
Dr. Stacy Southerland, Professor of Modern Languages, Literature & Culture
Dr. David Macey, Professor of English
Ms. Karen Henderson, Director of Assessment
UCO’s Experience with NSSE
Administered Spring 2001 with a 33% response rate (42% for all NSSE 2001 institutions)
Administered Spring 2003 with a 34% response rate (43% for all NSSE 2001 institutions)
Next administration is Spring 2006
In 2001 NSSE Reported
Would They Start Over at UCO? Students with a good
or excellent experience would attend UCO if they were starting their educational career over again. 0%
20%
40%
60%
80%
100%
GoodExperience
SameInstitution
Fresh Senior
College Satisfaction
2001 Benchmarks
41 questions are assigned to five clusters of similar activities and
conditions to make up the national benchmark of effective
educational practices
Comparison of UCO to NSSE Schools on Benchmarks 2001
BenchmarkLevel of Student
**Standardized Residual (SR)
SR greater than approximately ___ % of NSSE schools
Level of Academic Challenge First year -2.2 1%Senior -0.3 38%
Active and Collaborative Learning First year -0.1 46%Senior 0.2 58%
Student Interactions with Faculty Members First year -0.6 27%Senior 1.5 93%
Enriching Educational Experiences First year -0.2 42%Senior 1.5 93%
Supportive Campus Environment First year -1.4 9%Senior 0.5 69%
*Actual benchmark scores statistically adjusted for the types of students at UCO and other institutional characteristics.
**Standardized residual is an estimate of the degree to which UCO exceeded or fell short of its predicted score relative to all other NSSE institutions.
Level of Academic Challenge Time spent preparing for class, amount of reading
and writing, and institutional expectations for academic performance
0.0
20.0
40.0
60.0
First-Year Senior
UCO Master's National (NSSE)
College of Liberal Arts (LA) Said
Academic Challenge Items focused on in the local LA Instruments
• Used Local Survey Items and Focus Groups to study
• Preparing for class• Worked harder than you thought you would to
meet an instructor’s standards or expectations
• Bloom’s taxonomy items• Campus environment emphasizes spending
significant amounts of time studying and on academic work
Academic Challenge Items focused on in the local LA Instruments
• Syllabi Review was used to study• Number of assigned textbooks, books, or
course readings• Number of written papers or reports of 20
pages or more….between 5 and 19 pages ….fewer than 5 pages
• Indication of significant expectations for work outside of the classroom.
Locally Developed Surveys & Focus Groups
Student Engagement Academic Standards Foundation Courses
Survey response averages 275 forms.
Focus Groups averages 14 participants.
TOPICS
College Level Faculty Input
Locally developed Surveys and Focus Groups Address issues tied to the College and Academic
Challenge Provides feedback regarding satisfaction with the
college administration
So We Took ACTION To Change!
Action Taken
Set College Goals Based on NSSE Data Writing Initiative First Day Engagement
Reinforce Positive Faculty Behavior Teaching Awards Strategic Plans Perception of Instructional Effectiveness Forms
Action Taken
Explored Options for Improving Academic Challenge Asked students to define Academic Challenge Asked faculty how to enhance Academic
Challenge
LA Academic Goals
2002 – Every class will have a writing requirement
2004 – First Day Engagement Initiative
Academic Standards
2001-02 54% claimed that ¾ or all of their LA courses were easier
than expected.
2005 Only 50% reported that ¾ or all of their upper division LA
courses were easier than expected. Only 36% reported ¾ or all of their lower division courses
were easier than expected.
“Overall, my classes are easier than I expected.”
Academic Standards
2001-02 11% reported none of their LA courses were
easier than expected. 2005
20% reported none of their LA courses were easier than expected.
“Overall, my classes are easier than I expected.”
Academic Standards
2001-02 Only 15% reported that they were intellectually
challenged in ¾ or all of their LA courses. 2005
65% (lower division) and 72% (upper division) claimed that they were intellectually challenged in ¾ or all of their LA courses.
Academic Standards
2001-02 42% claimed none of their courses challenged
them intellectually. 2005
Only 7% (lower division) and 8% (upper division) reported that none of their LA courses were intellectually challenging.
Academic Standards
“I take LA courses because it is a nice respite from biology.” Many members commented that LA courses allowed them to take “a breather.”
Students did comment on more oral presentations and some claimed that LA courses require more writing than courses in other colleges.
Student focus groups in 2001-02 reported:
Students have reported an increase in writing requirements in the last year.
“I don’t like it when teachers are giving writing assignments because it’s expected now.”
“I get aggravated with taking 12 hours and having instructors who are acting as if I’m only taking their class. I can’t have four hard teachers and make it through, so I look for easier classes.”
Student focus groups in 2005 reported:
Our first day engagement data (2005, 738 students) revealed the following
Their interest increased (68%) They felt the class would be more
challenging than they expected (50%) They believed they could be successful
in the course (93.5%)
Students reported that after the first day of class:
Writing Initiative
2001-2002: 13% stated ¾ or all of their LA courses. 48% reported that they were not required to write a
paper in any of their courses.
2005: 74% (lower division) and 86% (upper division)
reported required writing in ¾ or all of their LA courses.
Only 8% (lower division) and 4% (upper division) claimed that none of their classes required a paper.
“I am required to write a paper”
Writing Initiative
In 2001-02, 28% reported taking exams that included essays in ¾ or all of their courses.
In 2005, 66% (lower division) and 75% (upper division) reported taking exams that included essays in ¾ or all of their courses.
Writing Initiative
In 2001-02, 29% claimed that none of their LA courses included essays in their exams.
In 2005, only 9% (lower division) and 3% (upper division) reported that none of their LA courses included essays in their exams.
Faculty Reaction
“Our department has made a conscious effort to raise standards and rigor at the general education and survey level as well as in upper-division and graduate courses. Our cornerstone course offers students an opportunity to spend a semester researching and writing a fifteen-page paper based . . . on primary sources . . . . Students who are engaged in the course produce scholarly papers that compete well at regional conferences against students from OU and OSU.”
- History Faculty Member
Faculty Reaction
“Our department has begun producing custom readers for our general education courses because we found that commercially available readers did not respond effectively to the range of writing challenges that our first-year students face. Expanding the scope of our first-year writing curriculum has enabled us to increase the number and variety of writing assignments, including both shorter essays and longer research papers, that our first-year students complete.”
- English Faculty Member
Faculty Reaction
“I have, as a result of assessment activities, reevaluated how and what I communicate to the students with regard to my expectations for their achievement on writing assignments. I also give more detailed information about specific evaluation criteria by providing the form that I use to break down the score into areas of thesis statement, development, grammar, MLA style, and so on.”
- Modern Languages Faculty Member
Faculty Reaction
“In each of my junior- and senior-level courses, students complete at least one major research paper . . . . I emphasize throughout the semester that utilizing a grounded style of writing is preferable to a ‘my way of writing’ paper or a disorganized one.”
- Mass Communication Faculty Member
Faculty Reaction
“In the senior-level seminar, I assign three types of readings for discussion at meetings and writing about two of them for almost every meeting . . . . In the sophomore-level course, students are required to submit an essay on Turnitin.com . . . and a minimum three-page paper following a required visit to an art museum”
- Humanities & Philosophy Faculty Member
Faculty Reaction
“In 2001, we introduced a new, sophomore-level ‘cornerstone’ class focusing on writing and research strategies specific to our discipline. Our majors now receive a systematic introduction to academic writing in English Studies before advancing to junior- and senior-level courses . . . . Because instructors are able to count on students having this foundation, they are able to assign more varied writing projects and to require more writing without having to ‘backtrack’ in class to fill in gaps in students’ academic preparation.”
- English Faculty Member
Faculty Reaction
My junior-level course has two group work activities, four exams, and six quizzes. This year I added a written component. Now they summarize one of the chapters not covered in class and set up an interview with a practitioner working in the area. They then write up that interview in essay style. . . . . I also added a written component to my senior-level course. The best papers were presented in class. In researching these essays, a number of students changed their mind about the issues in question.”
- Mass Communication Faculty Member
New College Goal
2005 – Every class will hold students accountable for assigned reading.
Academic Affairs Mission
“Helping students learn so that they may become productive, creative, ethical, and engaged citizens.” Academic Values
Integrity Student Centered Academic Excellence Continuous Improvement Collegiality Community
Strategic Plan Process Implemented
Process has Four Criteria Helping Students Learn Accomplishing Other Distinctive Objectives Understanding Students and Other
Stakeholders’ Needs Valuing People
Perception of Instructional Effectiveness Form Revised
Based on 7 Principles of Good Undergraduate Education
Piloted in Summer 2002 Partially implemented in Fall 2002 Instrument administered on-line
Changes in Benchmarks
2001 2003
BenchmarkLevel of Student
**Standardized Residual (SR)
SR greater than approximately ___ % of NSSE schools
**StandardizedResidual (SR)
SR greater than approximately ___ % of NSSE schools
Level of Academic Challenge First year -2.2 1% -1.2 14%
Senior -0.3 38% -0.5 31%
Active and Collaborative Learning First year -0.1 46% -0.2 44%
Senior 0.2 58% -0.4 38%
Student Interactions with Faculty Members First year -0.6 27% 0.1 52%
Senior 1.5 93% 0.1 52%
Enriching Educational Experiences First year -0.2 42% -0.8 20%
Senior 1.5 93% 0.1 52%
Supportive Campus Environment First year -1.4 9% -0.7 24%
Senior 0.5 69% 0.0 50%
Can’t Wait!
Spring 2006 UCO is conducting the National NSSE again.
Will the benchmarks go up?
Thank You!
Dr. Gary Steward, Associate Dean & Professor of Sociology, [email protected]
Dr. Stacy Southerland, Professor of Modern Languages, Literature & Culture, [email protected]
Dr. David Macey, Professor of English, [email protected]
Ms. Karen Henderson, Director of Assessment, [email protected]