using student engagement to stimulate change on campus john hayek, ph.d. senior associate director
TRANSCRIPT
Using Student Using Student Engagement to Stimulate Engagement to Stimulate Change on CampusChange on Campus
John Hayek, Ph.D.John Hayek, Ph.D.Senior Associate DirectorSenior Associate Director
Advance OrganizerAdvance Organizer
What kind of information about the student experience is compelling and useful for stimulating change and improvement on campus?
OverviewOverview
NSSE Primer
Ways to Stimulate Change
Institutional Examples
Open Discussion
Foundations of Student Foundations of Student EngagementEngagement
Assessing the Student Experience
1970s
Quality of Student Effort (Pace)
1980s
Student Involvement (Astin)
Social & Academic Integration (Tinto)
Good Practices in Undergraduate Education (Chickering & Gamson)
Learning and Development Model (Pascarella)
1990s
Student Engagement (Kuh)
National Survey ofNational Survey ofStudent EngagementStudent Engagement
Started in 1999 with 12 institutions – grown to 530+ in NSSE 2005
Over a half million students (first-year students and seniors) at 850 colleges and universities (2000-2004)
Focuses on promoting effective educational practice and institutional improvement
Web and paper versions; extensively tested to ensure validity and reliability
Assesses the extent to which student are engaged in educational practices related to high levels of learning and development
Using Student Engagement To Stimulate Change
1) Link to Mission & Accreditation
2) Share Information Widely3) Enhance Faculty
Development4) Benchmark (External &
Internal)5) Connect to Outcomes and
Other Campus Data6) Emphasize Effective
Educational Practices
1. Link to Mission & 1. Link to Mission & AccreditationAccreditation
AASCU Mission StatementsKey Words
DiversityTeaching and LearningGeneral EducationInformation TechnologyLiberal ArtsCommunityProductive CitizensScholarshipResearchAcademic ExcellenceHigh Quality EducationCritical ThinkingHealth and WellnessStudent DevelopmentComprehensiveGlobal
NSSE Areas of Focus
Academic & Social Experiences
Technology
Diversity
Higher Order Thinking
Reading and Writing
Time Usage
Enriching Educational Experiences
Quality of Relationships with Students, Faculty, and Staff
Arts, Wellness, & Spirituality
Civic Engagement
Campus Environment
Advising and Mentoring
Satisfaction
Personal and Educational Growth
1. Link to Mission & 1. Link to Mission & AccreditationAccreditation
NSSE Accreditation Toolkit
Links Between NSSE & Accreditation
Mapping NSSE to Accreditation Standards
Vignettes of Institutional Usage
2. Share Information 2. Share Information WidelyWidely
“NSSE is a great way to stimulate reflection and debate about what we do more and less well, and why. For us it’s proving an exciting and enlivening tool for self-reflection and self-improvement.”
Internal External
President Governing Boards
Faculty / Committees / Deans / Chairs
Accreditation
Students / Groups / Organizations Alumni
Service Learning Prospective Students
Enrollment Management / Admissions
Media
Student Affairs / Student Services Parents
First-year Experience Fund Raising
Advising State Policy Makers
Assessment & Institutional Research Performance Indicators
Michael McPherson, President of The Spencer Foundation(former President of Macalaster College)
2. Share Information Widely2. Share Information Widely
3. Enhance Faculty 3. Enhance Faculty DevelopmentDevelopment
Mini-Grant: Early Engagement of Mini-Grant: Early Engagement of First-Year StudentsFirst-Year Students
Vice President for Academic Affairs
Identified start-up resources (up to $2,500 per year for two years)
Help academic departments introduce students engagement initiatives
Designed to improve department’s engagement of its students during their first semester at the university
• Faculty Retreats & Workshops
• Scholarship of Teaching & Learning
• Course Evaluations
• Tip of the Week
• Incentive Programs
• FSSE / NSSE Gap Analysis
3. Enhance Faculty 3. Enhance Faculty DevelopmentDevelopmentFSSE / NSSE Gap MatrixFSSE / NSSE Gap Matrix
Mass Comm 4307 Importance x Engagement Matrix
Faculty Say 50% +
“Important” or Students Say
“Very Important” Never
(14) Participate in a community-based project
(18) Talk about career plans with instructor
Faculty Say 50% +
“Important” or Students Say
“Very Important” “More than 5 times”
(1) Ask questions during class
(9) Work with classmates on projects during class
(11) Put together ideas/concepts form different courses when completing assignments
(12) Put together ideas/concepts from different courses during class discussions
(24) Analyze basic elements of an idea, experience, or theory
(25) Synthesize/organize ideas, information, or experiences
(26) Make judgments about the value of information, arguments, or methods
(27) Apply theories/concepts to practical problems
50% +
Faculty Say Students Say
“Not Important” Never
(13) Tutor/teach other students
50% +
Faculty Say Students Say
“Not Important” “More than 5 times”
(10) Work with other students outside of class to prepare assignments
(22) Discuss ideas from class with others outside of classHit
HitMiss
Miss
4. Benchmark - External4. Benchmark - External
Variable
Bench-mark Class Mean Mean Sig a
Effect
Size b Mean Sig a
Effect
Size b
1. Academic and Intellectual Experiences
FY 1.96 2.50 2.69
SR 3.11 3.10 3.18
FY 2.38 2.23 *** .48 2.24 *** .50
SR 2.65 2.74 2.45
FY 2.96 2.49 ** .29 2.44 ** .32
SR 2.53 2.66 ** .27 2.68 *** .32
FY 3.20 3.04 3.05
SR 3.27 3.35 3.35
FY 2.82 2.73 2.73
SR 2.73 2.81 2.81
FY 1.77 1.89 1.99 * -.24
SR 1.82 2.10 2.03 * -.18
FY 2.45 2.37 2.33
SR 2.68 2.51 * .20 2.44 ** .28
FY 2.05 2.32 ** -.33 2.39 *** -.41
SR 2.54 2.71 * -.19 2.73 ** -.21
FY 2.57 2.44 2.47
SR 2.68 2.84 * -.20 2.86 * -.22
ACL
ACL
ACL
ACLWorked with classmates outside of class to prepare class assignments
OCCGRP
Put together ideas or concepts from different courses when completing assignments or during class discussions
INTIDEAS
Come to class without completing readings or assignments
CLUNPREP
Worked with other students on projects during class
CLASSGRP
Worked on a paper or project that required integrating ideas or information from various sources
INTEGRAT
Included diverse perspectives (different races, religions, genders, political beliefs, etc.) in class discussions or writing assignments
DIVCLASS
Made a class presentation CLPRESEN
Prepared two or more drafts of a paper or assignment before turning it in
REWROPAP
Nesseville State UniversityNSSE 2004 Means Comparison Report
Asked questions in class or contributed to class discussions
CLQUEST
In your experience at your institution during the current school year, about how often have you done each of the following? 1=never, 2=sometimes, 3=often, 4=very often
NSSE 2004Nesseville State
Nesseville State compared with:
Master's
a.
b.
c.
d.
e.
f.
g.
h.
i.
National Survey of Student Engagement
• Consortia
•ADP
• Peer Groups
• Aspirant Groups
• Special Analyses
4. Benchmark - External4. Benchmark - External
2003 IUB Benchmark Deciles
National 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100
Level of Academic Challenge
45 49 50 51 52 53 55 56 57 60 68 40 53 54 55 56 57 58 59 60 63 74
Active and Collaborative Learning
30 36 37 39 40 41 43 44 46 48 60 38 45 47 48 49 50 51 52 54 55 66
Student-Faculty Interaction
23 31 32 34 35 36 38 40 41 45 74 28 36 38 40 42 43 45 47 50 54 70
Enriching Educational Experiences
40 48 51 53 56 58 59 61 64 67 80 31 41 42 44 46 48 51 53 55 59 75
Supportive Campus Environment
46 55 57 59 60 62 63 65 66 69 85 45 51 54 55 57 59 60 62 64 66 76
Res/Doc-Ext 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100
Level of Academic Challenge
47 48 49 50 51 51 53 54 55 57 62 50 52 53 54 55 55 55 57 57 59 61
Active and Collaborative Learning
33 35 35 36 37 38 38 40 41 42 47 39 43 44 45 45 46 47 47 48 49 56
Student-Faculty Interaction
29 31 32 32 33 34 34 35 36 38 44 31 36 37 38 38 39 41 42 43 44 50
Enriching Educational Experiences
48 52 53 55 56 58 59 60 61 64 71 39 43 44 45 46 47 48 49 52 54 58
Supportive Campus Environment
50 54 55 56 57 58 59 59 60 61 72 45 48 50 51 52 53 54 55 57 58 69
First-Year Senior
Level of Academic Challenge
Nesseville ConsortiumCarnegie National
First-Year 52.5 52.4 51.8 53.4
Senior 56.3 55.6 54.9 57.0
Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote high levels of student achievement by emphasizing the importance of academic effort and setting high expectations for student performance
25
35
45
55
65
75
Ben
chm
ark
Scor
es
Nesseville 52.5 56.3
Consortium 52.4 55.6
Carnegie 51.8 54.9
National 53.4 57.0
First-Year Senior
Level of Academic Challenge Items:
Preparing for class (studying, reading, writing, rehearsing, etc. related to academic program)
Number of assigned textbooks, books, or book-length packs of course readings
Number of written papers or reports of 20 pages or more; number of written papers or reports of between 5 and 19 pages; and number of written papers or reports of fewer than 5 pages
Coursework emphasizing analysis of the basic elements of an idea, experience or theory
Coursework emphasizing synthesis and organizing of ideas, information, or experiences into new, more complex interpretations and relationships
Coursework emphasizing the making of judgments about the value of information, arguments, or methods
Coursework emphasizing application of theories or concepts to practical problems or in new situations
Working harder than you thought you could to meet an instructor's standards or expectations
Campus environment emphasizing time studying and on academic work
Benchmark Act.Pred
.Residua
lStandard Residual
Academic Challenge 56.0 52.4 3.5 1.3
Active Learning 43.5 41.4 2.1 .6
Stu-Fac Interaction 42.7 35.8 6.9 1.7
Enriching Experience 50.6 50.4 .2 0.0
Supportive Environment 70.8 63.3 7.5 2.0
4. Benchmark – Internal4. Benchmark – Internal
4. Benchmark – Internal4. Benchmark – Internal
30
35
40
45
50
55
60
Lowest Major
Average
Highest Major
Business Engineering Other Education Profes-sional
Arts &Humanities
SocialSciences
BiologicalSciences
Math &PhysicalSciences
Management
ChemicalEngineering
MechanicalEngineering
CriminalJustice
Kinesiology
PhysicalEducation
Elem./MiddleEducation
Pharmacy
Pre-Med
Theater orDrama
Speech
PoliticalScience
Sociology
Biochemistry
EnvironmentalScience
Chemistry
Math
InternationalBusiness
5. Connect to Outcomes & 5. Connect to Outcomes & Other Campus DataOther Campus Data
Retention Rates to Second Year by Engagement
79
84 8582 82
8488
86 8785
91 91 9188 89
9395
9289
9292
84
9193 93
0
10
20
30
40
50
60
70
80
90
100
Level of AcademicChallenge
Academic andCollaborative Learning
Student Interactionw ith Faculty
Enriching EducationalExperiences
Supportive CampusEnvironment
Very Low Low Average High Very High
NSSE BenchmarksGrad. Rate
SR
Satisfaction
FY
Satisfaction
SR
Academic Challenge .46 .29 .28
Active & Collaborative Learning
.09 .25 .23
Student Faculty Interaction .37 .25 .29
Enriching Educational Experiences
.48 .22 .23
Supportive Campus Environment
.26 .56 .60
5. Connect to Outcomes 5. Connect to Outcomes & Other Campus Data& Other Campus Data
5. Connect to Outcomes 5. Connect to Outcomes & Other Campus Data& Other Campus Data
In-house surveys
National surveys CIRP / CSS
YFCY
CSEQ / CSXQ
EBI Benchmarking surveys
Noel Levitz Student Satisfaction Inventory
ETS Major Field Tests ACT Collegiate Assessment of Academic
Proficiency
Institutional data such as GPA, financial aid, transcripts, retention, certification tests, etc.
6. Emphasize Effective 6. Emphasize Effective Educational PracticesEducational Practices
1) A “living” mission and a “lived” educational philosophy
2) An unshakeable focus on student learning
3) Clearly marked pathways to student success
4) Environments adapted for educational enrichment
5) An improvement-oriented campus culture
6) Shared responsibility for educational quality and student success
Based on higher than predicted graduation rates and student engagement
Institutional ExamplesInstitutional Examples
Many schools are stimulating change and improvement on campus by using student engagement data.
University of Missouri – St. University of Missouri – St. LouisLouis
Glen Hahn Cope
Provost and Vice Chancellor for Academic Affairs
Using NSSE Data to Stimulate ChangeAASCU 2005 San Diego Meeting
University of Missouri University of Missouri inin St. St. LouisLouis
Urban public research university
Recent leadership and vision changes
UM System Strategic Planning UMSL Action Planning
NSSE Participation (begins with 2000 pilot)
From “administrative use only” in 2000 to
2003, 2004 campus-wide forums From “denial of results” to colleges
verifying, faculty discussions, and Action Plan benchmarks
University and Campus University and Campus Activities Activities
University System-wide Efforts UM New Faculty Teaching Scholars President’s Academic Leadership Institute
Campus-wide Efforts Center for Teaching and Learning (est. 10/2000)
Faculty and TA programs, orientations Kuh presentations with academic and student
affairs leaders, early career faculty (2/02) Engagement concepts routinely used in
program names UMSL at 40:Campus Conversation Series 2003-
2004 November: Student engagement February: Engaged research April: Community Engagement February ‘05:Creating an Engaged University
Action Planning 2003-2004 Office of the Provost created August 2004 Reliance on faculty governance, input Benchmarks with NSSE data
College ActivitiesCollege Activities
College efforts – sampling Arts and Sciences appended with
permission 15 NSSE items to fall semester 2002 course evaluations
College of Business Administration includes all majors in capstone course in its sample
Honors College oversamples its freshmen and seniors
College efforts – discussion and action Presentations about NSSE invited by
COE, CoBA A & S Dean’s charge to departments Embracing undergraduate research
OutcomesOutcomes Increased campus-wide awareness (FSSE,
NSSE) Persistent conversations Increased communication Responsibility assumed at unit level
Increased acceptance of methodology, data
Interventions identified in colleges and departments
Increased response rates NSSE FSSE2003 38.0% 26.4%2004 47.5% 44.5%
Norfolk State University (NSU)
• University of opportunity
• Wide variety of programs for students seeking access to an affordable high-quality education
• Founded in 1935
• Located in the downtown Norfolk, Virginia
• Virginia’s largest public historically black university (HBCU)
• Seventh largest HBCU in the nation
• Approximately 6,000 culturally diverse students
Surveys of Student Engagement at NSU
• Spring 2002– NSSE (paper mode)
• Spring 2003– NSSE (web-based mode)– FSSE (web-based mode)
• Spring 2004– NSSE (local administration)
• Fall 2004– BCSS (paper mode)
• Spring 2005– NSSE (web+ mode)
Using NSSE for Quality Enhancement
• NSSE Data
• NSSE Process
• NSSE Concept
Using NSSE for Quality Enhancement: Data
• Set up internal and external benchmarks to assess and monitor NSU performance on NSSE benchmarks and individual items, salient for NSU
• Provide information for internal decision-making and strategic planning
• Engage faculty, administrators, and students in conversations to explore best educational practices
Using NSSE for Quality Enhancement: Data (Cont’d)
• Advance campus initiatives
• Articulate and affirm effective institutional practices and improve NSU self-image and community perception
• Triangulate internal reports and research projects
• Triangulate external reporting
Using NSSE for Quality Enhancement: Process
• Provide information for internal decision-making
• Advance campus initiatives
• Identify effective methods to administer other university-wide surveys.
Using NSSE for Quality Enhancement: Concept
• Begin developing a more comprehensive institutional concept of academic quality
• Attract faculty and administrators’ attention to best practices in the undergraduate education
Using NSSE for Quality Enhancement: Future Plans
• Assessment of new campus initiatives– First-Year Experience
– American democracy project (ADP)
• Reaffirmation of Accreditation– Southern Association of Colleges and Schools (SACS)
Compliance Certification Audit
– SACS Quality Enhancement Plan (QEP) Theme
Contact Information
Nuria M. Cuevas, Ph.D.
Associate Vice President for Academic Affairs andDirector, Institutional Effectiveness and Assessment
NORFOLK STATE UNIVERSITY
Phone: (757) 823-8408E-Mail: [email protected]
Web: www.nsu.edu/iea
Using NSSE DataUsing NSSE Data
Sharon Hahs
ProvostFebruary 6, 2005
A Metropolitan University in a Pastoral Setting
◆ In Illinois, 20 minutes from the St. Louis Arch
◆ Only public university in Southwestern Illinois
◆ Most populous region of downstate Illinois
◆ St. Louis metro area includes 2.7 million people
With Programs in:
◆ Arts and Sciences
◆ Business
◆ Dental Medicine
◆ Education
◆ Engineering
◆ Nursing
◆ Pharmacy
◆ 13,493 Students
◆ More than half of students receiving baccalaureate degrees started as transfer students.
◆ About 30 percent of undergraduates are new each fall, two-fifths of those as transfer students.
◆ On-Campus Housing for 2,900 Students◆ Most students live and work in a 60 mile radius and commute to classes.
10,811 Undergraduate
2,485 Graduate
198 Professional
Long Term Goals ◆ Revisited Every 5 to 10 Years
◆ Measured Annually
◆ Lead to Short Term Goals
Short Term Goals◆ One to Three Years Long
◆ Founded on evidence
◆ Measured and Evaluated
◆ Lead to Actionable Projects
SIUE’s Long-Term GoalsSIUE’s Long-Term Goals
1. Engaged Student and Capable Graduates
2. Innovative High Quality Programs3. Committed Faculty and Staff4. Harmonious Campus Climate5. Active Community Engagement6. Sound Physical and Financial
Assets7. Excellent Reputation
Measures for Long Term GoalsMeasures for Long Term Goals
◆ Multiple Measures
◆ Longitudinal Data
◆ External Comparative Data
◆ Intra-Institutional Data
Has content parallel to existing alumni and faculty surveys
Annual since 2000
Urban Consortium
Can be “cut” to components within SIUE
NSSE:
Uses of NSSE in Measuring Uses of NSSE in Measuring Long Term GoalsLong Term Goals
◆ Measures of Student Engagement
◆ Measures of Faculty and Staff Commitment to Educational Opportunity
◆ Measures of Harmonious Climate
◆ Measures of Community Engagement
◆ Measures of SIUE Reputation
Freshman Seminar ProposalFreshman Seminar ProposalApril 2002April 2002
• Piloted four options:– Honors Seminars– University Experience Course– Culture, Ideas, Values Course (CIV)– Learning Communities in Academic
Development
Common goals for freshman seminars—Freshman Seminar Proposal 2002
1--To assist new freshman in making the transition to college level work and expectations;
2--To orient students to the services and culture of the University;
3--To engage students in an intellectual community of students and faculty.
Review committee and use of Review committee and use of special studyspecial study
• Ad hoc subcommittee of the Curriculum Council, Faculty Senate– Literature Search– Focus Groups– NSSE study
• Summary Report—NSSE 2003 Special Course Oversample April 19, 2004
Of the questions that showed significance at the 0.1 level, four either directly or indirectly related to the objectives of the freshman seminar course as outlined in the proposal. These included the following questions:
1 A--Asked questions class or contributed to class discussion.
10 F--Attending campus events and activities (special speakers, cultural performances, etc.)
11 K--Understanding yourself
13--How would you evaluate your entire educational experience at this institution?
(Summary Report—NSSE 2003 Special Course Oversample April 19, 2004)
New Student Seminar Task Force Report and New Student Seminar Task Force Report and Recommendations (June, 2004)Recommendations (June, 2004)
• Recommendation 1: Adopt a freshman seminar requirement
• Recommendation 2: Include a freshman seminar in general education reform.
• Recommendation 3: Create committee for implementation and management.
Question 7. Which of the following have you done or do you plan to do before you graduate from your institution?h. Culminating senior experience (comprehensive
exam, capstone course, thesis, project, etc.)
• Response to 7.h.:– Yes—70%– No—25%– Undecided—5%
Senior Assignment is a graduation requirement
Activity--NSSE questions Activity--NSSE questions related to Learning Objectivesrelated to Learning Objectives • Compare the NSSE questions with the
SIUE Statement of Objectives – 1. Identify a NSSE question that could make
a difference.– 2. What objective does this question
measure and how does the question measure it?
– 3. How is this question actionable? What could be done to improve the score?
Committee on AssessmentCommittee on AssessmentAQIP Action Project Recommendations
• Student perception and understanding of the Senior Assignment as a culminating experience (NSSE questions 2c, 7g,h, 11j,m)
• Student perception of academic advising (NSSE questions 1o, 12)
• Quantitative reasoning (NSSE questions 2b,d,e, 11f)
• Communication (speaking and writing) (NSSE questions 1a,p,q, 4c,d,e, 11c,d)
• Expectations, relationships, and diversity (NSSE questions 8, 9)
Open DiscussionOpen Discussion
www.nsse.iub.edu