nsse.iub.edunational survey of student engagement an experience-based view of university quality in...

nsse.iub.eduNational Survey of Student Engagement

An Experience-Based View of University Quality in the USA:

The National Survey of Student Engagement

Alexander C. McCormick

Center for Postsecondary Research

Indiana University Bloomington, USA

HEPI-HEA ConferenceThe British Academy, London

22 May 2012

• Context: Conceptions of quality in U.S. tertiary education

• The National Survey of Student Engagement (NSSE)

• Low stakes, high yield assessment• Student engagement trends• Controversies and cautions

Overview


Context

Higher education in the USALimited understanding of quality

Student engagement as a new approach


Higher Education in the USA

No national system of higher education Autonomy of institutions Diversity of institutions 4,600 degree-granting tertiary institutions Public and private Universities (undergraduate and

postgraduate) Colleges (undergraduate only)

• 4-year bachelor’s degree• 2-year associate’s degree (community colleges)


Higher Education in the USA (2)

Diversity of students Full- and part-time Traditional college-age Older and returning students Widely varying levels of preparation


What Do We Really Know About University Quality?


Quality Assessment in US Higher Education

Accreditation Limited public understanding Limited disclosure Traditional emphasis on capacity and resources Limited attention to teaching & learning (until

recently)

Newsmagazine rankings Emphasize reputation, resources, other inputs

Research rankings Blind to undergraduate education


Origins of the “National” Survey of Student Engagement (NSSE)

A reaction to the dominant discourse about university quality in the USA

Shift the focus to teaching and learning

Promote evidence-based improvement


The Big Idea

Ask students about their experience

Focus on behavior: empirically confirmed effective practices


Why a Student Survey?

Direct assessment of learning is problematic Measurement challenges Inferential challenges Cost challenges Motivation challenges

Survey approach offers advantages Cost-effective and efficient Diagnostic utility of behavioral data Students are the experts on their experience


NSSE’s Key Aims

Enrich the impoverished national discourse about university quality Shift the focus to teaching & learning,

through the lens of effective practices

Provide diagnostic information that can be used to improve undergraduate education Actionable information based on

valid & reliable measures Meaningful comparisons to other institutions


What is Student Engagement?

The extent to which students engage in and are exposed to effective educational practices Activities and practices

known to be related to desired outcomes


OK, but what is Engagement?

Challenging academic work

Active learning Deep approaches to learning

High-impact practicesInvolvement with instructors

In a campus context that

promotes and supports success


Conceptual & Empirical Foundations

Time on task (Tyler, 1930s)

Quality of effort (Pace, 1960s & 70s)

Student involvement (Astin, 1984)

Academic & social integration (Tinto, 1975 & 1987)

Good practices in undergraduate education (Chickering & Gamson, 1987)

College impact (Pascarella, 1985)

Student engagement (Kuh, 1991, 2005)


Why Does Engagement Matter?

The impact of college is largely determined by individual effort and involvement in the academic, interpersonal, and extra-curricular offerings on a campus. It is important to focus on the ways an institution can shape its offerings to encourage student engagement.

Paraphrased from Pascarella & Terenzini (2005), p. 602

How Does it Work?

What Do We Ask?


How NSSE Works

• Paper or Web-based survey• First- & final-year

undergraduates• Uniform, centralized survey

administration• Institutions receive detailed

reports and an identified student data file

• Customizable comparison groups

• Results are confidential• Supported by institution fees


Participation

550 to 770 U.S. & Canadian institutions per year

About 1,500 since NSSE’s launch in 2000

Most repeat every 1-4 years


Survey Content

Academic activities & experiences• Academic

challenge• Reading and

writing• Active learning• Cognitive tasks in

courses• Enriching

educational experiences

Time use Study, work,

socializing…

Co-curricular activities

Institutional emphases

Quality of campus relationships

Cognitive and noncognitive gains

Satisfaction Demographic & enrollment characteristics


Sample Questions

Asked questions in class or contributed to class discussions?

Made a class presentation?Prepared two or more drafts of a paper or assignment before turning it in?

In your experience at your institution during the current school year, about how often have you…


Sample Questions

Put together ideas or concepts from different courses when completing assignments or during class discussions?

Discussed ideas from your readings or classes with faculty members outside of class?



Sample Questions

Received prompt written or oral feedback from faculty on your academic performance?

Worked harder than you thought you could to meet an instructor’s standards or expectations?



Sample Questions

Memorising facts, ideas, or methods…

Analysing the basic elements of an idea…

Synthesising and organizing ideas, information…

Making judgments about the value of information…

Applying theories or concepts…

During the current school year, how muchhas your coursework emphasized the following mental activities?


Sample Questions

Examined the strengths or weaknesses of your own views on a topic or issue?

Learned something that changed the way you understand an issue or concept?

During the current school year, about how often have you done each of the following?


Using NSSE

Results

How are we doing?

How can we improve?


Using NSSE Results

Diagnostic purpose: analysing the undergraduate experience

Identify strengths and weaknesses, areas where improvement is needed

Stimulates conversations about quality

Informs decision-making with “actionable” information


Many Ways to Use Results

Peer comparisons How do we measure

up to others?

Self comparison Are we improving?

Absolute standard How do we measure

up to our ideals?

Internal variation Who is least

engaged?

What to look at and how to look at it

NSSE results as a catalyst for campus discussions

about teaching & learning


Comparing Student andInstructor Perceptions

First-year students

Instructors Final-year students

Instructors0%

20%

40%

60%

80%

100%

69%

33%

62%

23%

Courses emphasised memorisation “Quite a bit” or “Very much”

Faculty Survey


Comparing Student andInstructor Perceptions

First-year students

Instructors Final-year students

Instructors0%

20%

40%

60%

80%

100%

69%

33%

62%

23%

Courses emphasised memorisation “Quite a bit” or “Very much”

Results at institutions with

several NSSE administrations show many with positive trends


Summary of Findings

• 41% of institutions demonstrated a pattern of improvement in at least one measure for first-year students

• 28% did so for final-year students

• Trends found across many institutional types Control (public & private), Size, Research emphasis

• Ratio of positive to negative trends: 7:1


Overview of Observed Changes: FY

Diversity Exp.

High-Impact

Integrative

Higher-Order

Supp. Campus

Stu-Faculty Int.

Active & Collab.

Academic Chall.

-10 0 10 20 30 40 50 60 70 80 90 100

Percentage with positive or negative trends

PositiveNegative


Overview of Observed Changes: Sr

Diversity Exp.

High-Impact

Integrative

Higher-Order

Supp. Campus

Stu-Faculty Int.

Active & Collab.

Academic Chall.

-10 0 10 20 30 40 50 60 70 80 90 100

Percentage with positive or negative trends

PositiveNegative


Cautions & Controversies

NSSE is no “magic bullet”The risks of high-stakes uses

“Participation” vs. “use”


Cautions

NSSE is no “magic bullet”: Best used in combination with other information

NSSE’s research foundation is based on studies of US university students

Most variation is between students, not institutions


Controversies

Confidentiality of results: Institutions control disclosure

Attaching high stakes to NSSE is risky Institutions will decline to participate if

public relations cost > institutional benefit Reliability of responses could be

compromisedMere participation is not sufficient Results must be analysed and used


Understanding Improvement

Organizations that improve… create and nurture agreement on what is worth achieving, and they set in motion the internal processes by which people progressively learn how to do what they need to do in order to achieve what is worthwhile.

(Elmore in Fullan, 2001)

Thank you.

nsse.iub.edunational survey of student engagement an experience-based view of university quality in...

Documents