p ublic and p rivate ? using students’ academic engagement to understand one dimension of quality...
TRANSCRIPT
PUBLIC AND PRIVATE?
Using students’ academic engagement to understandone dimension of quality in Lithuanian higher education institutions
EAIR 2009 -- VilniusScott Barge
Harvard University
1
Lithuanian Higher Education• Re-organization of universities
post-independence• Emergence of private/non-state
institutions• Participation in Bologna
process• 2009 reforms– Governance– Financing
CO
NT
EX
T
2
Research QuestionAre students at private institutions more or less engaged in their academic work than students at public universities?
•Engagement is only 1 dimension of quality
•How do we define engagement?•Does engagement matter?•How do we measure engagement?
TH
EO
RE
TIC
AL
FR
AM
EW
OR
K
3
How do we define engagement?
“The time and energy students devote to educationally purposeful activities.”
(Kuh, 2001)
• These activities are, taken together, the single best predictor of students’ learning and personal development.
• Research includes: Astin (1993), Pascarella & Terenzini (1991), Pace (1980).
TH
EO
RE
TIC
AL
FR
AM
EW
OR
K
4
What are “educationally purposeful” activities?
“The time and energy students devote to educationally purposeful activities.”
(Kuh, 2001)
Seven Principles for Good Practice in Undergraduate Education
• Student-faculty contact• Cooperation among students• Active learning• Prompt feedback• Time on task• High expectations• Respect for diverse talents and ways of
learning(Chickering & Gamson, 1987)
TH
EO
RE
TIC
AL
FR
AM
EW
OR
K
5
What is the role of the institution?“The time and energy students devote
to educationally purposeful activities.”
(Kuh, 2001)
Seven Principles for Good Practice in Undergraduate Education
(Chickering & Gamson, 1987)
• Universities can shape curriculum, programs and policies in ways that promote these activities.
TH
EO
RE
TIC
AL
FR
AM
EW
OR
K
6
How do we measure engagement?Survey questions:
• “How often do you…”– Participate in class discussions?– Work with other students outside of class on
assignments?– Discuss your career plans with a faculty member?– Receive prompt feedback from instructors?– Have serious conversations with students of other
racial/ethnic or religious or political backgrounds?
• 22 questions forming a scale (α = .86)– Subscales: stu-instructor; stu-stu; diversity;
individual engagement
ME
AS
UR
ES
7
How do we measure engagement?
Additional measures:• Time preparing for classes (outside
of class)• Higher-level mental activity– Based on Bloom’s taxonomy– 4 items such as:
“How much has your coursework emphasized synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships?”
– (α = .70)
ME
AS
UR
ES
8
What else might be interesting?Non-engagement measures:
• Academic workload• Satisfaction• Time on other activities• Personal development (17 items; α
= .91)– “How much did your experience at this institution
contribute to your development in the following areas?”• A broad, general education• Job or work-related knowledge/skills• Writing clearly• Analyzing quantitative problems• Developing a personal code of values & ethics• Contributing to the welfare of your community
ME
AS
UR
ES
9
Who was in the sample?• Lithuanian Student Union (LSS)– Database of current students– Approximately 50,000 names– Bachelor’s, Master’s, Doctoral students
• ISM University of Management and Economics– Bachelor’s and Master’s students
• LCC International University– Bachelor’s students
SA
MP
LE
& D
AT
A C
OL
LE
CT
ION
10
Who was in the sample?• Primarily online administration• Personalized invitations &
reminders
Participation:• Overall response rate around 45%• About 21,000 responses
SA
MP
LE
& D
AT
A C
OL
LE
CT
ION
11
How did I analyze the data?• Main analysis for research question:–Multi-level linear regression over
indicator (dummy) variables– Allows for statistical control• Gender & residence• Year in program & level• Program
– Addresses the nesting of students in institutions
– Specification:
Where QUESTION is an indicator/dummy variable coded 1 if student from a private institution, 0 if student is not; X is a vector of covariates.
DA
TA
AN
AL
YS
IS
12
ENGAGEMENTij 0 1QUESTION j 2X i (ij i)
Are public institutions different?
FIN
DIN
GS
13
College and University Differences?F
IND
ING
S
14
So what have we learned?• Students at private institutions are, on
average, more engaged in “educationally purposeful activities” than students at public institutions
• Overall engagement at private universities is notably higher than at public universities (> .5 s.d.)
• Engagement at public and private colleges is between that at public and private universities.
PR
EL
IMIN
AR
Y F
IND
ING
S
15
What about outcomes?A
DD
ITIO
NA
L F
IND
ING
S
16
Limitations•Pitfalls of survey data–Non-response bias–Inaccurate recall
•Matrix sampling approach•Currently, Vilnius University is represented only to a limited degree• Possible translation effects
DIS
CU
SS
ION
17
Where do we go from here?• Adding Vilnius University– Currently, approximately 700 responses
• Additional studies of the survey itself– Accuracy of recall (LCC syllabus analysis)– Online question formatting– Translation effects
FU
TU
RE
RE
SE
AR
CH
18
• ISM, Petras Baršauskas, Jolita Butkienė, Ieva Martinkienaitė, Miglė Grigaitienė, Lineta Ramonienė
• LCC, Marlene Wall and colleagues• VDU, Kęstutis Pukelis, Izabela
Savickienė and colleagues• VU, Inga Milišiūnaitė• LSS, Jonas Okunis• VUSA, Paulius Simanavičius• Translation: Gražina Bielousova, Aistė
Motekaitienė, Radvyda Vaišvilaitė, Eglė Zalatoriūtė
Thanks to…C
ON
CL
US
ION
19
Appendix 1: Matrix SamplingA
PP
EN
DIC
ES
20
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
21
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
22
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
23
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
24
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
25
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
26
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
27
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
28
Appendix 2: Additional ResultsA
PP
EN
DIC
ES
29