libqual+® and the evolution of library as place at radford university, 2002-2008 eric ackermann...
TRANSCRIPT
LIBQUAL+® AND THE EVOLUTION OF “LIBRARY AS PLACE” AT RADFORD UNIVERSITY, 2002-2008
Eric AckermannRadford UniversityLibrary Assessment ConferenceUniversity of WashingtonAugust 3-7, 2008
Introduction
Me, the 15% Assessment Librarian
Radford University McConnell Library
Background
Changes to Library as Place continuous since 2002
Examples Coffee shop Designated quiet areas Group study areas “No cell phone” zones New furniture Improved signage
Goal of this study
Are these physical changes reflected in our LibQUAL+® survey results?
Study design
Data types: Score (rankings) & comments Data source: 2002, 2005, 2006, 2008
LibQUAL+® surveys Respondent groups:
Faculty Graduate students Undergraduates
Level of analysis: “Library as Place” dimension
Methodology
Meta-analysis A data reduction method used to
combine results across studies using
Effect size meta-analysis Statistical procedure that converts
incomparable metrics into a common effect size metric.
Data analysis
Organize the data & respondent types into
comparison groupsFaculty: 2006 v. 2002Grad students: 2005 v. 2002, 2008 v.
2005Undergraduates: 2005 v. 2002, 2008 v.
2005All groups: 2005/6 v. 2002, 2008 v. 2005/6
Data analysis (con’t)
Determine effect size for each comparison group using mean adequacy gap (score data) odds ratio (positive: negative
comments) Determine final weighted average
effect size d for each respondent group for 2002-2008
Results reported as d, .95 CI, and BESD
How did we measure success?(aka, Practical significance)
Practical significance
SuccessLevel Significanc
e
Action criteria Action
d BESD
Complete
Userssatisfied
0.31 or
more
15.1% or
moreCelebrate!
PartialUsers
somewhatsatisfied
0.3 to -0.3
15% to -15%
Look for improvement
s;implementchanges
Un-success
ful
Usersunsatisfie
d
-0.31 or
less
-15% or less
High Priority review/chang
e
What did we expect to find?
In the literature…
Nothing found about specifically using LibQUAL+® to measure satisfaction with changes in Library as Place
All reports found about measuring change due to drastic overhaul of building or new construction.
Conclusion…
If dramatic changes to the Library as Place yield dramatic, immediate increases in user satisfaction, then incremental changes will yield modest gains in user satisfaction over time.
What did we actually find?
Response rate & representativeness Response rate (average of 2002-2008)
Faculty: 15.7% Grad students: 22.4% Undergraduates: 26.4%
Representativeness (average of 2002-2008) Faculty: 8.2% Grad students: 7.4% Undergraduates: -13.1%
So what?(Practical significance again)
Practical significance
Partial success (Users somewhat satisfied) Action Criteria: d + 0.30 (or BESD + 15%)
Faculty (0.01 or 0.6%) Grad students (0.19 or 9.2%) Undergrads (0.08 or 4.1%)
Action Looked for improvements (e.g., examined
“Suggestion” coded comments, conducted focus groups)
Implemented changes (see next slide…)
Review/change
Improvements since 2008 LibQUAL+ survey Created Front Desk from Media Services
and Circ desks Reorganized quiet study floor Reduced noise level on quiet study floor
More improvements planned New furniture for the lobby Walls to be painted Reference area reorganization
Conclusion
Effective Method is statistically defensible Told us what we did (and did not) want to hear
Practical Metrics have practical meaning at the local
level Provide criteria for actionable results
Sustainable Commitment to assessment by library
administration Funded by university
Questions?
Contact information
Eric AckermannReference/Instruction and Assessment
Librarian [email protected] 540-831-5488 Box 6881, McConnell Library, Radford
University, Radford, VA 24142