assessment-based strategies for building connections with academic departments 2008 library...
TRANSCRIPT
Assessment-based strategies for
building connections with academic departments
2008 Library Assessment Conference
August 5, 2008Seattle, WA
Goals (and evaluation criteria?)
Recognize the evaluation capacity building benefits of process evaluation
Gain insight into ways that assessment and outreach activities can be linked
Identify approaches that might be applied in your library to inform and engage stakeholders
Presentation outline
Background, approach, context
Case study illustration - Academic department reports
Findings
Implications
Context and background
Perkins Library & DukeUniversity re-accreditation
Library strategic plan emphasis on assessment
Center for Instructional TechnologyHigh visibility and reputation for leadership in assessment
Library public services and CIT shared focus and challenges for outreach to academic departments, engagement
Utilization-focused evaluation
Engaging stakeholders in the entire evaluation process from design to implementation of recommendations
Prioritize issues of greatest importance to those in a position to directly make use of findingsReduce org. culture barriers that inhibit use of results by increasing transparency, empowering stakeholders
Process Use benefits
Process use
‘Ways in which being engaged in the processes of evaluation can be useful quite apart from the findings that may emerge from these processes’ (Patton, 1997)
Includes -
Organizational development, specifically evaluation capacity building
Increased capacity to make use of evaluation findings, know how to use evaluation information
Patton, 2004, “On Evaluation Use: Evaluative Thinking and Process Use”
Evaluation capacity building
Adapted from Cousins, Goh, Clark & Lee (2004)
Assessment & evaluation activities
Direct consequences• Knowledge• Use of findings
Process useEvaluation
capacity• Skills
• Eval knowledge & logic
Organizational learning capacity
• culture of experimentation
Case Study Academic department reporting
CIT department report experience
Internal evangelism for stakeholder-focused assessment
General heightened interest in assessment and effective use of data among library leadership
Institutional context
Overview of the project
Data audit
Buy-in from leadership, key internal constituencies
Prototypes
Internal stakeholder review (multiple iterations)
Distribution and outreach
Assessment Sample reports, cover letter: http://ww.duke.edu/~ybelang/lac08
Types of data included
Service descriptions & contact info
Courses receiving custom support (instruction sessions, web guides, e-reserves, digitization, etc.)
Funded/supported faculty projects
Faculty inquiries, consultations (anonymous, CIT only)
CMS use (Blackboard)Sample reports, cover letter: http://ww.duke.edu/~ybelang/lac08
Gory technical details
Business Intelligence software
to group, filter, summarize and funnel static and dynamic
Content into formatted template(Crystal Reports)
Live server data connections
• ODBC to MySQL database
Offline data sources
• Excel spreadsheets
• XML files extracted from Aleph
PDF or RTF reports, cover letters
Back-end view of Crystal Reports
Static content
Subreports pull from multiple dynamic live databases and clean warehoused sources
Key considerations
Who’s your audience?A department’s chairs point of view
What kind of reaction do you hope for? Fear?
What action do you want the reader to take?
Best and worst case scenarios, political considerations
Major hurdles / milestones
Pervasive unit of reporting
Finding and implementing the right software, license $
Figuring out what data of value exist, and who has it
Data clean-up, reformatting
Getting buy-in from multiple stakeholder groups with different perspectives
Managing those you don’t manage
Lessons learned
Get input from different kinds of stakeholders early in the project
Patience and persistence
Look for opportunities to demonstrate value
“Good, quick, cheap – choose 2”
Works Cited
Cousins, J. B., Goh, S., Clark, S., & Lee, L. (2004). Integrating evaluative inquiry into the organizational culture: A review and synthesis of the knowledge. Canadian Journal of Program Evaluation, 19(2), 99-141.
Patton, M. Q. (2004). "On evaluation use: Evaluative thinking and process use." The Evaluation Exchange IX(4).
Patton, M. Q. 1997. Utilization-focused evaluation: The new century text (3rd ed.). Thousand Oaks, CA: Sage.
Contact information
Yvonne BelangerHead, Program Evaluation
Academic Technology & Instructional Services
Perkins Library, Duke University