meaningful measurement how do we articulate the value of public libraries in a contemporary context?...
TRANSCRIPT
Meaningful Measurement
How do we articulate the value of public libraries in a contemporary context?
SWITCH Conference 2015
Jackie Bailey, Principal BYP Group
http://bypgroup.com
(c) BYP Group 2015 http://bypgroup.com
Exercise:
Jot down words or doodle pictures in answer to the following question:
When I think about ‘evaluation’ and libraries, I feel….
What is evaluation?
• What, if anything, has changed because of us?
• Who would miss us if we were gone?
• Why would they they miss us?
• What value do we create for society, which no one else can (or at least, not as efficiently as we do)?
(c) BYP Group 2015 http://bypgroup.com
Society and culture
Library users
You and library staff
Why evaluate?
(c) BYP Group 2015 http://bypgroup.com
Improve•Use your findings to continuously improve•Challenge your own or others’ assumptions•Discover unexpected outcomes
Communicate
•Tell people about the value you create in ways they can understand
Advocate•Use your findings to advocate for what libraries do, which no one else can
The unexamined life is not worth living - Socrates
Formative evaluation
•Establish the baseline•Identify a ‘theory of change,’ linked to the research•Identify goals and indicators or success
Process evaluation
•Look at how you work e.g. partnerships, operations
Summative evaluation
•Measure the impact of your activity over the short, medium and long term, and on various people or groups
Stages of evaluation
(c) BYP Group 2015 http://bypgroup.com
What do I evaluate?
(c) BYP Group 2015 http://bypgroup.com
• Who is this evaluation for?
• How will this evaluation be used?
• What do I need to know?
• What is a nice to have, and what is a must have?
• What can actually be measured, given my time and resource constraints?
Evaluation is…
for us (e.g. program improvement)
Process evaluation
Impact evaluation
for funders (e.g. lobbying for funding)
Impact evaluation
Return on investment
Funders’ objectives
For example…
What are ‘impacts’ and how (on earth) do I measure them?
(c) BYP Group 2015 http://bypgroup.com
Warning: these ‘impacts’ are off the top of my head and should just be
read as an example!
What are the impacts?•Access to resources they would not otherwise easily afford•Opportunity for social bonding•Opportunity for self-expression and identity•Sense of community connectedness, belonging•Personal capacity development as a result of using library resources
How do I measure them?
•Number of users and frequency of usage•User demographics, including access to like resources•Survey of user experience, benefits, suggestions, uses, self-assessment of the impact of the library on them•Survey of user capacity development (preferably referring to benchmark data)•Gather stories of ‘change’ amongst library users – e.g. how people’s lives are different BECAUSE of the library•Creative tools e.g. before and after pictures, ‘if there was no library,’ vox pops, chalkboards
Library Users
What are ‘impacts’ and how (on earth) do I measure them?
(c) BYP Group 2015 http://bypgroup.com
Warning: these ‘impacts’ are off the top of my head and should just be
read as an example!
What are the impacts?
•Sense of community identity, public spaces, civil society•‘Institutional value’ of the existence of libraries (John Holden)•Sense of having resources if needed
How do I measure them?
•Survey of wider community perceptions and attitudes towards libraries (e.g. through local council surveys)•Contingent valuation survey
Community members
What are ‘impacts’ and how (on earth) do I measure them?
(c) BYP Group 2015 http://bypgroup.com
Warning: these ‘impacts’ are off the top of my head and should just be
read as an example!
What are the impacts?•Social and cultural capital•Economic outcomes e.g. support for small businesses, further education and employment
How do I measure them?•Theoretical assertions based on your findings, and drawing sound conclusions about your contribution to the determinants of social and cultural capital as outlined in the research literature•Study of business, education and job seeker usage
Society, economy and culture
What are ‘impacts’ and how (on earth) do I measure them?
(c) BYP Group 2015 http://bypgroup.com
Warning: these ‘impacts’ are off the top of my head and should just be
read as an example!
What are the impacts?•Contribution to the ongoing development of library culture and information systems•Capacity development of staff
How do I measure them?
•Staff surveys•Internal team reflection and discussion•Conversations with respected peers or mentors•Awards
You and the art of ‘librarianism’
Possible methods
(c) BYP Group 2015 http://bypgroup.com
User impact surveys
• Demographics of users• Level of access to like resources• Suggestions and needs• Qualitative impacts e.g. sense of connection, social bonding, confidence, self-efficacy
Stories of change
• Vox pops• Writing exercise• Pictures of before and after, or the community with/without a library• Chalkboard words• Interviews and group discussions
Wider community value survey
• Contingent valuation• Public perception of libraries
Return on investment studies
• Social ROI• Economic ROI
How some organisations use their evaluations
Penrith City Council Booklet & DVD Awards – Australian National
Disability Award for Excellence in Social Participation; Arts and Health Award
Arts Centre Melbourne: Published report and summary
report (flipbook) Online video for donors
MCA Funding reports Peer to peer learning
Improve, Communicate, Advocate
(c) BYP Group 2015 http://bypgroup.com
(c) BYP Group 2015 http://bypgroup.com
Exercise:
Jot down words or doodle pictures in answer to the following question:
Now, when I think about ‘evaluation’ and libraries, I feel….
(c) BYP Group 2015 http://bypgroup.com
What, if anything, changed?
Discussion
Jackie Bailey, PrincipalBYP Group
http://bypgroup.comM: 0428 576 372