Lynn Silipigni Connaway, Ph.D.Senior Research ScientistOCLCChair of ExcellenceDepartmento de Biblioteconomía y Documentación Universidad Carlos III de Madrid@[email protected]
May 15, 2014
Exploring Shifting Changes in User Engagement
Then & Now
• Then: The user built workflow around the library
• Now: The library must build its services around user workflow
• Then: Resources scarce, attention abundant
• Now: Attention scarce, resources abundant
(Dempsey, 2008)
The library? What’s that?
• Website hard to navigate• Inconvenient
• Limited hours• Distance to library• Physical materials
• Don’t think electronic resources are library resources• Associate with books
(Connaway & Dickey, 2010)
“…there was a book in the library that I just did not want to leave my house to go to. It is a 50-minute drive, I didn’t want to do that, but I was writing my paper and so I used Google books instead and really they only had a section of the book available but that was the section I used.”
(Digital Visitors and Residents, USG4, Female, Age 23, Latin American Studies)
Current Environment
• Challenges• Budget cuts• High retirement rates• Hiring freezes
• Opportunity• Best value for most use• Understand how, why, &
under what circumstances individuals use systems & services
Articulate the Value of Libraries
Why Assessment?
• Answers questions:• What do users/stakeholders want & need?• How can services/programs better meet needs?• Is what we do working?• Could we do better?• What are problem areas?
• Traditional stats don’t tell whole story
Importance of Assessment
“Librarians are increasingly called upon to document and articulate the value of academic and research libraries and their contribution to institutional mission and goals.”
(ACRL Value of Academic Libraries, 2010, p. 6)
Outcomes Assessment Basics
• Outcomes: “The ways in which library users are changed as a result of their contact with the library’s resources and programs” (ALA, 1998).
• “Libraries cannot demonstrate institutional value to maximum effect until they define outcomes of institutional relevance and then measure the degree to which they attain them” (Kaufman & Watstein, 2008, p. 227).
Interpreting
Analyzing
Collecting
Assessment Defined
Process of…• Defining• Selecting• Designing• Collecting• Analyzing• Interpreting• Using information to increase
service/program effectiveness
Formal vs. Informal Assessment
• Formal Assessment • Data driven• Evidence-based• Accepted methods• Recognized as rigorous
• Informal Assessment • Anecdotes & casual
observation• Used to be norm• No longer acceptable
The Assessment Process
Identify purpose
Identify team
Choose model/approach/method
Training and Planning
Why?
Who?
How?
What We Know About Assessment
• Ongoing process to understand & improve service
• Librarians are busy with day-to-day work & assessment can become another burden
• Can build on what has already done or is known
Library Factors for Assessment
• Instruction: Games, single/multiple session, course embedded, tutorials
• Reference• Physical space• Discovery: Institutional web, resource guides• Collections• Personnel
Variety of Tools/Methods
• Survey• Interviews• Focus group(s)• Observation• Pre/Post test• Rubric• Student portfolio
• Research paper/project• Other class assignment• Test scores• GPA• Degree completion rate• Retention rate
Principles for Applying Outcomes Assessment
• Center on users• Assess changes in service/resources use
• Relate to inputs - identify “best practices”• Use variety of methods to corroborate conclusions
• Choose small number of outcomes• Need not address every aspect of service
• Adopt continuous process
How Individuals Work
• Convenience• Value human resources• Contextually based
rational decisions• Situational needs
determine search• Satisfice
(Connaway & Radford, 2011)
“I don't know how to access computer library service. When I need to look something [sic] up I use google.”
(Seeking Synchronicity, NOS-61939, Male Age 46-55)
VRS & SQA User Interviews:Finding What is Needed
6%, 3
36%, 2058%, 32
1 Rarely2345 Very often
When you search, how often can you usually find what you need?
VRS Potential User Online Survey: Prefer Face-to-Face Interactions
49%, 60
12%, 14
27%, 33
12%, 15
"I most enjoy using"
FtFPhoneEmailText Messaging
Digital Visitors & Residents: Means of Contact
Emerging (n=43)
Establishing (n=10)
Embedding (n=10)
Experiencing (n=10)
30%
40%
50%
60%
70%
80%
90%
100%
60%, 26
100%, 10 100%, 10 100%, 10
84%, 36
80%, 8
70%, 7
50%, 5
77%, 33
90%, 9
70%, 7
70%, 7
60%, 26 60%, 6
40%, 4
70%, 7Email
Texting
Phone Calls
Face-to-Face
Information-Seeking Behavior
• Power browsing• Scan small chunks of
information• View first few pages• No real reading
• Squirreling• Short basic searches• Download content for
later use • Differ with discipline (Research Information Network, 2006)
(Consortium of University Research Libraries, and Research Information Network, 2007)
(Connaway & Dickey, 2010
Skills for Finding & Using Information
• Students• Determine credibility by:
• Common sense (77%)• Cross-checking (69%)• Reputation of
company/organization (67%)• Credible recommendations
(48%)• Researchers
• Self-taught in discovery services• No formal training (62%)
• Doctoral students learn from dissertation professor
(Research Information Network, 2006) (De Rosa, 2010)
Digital Visitors & Residents: Authority, Time, Convenience
Emerging (n=43)
Establishing (n=10)
Embedding (n=10)
Experiencing (n=10)
0%
20%
40%
60%
80%
100%
120%
79%, 34
60%, 6
90%, 9
70%, 7
40%, 17 40%, 450%, 5
50%, 5
91%, 39100%, 10 100%, 10
90%, 9 Authority, Legitimacy
Available Time
Convenience, Ease of Use, Accessibility
Digital Visitors & Residents: Authority, Time, Convenience
Authority, Legitimacy Available Time Convenience, Ease of Use, Accessibility
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
81%25
39%12
94%29
75%9
42%5
83%10
Emerging 1st Group (n=31)
Emerging 2nd Group(n=12)
Tools Used: Students
• Undergraduate Students• Google, Wikipedia• Also use library website & e-
journals• Human resources
• Other students/classmates• Family & relatives • Friends
• Graduate students• Professors, advisors, mentors• Electronic databases (Connaway & Dickey, 2010)
(De Rosa, 2010)
27
“This year I don’t think I have ever picked up a book out of the library to do any research, all I have used is my computer.”
(Digital Visitors and Residents, USU1, Female, Age 19, Undeclared)
VRS & SQA User Interviews:Searching the Web
16%, 9
26%, 14
20%, 11
38%, 21
Occasionally1-3 searches per day4-6 searches per day7-10 searches per dayMore than 10 searches per day
How often do you search the web?
Tools Used: Researchers
• Online resources• 99.5% use journals as primary
resource• Google, Web of Science,
PubMed, Science Direct, JSTOR• Human resources
• 90% mention expertise of individuals as important resource
• Coworkers• Colleagues• Other professionals (Research Information Network, 2006)
(Connaway & Dickey, 2010)
Journals & Databases
• Journals• Access more important than discovery• Want full text, online versions• Expect seamless Discovery-to-Delivery• Backfiles difficult to access• Content often discovered through Google• Visit only a few minutes
• Databases• Electronic databases not perceived as
library sources• Frustration locating & accessing full-text
copies
(Research Information Network, 2006)
Digital Visitors & Residents:Digital Sources
Emerging (n=43)
Establishing (n=10)
Embedding (n=10)
Experiencing (n=10)
10%
20%
30%
40%
50%
60%
70%
80%
90%
19%, 8
50%, 5
80%, 8 80%, 8
33%, 1430%, 3
70%, 7
40%, 4
28%, 12
40%, 4
50%, 5
30%, 3
Databases
University databases
University websites
Digital Visitors & Residents:Digital Sources
Databases University databases University websites0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
30%20
41%27
26%1720%
21 17%18
32%33
UKUS
Examples of Outcomes
User matches information need to information resources
User can find appropriate resources
User effectively searches online catalog & retrieves relevant resources
User can organize an effective search strategy
What can we change?
• Improved OPACs• Community as content
• Full text, online accessible• Seamless discovery to
delivery• Access more important
than discovery• Mobile access
• Presence in social networks• Facebook• Twitter
“Because I mean the thing that annoys me most is when these things are online, unlike library catalogues that’s supposed to be a really good way for looking for books but usually they are so bad that you are sort of stuck between the two worlds...”
(Digital Visitors and Residents, UKG5, Female, Age 25, Early Modern History)
The Simple Search Bar
Community is Content
• Social networks formed around social objects• Music, photos, videos, links• Reviewing• Tagging• Commenting• Rating
• Refines interaction with resources
(Dempsey, 2012)
What can we do?
• Advertise resources, brand, & value
• Provide search help at time of need
• Chat & IM• Mobile technology
• Design all of our systems with users in mind• Familiar formats
• Model services on popular services
Amazon.com
Westerville Public Library
Familiar Formats
Continually evaluate services
Ask those who use and do not use them
infoKit
What is it?• Contains advice on evaluating digital/online services
within the broader context of traditional services.
Why did we create it?• To understand the contexts surrounding individual
engagement with digital resources, spaces and tools.
Who will use it?• Librarians and information technology staff
(White, Connaway, Lanclos, Hood, & Vass, 2014)
©2014 OCLC. This work is licensed under a Creative Commons Attribution 3.0 Unported License. Suggested attribution: “This work uses content from [presentation title] © OCLC, used under a Creative Commons Attribution license: http://creativecommons.org/licenses/by/3.0/”
Thank You!
Lynn Silipigni Connaway, [email protected]@LynnConnaway
ReferencesAssociation of College and Research Libraries. (2010). Value of academic libraries: A comprehensive
research review and report. Chicago: Association of College and Research Libraries.
Bertot, J. C., Berube, K., Devereaux, P., Dhakal, K., Powers, S., & Ray, J. (2012). Assessing the usability of WorldCat Local: Findings and considerations. The Library Quarterly, 82(2), 207-221.
Connaway, L. S., & Dickey, T. J. (2010). Digital information seekers: Report of findings from selected OCLC, RIN, and JISC user behavior projects. http://www.jisc.ac.uk/media/documents/publications/reports/2010/digitalinformationseekerreport.pdf
Connaway, L. S., & Dickey, T. J. (2010). Towards a profile of the researcher of today: What can we learn from JISC projects? Common themes identified in an analysis of JISC Virtual Research Environment and Digital Repository Projects. http://ie-repository.jisc.ac.uk/418/2/VirtualScholar_themesFromProjects_revised.pdf
Connaway, L. S., & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf
Consortium of University Research Libraries, and Research Information Network. (2007). Researchers’ use of academic libraries and their services: A report. London: Research Information Network and Consortium of University Research Libraries (CURL).
Cunningham, S. J., & Connaway, L. S. (1996). Information searching preferences and practices of computer science researchers. In J. Grundy (Ed.), Proceedings: Sixth Australian conference on computer-human interaction, November 24-27, 1996, Hamilton, New Zealand (pp. 294-299). Los Alamitos, CA: IEEE Computer Society Press.
Dempsey, L. (2008). Always on: Libraries in a world of permanent connectivity. First Monday, 14(1). Retrieved from http://www.firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2291/207
ReferencesDempsey, L. (2013, January 23). The inside out library: Scale, learning, engagement. Presented at
Hacettepe University, Beytepe, Ankara (Turkey). Dempsey, L. (2012). Thirteen ways of looking at libraries, discovery, and the catalog: Scale, workflow,
attention. EDUCAUSE Review Online. Retrieved from http://www.educause.edu/ero/article/thirteen-ways-looking-libraries-discovery-and-catalog-scale-workflow-attention
De Rosa, C. (2010). Perceptions of libraries: A report to the OCLC membership. Dublin, OH: OCLC Online Computer Library Center.
Dervin, B., Connaway, L. S., & Prabha, C. (2003-2006). Sense-making the information confluence: The whys and hows of college and university user satisficing of information needs. Funded by the Institute of Museum and Library Services (IMLS). Retrieved from http://imlsosuoclcproject.jcomm.ohio-state.edu/
De Santis, N. (2012, January 6). On Facebook, librarian brings 2 students from the early 1900s to life. Chronicle of Higher Education. Retrieved from http://chronicle.com/blogs/wiredcampus/on-facebook-librarian-brings-two-students-from-the-early-1900s-to-life/34845
Janes, J. (1999). On research survey construction. Library Hi Tech, 17(3), 321-325. Kaufman, P., & Watstein, S. B. (2008). Library value (Return on Investment, ROI) and the challenge
of placing a value on public services. Reference Services Review, 36(3), 226-231. Norris, D., Baer, L., & Offerman, M. (2009, September). A national agenda for action analytics.
Presented at the National Symposium on Action Analytics, St. Paul, Minnesota, http://lindabaer.efoliomn.com/uploads/settinganationalagendaforactionanalytic s101509.pdf
Priestner, A., & Tilley, E. (2012). Personalising library services in higher education: The boutique approach. Farnham, Surrey, England: Ashgate.
ReferencesRadford, M. L., & Connaway, L. S. (2005-2007). Seeking Synchronicity: Evaluating virtual reference
services from user, non-user, and librarian perspectives. Funded by the Institute for Museums and Library Services (IMLS). Retrieved from http://www.oclc.org/research/activities/synchronicity/default.htm
Radford, M. L., Connaway, L. S., & Shah, C. (2011-2013). Cyber Synergy: Seeking sustainability through collaboration between virtual reference and social Q&A sites. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC. Retrieved from http://www.oclc.org/research/activities/synergy/default.htm
Research Information Network. (2006). Researchers and discovery services: Behaviour, perceptions and needs. London: Research Information Network.
Wasserman, S. (2012, June 18). The Amazon effect. The Nation. Retrieved from http://www.thenation.com/article/168125/amazon-effect
White, D., & Connaway, L. S. (2011-2014). Visitors and residents: What motivates engagement with the digital information environment. Funded by JISC, OCLC, and Oxford University. Retrieved from http://www.oclc.org/research/activities/vandr/
White, D., Connaway, L. S., Lanclos, D., Hood, E. M., & Vass, C. (2014). Evaluating digital services: A Visitors and Residents approach. Retrieved from http://www.jiscinfonet.ac.uk/infokits/evaluating-services/
Wong, W., Stelmaszewska, H., Bhimani, N., Barn, S., & Barn, B. (2009). User behaviour in resource discovery: Final report. Retrieved from http://www.jisc.ac.uk/whatwedo/programmes/inf11/userbehaviourbusandecon.aspx
Zickuhr, K., Rainie, L., & Purcell, K. (2013). Library services in the digital age. Washington, DC: Pew Research Center’s Internet & American Life Project.