utilizing opac search logs and google analytics assessing opac effectiveness and user search...
TRANSCRIPT
Utilizing OPAC Search Logs and Google Analytics
Assessing OPAC Effectiveness and User Search Behavior
VALE Users'/NJLA CUS/NJ ACRL Conference
January 9, 2009
Rutgers University
Jia Mi Cathy Weng
Presenters
Jia MiElectronic Resources/Serials LibrarianThe College of New Jersey
Cathy WengHead of CatalogingThe College of New Jersey
2
The College of New Jersey (TCNJ)
Located in Ewing, NJ 6000 students Over half a million volumes in the Library’s
collection Ex Libris Voyager System in use since
2000 Activated Google Analytics in summer
20083
OPAC Search Transaction
OPAC search transaction -- “Consists of a question or query by the user followed by an answer or response from the system” (*)
Transaction log analysis – “Detailed and systematic examination of each search command or query by a user and the following database result output by the OPAC.”(*)
*Blecic, et al, 1998 4
Sample Search Transaction Log page
5
What is Google Analytics?
“Google Analytics (abbreviated GA) is a free service offered by Google that generates detailed statistics about the visitors to a website. ”
GA can track visitors from all referrers, usage, entry points, searches, etc. Reports can be viewed, emailed and exported.
This free version is limited to 5 million pageviews a month
6
Why Use Google Analytics?
Find out… How users get to our site Why they use our site: Which pages are the
most popular How they navigate the site How long they stay at the site What are their patterns of behavior Whether the web site effectively supports the
institution’s goals and mission regarding provision of access and services
7
Google Analytics Dashboard
8
Data Collection Voyager transaction log
Two OPAC transaction logs were collected: April 2007 and November 2008
Comparisons made in many aspects Library website was redesigned in August 2007;
resulted in different search patterns
Google Analytics November 2008 data collected Data from September to November 2008 also
examined and compared
9
Voyager Transaction Log Can assess
Overall statistics of search types usedQuery traffic, both on- and off-campus How searches were issued and refinedQuery trends
Cannot assessQueries successful or notQueries made by librarians or other users
10
When Analyzing the Log Many unanswered questions
Many queries do not have session IDsHow “Simple Search” was formed?Many queries do not have “search type”
Some educated guesses Queries from redirect (i.e., user clicked
hyperlinked field) Comprehensive analysis not possible
Focus on a few areas only
11
Study Focus
ComparisonQuery statisticsDefault searches Usage of Advanced SearchUsage of Subject search
Other findings related to user search skills and OPAC effectiveness
12
Categories Dates Queries PercentageTotal
Searches
4/2007 53,98511/2008 88,649
On-campus queries
(159.91…)
4/2007 39,997 74%11/2008 25,591 29%
Queries from local host
(127.0.0.1)
4/2007 6,004 11%11/2008 56,055 63%
Off-campus queries
4/2007 7,902 15%11/2008 7,003 8%
Query Statistics
13
April 2007 November 2008
Title Begins with
(Default)
Keyword – Relevance
Title Begins with
Keyword – Relevance (Default)
15,910 5,521 5,158 17,698
29.4% 10% 6% 20%
Title Phrase vs. Keyword Search
14
Subject Searches
Browse search: log does not show result hits
Heading (phrase) search: difficult to use as users rarely know the established terms
Search logs revealed two types: user-issued searches (Subject Heading Browse) and record redirect (Subject Browse)
15
Bibliographic Record Display
16
Subject Browse Screen
17
April 2007 November 2008
4511 (8% of total searches)
1734 (2% of total searches)
Queries typed in by user
Queries via system redirect
Queries typed in by user
Queries via system redirect
2975 1536 674 1060
65% 35% 39% 61%
Subject Queries
18
Advanced Search (Guided Search)
April 2007: 3,331 (6% of total searches) searches issued. (3057 <gui>+ 274 <bui>)
Nov. 2008: 1,761 (2% of total searches) searches issued. (1253 <adv> + 508 <bui>)
Data suggest that library users use Advanced Search interface less now than they did 18 months ago
19
April 2007 November 20083,331 1,761
6% 2%
Advanced Searches
• Statistics do not include searches issued by local host (127.0.0.1) which has quite a few searches using Advanced Search interface.
20
Call Number Browse
By clicking the call number hyperlinked field, user will receive a call number browse screen of nearby items
Approximately 500 queries in Nov. 2008 and 600 queries in April 2007 using call number browse hyperlink redirect
It is assumed that users clicked call number hoping to get information about item shelf location in the Library (not nearby items)
21
Other Findings
Related to system retrieving When doing title phrase search using Advanced
Search interface, it only searches main title, not subtitle.
E.g., “Rich and the super rich, a study in the power of money today”.
Main title and subtitle. User would not know the difference. System retrieval mechanism issue.
22
Other Findings
Related to OPAC DisplayWhen doing keyword search using Basic
Search interface, search results are arranged by relevance.
When doing keyword search using Advanced Search interface, search results are arranged by publication date.
More relevant results might not be in the first results display page.
23
24
25
Next page 26
Other Findings
Keyword in Basic vs. Advanced Search interface In Basic Search words like “the,” and “and”
are acceptable In Advanced Search, typing in “the” or “and”
will retrieve 0 hitsSystem retrieval mechanism issue
27
28
29
Other Findings
In Basic Search, using “Author (last, first)” results in a browse search and the x-ref in authority record works.
In Advanced search using author search key results in a heading search (instead of browse search) and the x-ref in authority record does not take effect.
30
Author Query in Basic Search
31
See Reference OPAC Display
32
Author Query in Advanced Search
33
In Advanced Search, need to use established form of name for records to be retrieved.
34
Other Findings
Advanced Search interface Using “as a phrase” as
default might have better chance retrieving 0 records.
Should the default be changed to “All of these”?
35Advanced Search Interface
What Can Be Done
Encourage use of author browse in Basic Search interface to take advantage of x-ref in authority file.
Provide information on search help page. Investigate making author browse possible
in Advance Search interface.
36
What Can Be Done
Look into system indexing structure more closely and their connection to OPAC searchesE.g., Title search in Advanced Search interface
to change to TALL (whole title), not just 245A (main title)
E.g. , Enhance Keyword search in Advanced Search to accept “stop words” (the, and, etc.)
37
What Can Be Done
Investigate the possibility of enhancing query results display from Advanced keyword search to be ranked by relevance.
38
What Can Be Done
OPAC search results
OPAC transaction log
• Enhance local search skills.
39
Google Analytics Dashboard: Comparison
40
Visitors Overview
41
Geographic Region: International
42
Geographic Region: United States
43
Geographic Region: New Jersey
44
Site Visiting Time
45
Site Usage Trend
46
Where Did Users Come From?
47
Direct Visits to TCNJ Library Catalog
48
What Content are Users Looking at?
49
/cgi-bin/Pwebrecon.cgi?DB=local&PAGE=First
This page was viewed 19 times via 1 keywords
Pageview: 2816
50
51
Pages with the Most Traffic (Homepage)
52
Top traffic sources (Homepage)
53
54
Most Popular Databases
55
Where is the Traffic Coming From?
56
57
58
InterLibrary Loan vs. “Request Button”
59
Total 883 requests:
705 from ILL page178 from the Library
Catalog using Request Button
Book Request
60
RSS Feed and Persistent Link?
61
Do Users Use “Limit to”
62
Do Users Use “New Books” page?
63
Google Analytics Pros and Cons
• Pros– User-friendly– Free– Accurate
• Cons– Users with high security settings can only be partially
tracked– Site visits browsed from mobile phones can not be tracked– Can slow down system operation (longer response time)
64
Using Google Analytics Data
Help us to Better understand
Who web users are Where they come from What they do on the site How use of site may be facilitated or enhanced
Improve web site designProvide more relevant materials for users
65
Combining Search Log Data and Google Analytics Data
Trace traffic to TCNJ Catalog from WorldCat. Detect usage of questionable sites.
66
Queries from WorldCat
Search log recorded searches by “ocm#” (OCLC control number)
“ocm#” query is the default query type set up locally on WorldCat.
Approximately 150 “ocm#” searches in November 2008.
Confirming traffic from WorldCat to TCNJ OPAC.
67
WorldCat link to TCNJ OPAC
68
Referral from Worldcat
69
Simple Search and Builder Search
Simple search and Builder search existed in TCNJ’s old OPAC interface equivalent to “Basic Search” and “Advanced Search” respectively.
Old search types recorded in OPAC transaction logs, suggesting the defunct interface was still being used off campus.
70
TCNJ Old OPAC Interface
71
Builder Search
A total of 274 searches in April 2007 and 508 searches in November 2008 recorded on search log are from Builder search.
Majority of them are ISBN searches (coming from search engines?)
72
Google Analytics Data
/cgi-bin/Pwebrecon.cgi
Pageview: 3791
73
What’s Next?
Work on enhancing Voyager indexing structure and OPAC search mechanism and display.
Continue to find more answers to questions.
More analysis to be completed.
74
