s abram nfais keynote feb 25 2013

Upping Our Game:

Leading on

Transformational

Analytics &

Getting Off the Hits

Train

Unlocking New Value from ContentStephen Abram, [email protected]

http://www.dysartjones.com/

Are you on the ‘HITS’ train?

BIGDATA

QUALITATIVE INFORMATION

QUANTITATIVE DATA

and

STATISTICS

MEASUREMENTS

and

What do we do when our buyers are asking for data that does not align with

their goals?

Have Journal Prices Really Increased Much in the Digital Age? (Scholarly Kitchen blog) http://bit.ly/11b3hP2

http://bit.ly/11b3hP2



Excellent Metaphor

“What if the only measurement of energy costs you followed was the price of oil, while everyone was shifting to cheaper and more efficient alternatives? And what if you completely ignored the fact that everything around you was using more and more power — your lights, your phone, your car, your heat, your media center? You might come to believe that energy is getting more expensive, when actually, it’s price is rising relatively slowly while your usage is what is skyrocketing.

The same thing might be happening with print journal prices and digital journal licenses…

Good Questions

What if prices of the predominant journal form have actually been falling?

What if we’ve been measuring the wrong things, or measuring insufficiently?

And what if the growth in expenses are not the result of price increases but a result of the growth in science?”

The Real Digital Story

Print subscription prices are a misleading and inaccurate method for tracking library serials spending

“. . . libraries’ spending on periodicals has increased three-fold while their collections have tripled in size”

“Spending three times as much to get three times as much tells a very different story from the “price increases” story. . . .”

Published article output has grown 3.5% to 4% per year since 1990

Growth in research spending has been increasing by 3-4% per year

In the US, spending on scientific research has more than doubled since 1990 (from $150.2 billion to $400.5 billion in 2010, in current dollars)

Numbers versus ROI

“In the midst of all this growth, prices have risen modestly. Gantz notes that while the economy in the US from 1990 to 2010 grew at a compounded rate of 66.8% due to inflation, the effective price of an average journal is only 9% higher over the same time period. In the UK, prices have actually gone down by 11% since 2004.”

“Price increases have been caused by more science, more papers, and more journals, not by price increases in licenses. In fact, per-journal prices seem to have peaked around 2000, and steadily declined from there, as shown by the black line in the chart below.”

http://scholarlykitchen.files.wordpress.com/2013/01/psp-chart.jpg

And this is all means?

We’re playing a fool’s game when we play the raw statistics game.

Are you locked into library financial mindsets?

What about value and impact?

Or shall we stick with this?

Grocery Stores

Cookbooks, Chefs . . .

http://cache.gawker.com/assets/images/4/2010/02/walee.jpg

What do we count and share?

Titles

Clicks

Downloads

Sessions

Session length

COUNTER, (Counting Online Usage of Networked Electronic Resources)

SUSHI, Standardized Usage Statistics Harvesting Initiative

etc.

Or should we measure?

Was there improved customer satisfaction?

Do librarians or types of end users have different values and behaviours?

Did learning happen?

Was there an impact on research or strategic outcomes?

Did the patient live, improve, survive, thrive?

. . .

Algorithms

Search differentiator

Commercial algorithms versus those based on big data

Measuring end user success versus known item retrieval…

“Romeo and Juliet”

Problems with the unmonitored trial Wrong tests Poor sampling Mindset issues

Sharing Learning and Research

Usability versus User Experience

End users versus librarians

Known item retrieval (favourite test) versus immersion research

Lists versus Discovery

Scrolling versus pagination

Devices and browsers and agnosticism

Satisfaction and change

Individual research experience vs. impacts on e-courses, LibGuides, training materials, etc.

Gale Analytics

Focus and Understand on the Whole Experience

Inside Lego™ Pieces

Foresee satisfaction and demographic data

Counter & Sushi data

Database usage (unique user, session, length of session, hits, downloads, etc.)

Google Analytics

Search Samples

ILS Data

Geo-IP data

What We Know (US/Canada)

27% of our users are under 18. 59% are female. 29% are college students. 5% are professors and 6% are teachers. Daily, 35% of our users are there for the very first time! Only 29% found the databases via the library website. 59% found what they were looking for on their first search. 72% trusted our content more than Google. But, 81% still use Google.

37

Statistics, Measurements and Analytics

• Counter & Sushi data are very weak metrics that don’t provide insights into the critical stuff

• Database usage (unique user, session, length of session, hits, downloads, etc.)

• Web and Google Analytics (6,000+ websites)

• Foresee satisfaction and demographic data

• Search Samples (underemphasized at this point.)

• Time of Year Analysis

• ILS Data (from clients &n partnerships)

• Geo-IP data, analytics and mapping.

• Impact studies and sampling.

38

Advocacy Work and GCL Alignment

Michigan Outcomes Study

Impact of In Context portals

Academic Libraries:

Central Michigan University

Grand Valley State University

Public Libraries:

Clinton Macomb Public Library

Howell District Library

Kent District Library

Portage District Library

Michigan Outcomes: How Do Libraries Tell Stories?

Who are our audiences?

• Librarians (several languages management, reference, acquisitions, systems, LMS, etc.)

• Institutional information technology and systems professionals

• eLearning professionals and developers

• Web design professionals

• Library Management team & Chief Librarian

• City or University administration, Provosts

Or End users?

Analytics

What do we need to know?

How do library databases compare with other web experiences and expectations?

Who are our core virtual users?

What are user expectations for satisfaction?

How does library search compare to consumer search like Google?

How do people find and connect with library virtual services?

What should we ‘fix’ as a first priority?

Are end users being successful in their POV?

Are they happy? Will they come back? Tell a friend?

0

7171 42 42 25

34

4035

17

3342

19

39

26 915

30 30

48

41

3033

59

37

30

59

30

48

6562

18 18 17 14 1822 14

1718

7

21 16

1010

Top-Level BenchmarksGale-Cengage Browse Survey

August 01, 2010 - August 31, 2010

Gale Library Databases Compare

Very Well to Other

Web Experiences

Digging Into Satisfaction

3 4 5 9 6 8 7 2 1

High School Student

Univers/ College Student Librar’n

Other, please specify Teacher

Other Profess-

ionalProfess-

or

Middle School Student

Elementary

School Student Overall

Responses: 3,043 2,920 1,570 709 576 576 488 477 148 10,507

29% 28% 15% 7% 5% 5% 5% 5% 1%

Content 70 77 79 69 77 74 67 68 40 73

Look and Feel 64 74 74 64 72 70 61 63 37 68

Navigation 65 71 69 60 69 66 60 63 45 66

Search 61 73 73 60 71 67 59 59 35 66

Site Performance

73 79 77 72 77 76 68 72 52 75

Satisfaction 63 74 72 60 72 66 59 62 35 67

Likelihood to Return

72 82 85 71 82 78 66 69 36 76

Primary Resource

63 73 68 56 70 60 51 60 33 65

Recommend 63 78 83 66 80 72 63 61 36 71

Users willReturn and

RecommendOnce Hooked

0.2 1.2 2.262

72

Content73, 1.1

Look and Feel68, 0.4

Navigation67, 0.9 Search

67, 2.8

Site Performance74, 0.4

Gale-Cengage Browse SurveyPriority Map

October 01, 2009 - October 31, 2009

IMPACT

SC

OR

E

STATUS QUO REQUIRED MAINTAIN OR IMPROVE

MONITOR TOP PRIORITYTOP PRIORITYKey:Position of each bubble indicates its score and impactSize of each bubble also indicates the relative size of impact

HighLow

High

Low

LibrarySearch

Needs toImprove

High School Student

University/ College Student

Librarian

Other, please specify

Teacher

Other Professional

Professor


Elementary School Student

0% 5% 10% 15% 20% 25% 30% 35%

29%

28%

15%

7%

5%

5%

5%

5%

1%

Who uses e-Resources?

The Core User For Library

E-Resourcesis Clear

Yes60%

No20%

Don't know yet20%

Gale-Cengage Browse SurveyDid you find what you were looking for?August 01, 2009 - September 06, 2010

(N: 10486)

Database UsersAre BeingSuccessful

Did not have any dif-ficulty navigating the

site52%

Links did not take me where I expected10%

Other, please specify9%

Would often feel lost, not know where I was8%

Could not navigate back to previous in-formation

8%

Too many links or navigational choices6%

Had technical difficulties (e.g. broken links, error messages)

4%

Links/labels are difficult to understand4%

Gale-Cengage Browse SurveyWhat type of difficulty, if any, did you encounter with the navigation process on this website?

August 01, 2009 - September 06, 2010

(N: 10486)

There are Training and

CommunicationOpportunities

Female59%

Male33%

Prefer not to respond8%

Gale-Cengage Browse SurveyWhat is your gender?


(N: 10484)

More Males UseLibrary

DatabasesThan Usual

18 and under30%

51+23%

36-5019%

19-2511%

26-359%

Prefer not to respond7%

Gale-Cengage Browse SurveyWhat is your age?


(N: 10486)

There is Not A DemographicSwing in Online

Usage

Yes73%

Haven't thought about it18%

No9%

Gale-Cengage Browse SurveyDo you trust the content on this website more than the content you find through web search

engines?August 01, 2009 - September 06, 2010

(N: 10486)

Library UsersTrust Library

DatabasesMore.

School assignment62%

Professional project17%

Personal interest10%


Just browsing5%

Gale-Cengage Browse SurveyWhich best describes the purpose of your research today?


(N: 10486)

School is the Sweet Spot –

But Other UsersAbound Too.

High School Student

University/ College Student

Librarian

Other, please specify

Teacher

Other Professional

Professor


Elementary School Student

0% 5% 10% 15% 20% 25% 30% 35%

29%

28%

15%

7%

5%

5%

5%

5%

1%

Gale-Cengage Browse SurveyWhich best describes your role on this website today?


(N: 10486)

Library’s Natural Allies Are Big Users

& PotentialPartners.

Librarian32%

School or library website

29%

Teacher24%

Used in the past7%


Web search engine3%

Gale-Cengage Browse SurveyHow did you find out about this website?


(N: 10486)

Wow! Only 29% ofUsers Find

E-Resources Through Library

Websites.

First time36%

Several times a month18%

Several times a week14%

About once a month10%

About once every 3 months8%

Every 6 months or less7% Daily

7%

Gale-Cengage Browse SurveyHow frequently do you use this website?


(N: 10486)

And 39% of YourUsers Are in OurDatabases for

For the VeryFirst Time!

Gale Analysis: Mobile

History (n=149)

Medicine/Health (n=137)

Biography/Genealogy (n=84)

Literature (n=98)

Cultural Studies (n=101)

Politics/Current Events (n=108)

Science/Technology (n=132)

Arts/Humanities (n=125)

Law/Government (n=89)

Business (n=105)

0% 20% 40% 60% 80% 100%

58%

59%

60%

60%

60%

60%

60%

62%

64%

67%

21%

15%

29%

24%

21%

22%

17%

22%

20%

16%

21%

26%

12%

15%

19%

18%

23%

15%

16%

17%

Yes No Not Sure

Mobile Device Users: Interest in Accessing Gale Mobile Site by Topic (Top 10)

Gale Analysis: MobileMarch 26 – September 25, 2012

How likely are you to rec-ommend this site to

someone else?Avg Score: 7.4

N*: 10486

0% 20% 40% 60% 80% 100%

12% 2%3%3% 5% 5% 7% 11% 14% 39%

Gale-Cengage Browse SurveyRecommend - Scores and DistributionsAugust 01, 2009 - September 06, 2010

(Avg. Score: 71)

1 2 3 4 5 6 7 8 9 10* N values exclude "Don't Know"

Your Users Will

Recommend Your

Databases

End Users Will

Recommend Our

Databases

How likely are you to use this site as your primary

resource for your research needs?

Avg Score: 6.9N*: 10486

0% 20% 40% 60% 80% 100%

12% 3% 4% 4% 6% 7% 10% 14% 13% 27%

Gale-Cengage Browse SurveyPrimary Resource - Scores and Distributions


(Avg. Score: 65)


End Users Respect

Our DatabasesAs Primary

Sources

How likely are you to return to this site?

Avg Score: 7.9N*: 10486

0% 20% 40% 60% 80% 100%

9% 2%2%2% 4% 4% 7% 11% 14% 46%

Gale-Cengage Browse SurveyLikelihood to Return - Scores and Distributions


(Avg. Score: 76)


End Users Are

Likely toReturn

What is your overall satis-faction with this site?

Avg Score: 7.2N*: 10486

How well does this site meet your expectations?

Avg Score: 7.1N*: 10486

How does this site compare to your idea of an ideal

website?Avg Score: 6.8

N*: 10486

0% 20% 40% 60% 80% 100%

9%

10%

10%

2%

2%

3%

3%

3%

4%

3%

4%

4%

5%

6%

7%

7%

6%

8%

11%

11%

13%

18%

18%

18%

19%

18%

15%

23%

22%

19%

Gale-Cengage Browse SurveySatisfaction - Scores and DistributionsAugust 01, 2009 - September 06, 2010

(Avg. Score: 67)


End Users Evaluate

Our Services as Meeting

Expectations

Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10

N 203 1153 1611 1391 760 814 1081 1138 1059 691 168 170 192 55

100

300

500

700

900

1,100

1,300

1,500

1,700

Gale-Cengage Browse SurveyNumber of Survey Respondents


Time Periods

Nu

mb

er o

f R

esp

on

den

ts

The School Cycle Drives Many Usage

Scenarios

There’s Great News!We had Room for

Improvement BUT Library Databases Compete

Very Well with User Expectations and Needs Fulfillment.

EDUCATEand Lead

Quick Poll

Should our industry

A

Invest in the development and promotion of a suite of end-user impact and value measurement tools that actually communicate the value in our products?

Should our industry

B

Just deliver the raw statistics that customers are asking for and let them perform the analyses independently?

Until lions learn to write their own story, the story will always be from the perspective

of the hunter not the hunted.

Stephen Abram, MLS, FSLAConsultant, Dysart & Jones/Lighthouse Partners

Cel: [email protected]’s Lighthouse Blog

http://stephenslighthouse.comFacebook, Pinterest, Tumblr: Stephen Abram

LinkedIn / Plaxo: Stephen AbramTwitter: @sabram

SlideShare: StephenAbram1

http://stephenslighthouse.com/

http://www.amazon.com/gp/product/images/0838909329/ref=dp_image_0/104-0100604-9311116?ie=UTF8&n=283155&s=books

s abram nfais keynote feb 25 2013

Documents