why is data collection & analysis important?€¦ · data collection & validation tips 21...

• Why does this matter?

• A Dangerous Mind

• Data Collection

• Data Analysis

• Data Interpretation

• Case Studies

Today’s Agenda

2

Why is Data Collection & Analysis Important?

3

4

Anecdotes vs. Data

“Rescue groups

cherry pick all of the

highly adoptable

animals.”

In 2015, rescue partners pulled

1,500 animals. Of those:

• 30% Neonatal Kittens

• 30% Medical Needs

• 40% Highly Adoptable

Which is more useful for designing

and managing programs?

“The shelters are full

of nothing but pit

bulls.”

Pit Bull Type dogs make up 20% of

total dog intake. On average, pit

bull type dogs are held for 35 days

prior to an outcome versus 14 days

for other dogs.

Ways to Use Data Analysis to Drive Impact

• Identify new program opportunities

• Geo target S/N or TNR programs

• Monitor compliance with policy

• Determine Length of Stay

• Communicate with community

• Design adoption incentives

• Identify best/worst kennels

• Monitor program effectiveness

• Incentivize and reward staff

• Motivate volunteers and donors

• Identify opportunities for better process

Save Time

Save Money

Save Lives

Salt Lake County Animal Services

• 10K sq ft

• 88 dogs kennels – all

indoor

• 65 cat cages –

stainless steel

• 9700 animals

• LRR – 58.5%

• Statistics available

through public records

request

Required Statistics for $

The Evolution

Analyzing both

numbers and

percentages

Evaluating hold

times based on

outcome

Historical Data

Continued Evolution

Today’s Agenda

• Why does this matter?

• A Dangerous Mind

• Data Collection

• Data Analysis

• Data Interpretation

• Case Studies

10

Your Brain – Incredibly Powerful, Slightly Dangerous

11

Cognitive Bias

12

Cognitive bias describes the inherent thinking errors that humans

make in processing information. These thinking errors can prevent

us from accurately understanding reality, even when we have all the

needed data and evidence to form an accurate view.

Said another way, cognitive bias is the common tendency to

acquire and process information by filtering it through one's

own likes, dislikes, and experiences.

Negativity Bias

13

Negativity Bias is a tendency to notice, pay more attention, or give

more weight to negative experiences or information over positive.

What is an animal welfare example

of negativity bias?

Frequency Illusion

Frequency Illusion is the phenomenon in which people who just

learn or notice something start seeing it everywhere.

What is an animal welfare example

of frequency illusion?

Today’s Agenda

• Why does this matter?

• A Dangerous Mind

• Data Collection

• Data Analysis

• Data Interpretation

• Case Studies

15

Steps in Data Analysis Process

7

Collect & Validate

Analyze

Interpret & Act

Accurate Data Collection is Key: Beware GIGO!

16

Garbage IN Garbage OUT

If input data are not complete, accurate, and timely, then

the resulting output is unreliable and of no useful value

GIGO Example: Cat Intake

17

Zip codes with

shelter locations

A Few Words on Shelter Software

18

• Different features and

functions mean no one

size fits all approach

• Data you input, you

should be able to extract

and analyze

• Standard and custom

reports usually available

• Use .csv or .xls exports

• Well worth having staff

and/or volunteer

member(s) with expertise

on your system

Google Sheets or Microsoft Excel?

18

• Free

• Cloud based

• Multiple users can be working

on or looking at

simultaneously

• Best for small data sets

• Will timeout if working on a

sheet with many tabs or more

than a few thousand rows

• Lots of add-ons for analysis

• Pivot tables are easier for

beginners

• Need a license

• Saved on your computer

• Only one user can be working

on, have to email versions

around

• Best for any data sets

• Can process lots of data

quickly

• No add-ons available, pretty

much everything you need is

built in

• Pivot tables are better

Demographic Data can be a Helpful Overlay

Demographics – The characteristics of

human populations and population

segments, especially when used to

identify consumer markets.

• Age, Race/Ethnicity, Income,

Education, Employment, Language

Spoken, Housing Type, Access to

Vehicle, etc.

Data Sources

• Zip code summary

www.esri.com/data/esri_data/ziptapestry

• Detailed zip data www.factfinder.census.gov

• City/County data www.quickfacts.census.gov

All Data by Species (Cat/Dog) and

age (</> 5 mo)

• Annual beginning and ending shelter count

• Intake

–Stray/At Large, Relinquished by Owner, Owner Intended

Euthanasia, Transferred in from Agency, Other

• Outcomes

–Adoption, Returned to Owner, Transferred to another Agency,

Returned to Field, Other Live Outcome

–Died in Care, Lost in Care, Shelter Euthanasia, Owner Intended

Euthanasia

Shelter Animals Count Basic Data

22

Resources

• Data Matrix & Definitions

• What’s Your Rate? Calculation Guidance

Additional Data is Very Helpful

23

Shelter Animals Count minimum data will allow you to calculate

release rates and do basic intake and outcome analysis.

Additional data will allow you to do deeper analysis that will benefit

your organization so we strongly suggest adding:

• Breed/Type

• Intake Address

• Length of Stay

Geographic Information Systems Also Powerful

Resourceswww.Mapline.com

www.OpenHeatMap.com

Tools Available via ASPCA X Maps Spot Project

http://aspcapro.org/resource/saving-lives-research-data/x-maps-spot-tools-gis

Data Collection & Validation Tips

21

Things to do BEFORE you start an analysis

1. Identify what data you want to analyze and why

2. Verify how that data is currently being captured in your

system (i.e. who, when, how)

3. Consider whether the data is processed or modified after it

is entered (i.e. intact status)

4. Validate that data values are within realm of reality (i.e.

negative LOS)

5. Look for anomalies or outliers that need to be explored

Don’t Despair – Help is Available!

42

Where to look for help:

• Your local government

• Area universities

–Professors

–Class projects

–Interns

• Your volunteers!

• LinkedIn

Today’s Agenda

• Why does this matter?

• A Dangerous Mind

• Data Collection

• Data Analysis

• Data Interpretation

• Case Studies

28

Data Analysis - OR – Lies, Damn Lies, and Statistics

25

Shelter A Shelter B Shelter C

Population Served 600,000 500,000 100,000

Intake 10,000 25,000 5,000

Euthanasia 5,000 6,000 1,500

Euthanasia as % of Intake 50% 24% 30%

Euthanasia per 1,000 Pop. 8.3 12 15

Which organization has the “best” life saving impact?

Reliance on any single metric will not give

a full picture of performance

Kitten

Rescue

Fluffy Dog

Recsue

Medical

Rescue

Foster Homes 10 15 20

Avg. LOS (Days to Adopt)

Annual Animals Placed

Annual Budget

Cost Per Animal

Rescue Groups Should Look At Metrics Too!

25

Which rescue group has the most life saving impact?

Kitten

Rescue

Fluffy Dog

Recsue

Medical

Rescue

Foster Homes 10 15 20

Avg. LOS (Days to Adopt) 45 20 60

Annual Animals Placed 240 270 120

Annual Budget

Cost Per Animal

Kitten

Rescue

Fluffy Dog

Recsue

Medical

Rescue

Foster Homes 10 15 20

Avg. LOS (Days to Adopt) 45 20 60

Annual Animals Placed 240 270 120

Annual Budget $36,000 $81,000 $120,000

Cost Per Animal $150 $300 $1,000

How does YOUR organization’s impact match with

identified community needs?

Let Your Data Speak – Good Graphs Part 1

26

8%

38%

48% 12%

38%

70%

12%

Population Served: 1,065,897

Intake per 1,000: 9.7

Euthanasia per 1,000: 4.6

Population Served: 669,561

Intake per 1,000: 6.7

Euthanasia per 1,000: 0.8

Good Graphs Part 2 – “Hygeine”

28

38%

Start data

axis at zero

To

compare,

keep same

axis scale

Use Data

Labels to

show #s

“Waterfall” style

graphs give great

data visualization

Pivot Tables are Your Friend!!

29

A pivot table is a tool in Google Sheets & Excel that

allows you to explore large sets of data

interactively. Once you create a pivot table, you can

quickly transform huge amounts of data into a

meaningful summary.

Pivot Table Example – Cat Intake & Outcomes

30

Nearly 50,000 rows of outcome data becomes. . . .

Pivot Table Tutorials

Microsoft https://youtu.be/qMGILHiLqr0

Google https://youtube.com/watch

Data Analysis Tips & Tricks

33

1. Look at numbers (#) and percentages (%)

2. Use graphs to visualize data; “Waterfall” style graphs are

especially useful

3. Use good graph hygiene; always start graph axis at zero to

avoid distortion

4. For comparison between graphs, keep same axis maximum

and scale

5. For comparison between geography, normalize per 1,000

human population

6. Two words: PIVOT TABLES!!

Beware Data Analysis Pitfalls

34

Garbage IN =

Garbage OUT

“Boiling the Ocean”

Analysis

Paralysis

Keep the “Big Picture” in Mind

Goal: Maximize Lifesaving Impact

Decrease

Intake

Increase

Live Release

Spay Neuter

Trap Neuter Return

Surrender Prevention

Adoption

Kitten Foster

Return to Field

Know your INTAKE

What pets are coming into

shelter? Why?

Know your OUTCOMES

What happens to pets in

shelter? Why?

37

Today’s Agenda

• Why does this matter?

• A Dangerous Mind

• Data Collection

• Data Analysis

• Data Interpretation

• Case Studies

38

Correlation is NOT Causation

38

• Studies have shown that people who eat yogurt regularly have a

healthier body weight than those who do not eat yogurt regularly

• Can we therefore say that if you eat yogurt you will have a healthier

body weight? YOGURT CAUSES HEALTHY WEIGHT

• Or, might it be that people who make healthier diet choices overall

tend to eat more yogurt? HEALTHY PEOPLE EAT YOGURT

• We can say that eating yogurt is CORRELATED with healthy

weight, we cannot say that eating yogurt CAUSES healthy weight

• Most results have multiple contributing causes

• Resist the urge to oversimplify cause and effect

Does Seasonal Kitten Intake Cause Higher Euthanasia?

40

0

500

1000

1500

2000

2500

3000

Ja

n

Fe

b

Mar

Ap

r

May

Ju

n

Ju

l

Au

g

Sep

Oct

No

v

Dec

2014 Feline Intake

Cat Intake Kitten Intake

0

500

1000

1500

2000

2500

3000

Ja

n

Fe

b

Mar

Ap

r

May

Ju

n

Ju

l

Au

g

Sep

Oct

No

v

Dec

2014 Feline Euthanasia

Kitten Euthanasia Cat Euthanasia

Data Interpretation Tips & Tricks

41

1. Don’t get too attached to your initial hypothesis, look for

other explanations

2. Remember that correlation does not imply causation

3. Recall that most outcomes are the result of multiple different

variables – resist the urge to over simplify!

4. Always ask yourself questions “What else could be going on

here?” “What else has changed?”

Today’s Agenda

• Why does this matter?

• A Dangerous Mind

• Data Collection

• Data Analysis

• Data Interpretation

• Case Studies

42

CAGE TYPE CASE STUDY

Does Cage Type Impact LOS for Cats at Adopt & Shop?

What data to analyze, and why?

• Adult cats, In a single kennel for entire stay, after Nov

Check data for errors and anomalies

• Eliminate records with negative LOS

Calculate metrics and graph

• Average LOS by kennel type, range

Interpret and act. . . .all grate front kennels?

Pull data from system and process

• Date of outcome – DOB = Age

Cage Type Data Collection & Analysis Process

Cage Type Data Collection & Analysis Process

Cats in

Sample19 11 9

What can we

conclude?

CITY OF LOS ANGELES CASE STUDY

2010 Los Angeles Intake & Outcomes - Numbers

58

33

1

15

90

10

20

30

40

50

60

70

Inta

ke

Dogs

Pup

pie

s

Ca

ts

Kitte

ns

Th

ou

san

ds

2010 Intake

22 90

7

70

10

20

30

40

50

60

Euth

ana

sia

Dogs

Pup

pie

s

Ca

ts

Kitte

ns

Th

ou

san

ds

2010 Euthanasia

62% Save Rate

58

33

1

15

90

10

20

30

40

50

60

70In

take

Do

gs

Pup

pie

s

Cats

Kitte

ns

Th

ou

san

ds

2010 Intake

22 90

7

70

10

20

30

40

50

60

Euth

ana

sia

Dogs

Pup

pie

s

Cats

Kitte

ns

Th

ou

san

ds

2010 Euthanasia

38 27 23 44 72% of

Intake

Biggest Opportunities for Lifesaving in LA in 2010?

2010 LA Outcomes by Animal Type - Percentages

NKLA Program Started 2011

Used 2010 data analysis and community input to design several

programs with aim to make Los Angeles No Kill by 2017

Initial programs included:

• Grants for zip code targeted spay neuter (low income)

• Adoption subsidies for rescue groups

• Kitten nursery/foster programs

• New adoption facilities

2010 vs 2014 Los Angeles Intake - Numbers

58

33

1

15

90

10

20

30

40

50

60

70

Inta

ke

Dogs

Pup

pie

s

Ca

ts

Kitte

ns

Th

ou

san

ds

2010 Intake

52

29

2

13

90

10

20

30

40

50

60

70

Inta

ke

Dogs

Pup

pie

s

Ca

ts

Kitte

ns

Th

ou

san

ds

2014 Intake

2010 vs 2014 Los Angeles Euthanasia - Numbers

22

9

0

7

7

0

5

10

15

20

25

Euth

ana

sia

Dogs

Pup

pie

s

Cats

Kitte

ns

Th

ou

san

ds

2010 Euthanasia

12 40

4

40

5

10

15

20

25

Euth

ana

sia

Dogs

Pup

pie

s

Ca

ts

Kitte

ns

Th

ou

san

ds

2014 Euthanasia

77% Save Rate

2014 Outcomes by Animal Type - Percentages

Next Steps for NKLA

CATS, CATS, CATS!!!

• Shifting S/N grant

focus to cats

• More kitten nursery/

foster capacity

• Cat adoption

promotions

• Targeting TNR to

areas with high kitten

intake

LENGTH OF STAY CASE STUDY

Do different transfer models impact LOS at Adopt &

Shop?

3 different transfer models1. A&S Staff select animals

2. Third Party select animals

3. Source shelter staff selects animals

Our assumptions• LOS will be lowest when the A&S staff

select the animals

• LOS will be highest when shelter staff select

animals since they may be sending those

that they cannot adopt

What data to analyze, and why?

• All animals by source shelter

Check data for errors and anomalies

• Eliminate records with negative LOS, eliminate

anomalies

Calculate metrics and graph

• Average LOS by source shelter, grouped by transfer

model

Interpret and act. . . .source more animals from shelters

with a lower LOS?

Pull data from system and process

• Date of outcome – Date of intake = LOS

• Total LOS / total number of entries = AVG LOS

LOS Collection & Analysis Process

LOS Collection & Analysis Process

Were our

assumptions

correct?

LOS Collection & Analysis Process

Is the

correlation

between how

the animal is

selected?

Or LRR?

What about

causation?

Shelter Published Live Release Rate

49

FOSTER CAPACITY CASE STUDY

What is it really going to take to save 1800 kittens?

2015 Statistics• ~200 foster homes

• <1000 kittens/mothers went

through program

• Minimal foster bottleneck

Assumptions• We are going to need A LOT of

new foster homes

Historical data to analyze

• AVG # litters per foster annually, AVG # kittens in a litter,

AVG LOS in a foster home

Pull data from system and process

• Total # kittens fostered/total # litters = AVG # kittens per

litter

• Total # litters/total # foster homes = AVG # of litters per

foster home

How many fosters homes are needed to save 1800

kittens in 12 months?

Historical data to analyze

• AVG # litters per foster annually, AVG # kittens in a litter,

AVG LOS in a foster home

Historical data to analyze

• AVG # litters per foster annually, AVG # kittens in a litter,

AVG LOS in a foster home

Pull data from system and process

• Total # kittens fostered/total # litters = AVG # kittens per

litter

• Total # litters/total # foster homes = AVG # of litters per

foster home

Check data for errors and anomalies

Calculate metrics and graph

Interpret and act

• Do we have enough foster homes?

LOS Collection & Analysis Process

Raw Data Processed Data

Charting the Results

65%22%

7%

3% 2%

1 Litter 2 Litters 3 Litters 4 Litters 6 Litters

0

20

40

60

80

100

120

140

160

1 Litter 2 Litters 3 Litters 4 Litters 5 Litters 6 Litters

Chart both numbers and percentages to see the full picture.

Findings• 87% of our foster homes (FH) take 1-2 litters annually

• AVG litter size is 2 kittens

• 226 active foster homes

Calculations• 1800 kittens/2 kittens per litter = 900 litters

• 900 litters/2 litters per FH = 450 FHs needed

Optional Paths • Recruit an additional 224 FHs

• Increase # of kittens per litter (1800 kittens/3 kittens per litter = 600 litters/2

litters per FH = 300 FHs needed)

• Increase # of litters per FH (1800 kittens/2 kittens per litter = 900 litters/3 litters

per FH = 300 FHs needed)

• Increase # of kittens per litter and litters per FH (1800/3 kittens per litter = 600

litters/3 litters per FH = 200 FHs needed)

Results

49

ADOPTER DISTANCE CASE STUDY

How far are adopters willing to drive in L.A. traffic?

Historical data to analyze

• AVG # litters per foster annually, AVG # kittens in a litter,

AVG LOS in a foster home

Pull data from system and process

• Total # kittens fostered/total # litters = AVG # kittens per

litter

• Total # litters/total # foster homes = AVG # of litters per

foster home

Where should we spend our marketing dollars?

Historical data to analyze

• AVG # litters per foster annually, AVG # kittens in a litter,

AVG LOS in a foster home

Historical data to analyze

• All adopters with addresses

Pull data from system and process

• Standard report available with adopter information

• Remove foster and employee adoptions

Check data for errors and anomalies

• Out of state

Calculate metrics and heat map

Interpret and act

• How far from the store locations should we be marketing

to potential adopters?

Heat Mapping

Questions?

April Harris

april@adoptapet.com

58

Follow Found Animals!