why is data collection & analysis important?€¦ · data collection & validation tips 21...
TRANSCRIPT
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
Today’s Agenda
2
Why is Data Collection & Analysis Important?
3
4
Anecdotes vs. Data
“Rescue groups
cherry pick all of the
highly adoptable
animals.”
In 2015, rescue partners pulled
1,500 animals. Of those:
• 30% Neonatal Kittens
• 30% Medical Needs
• 40% Highly Adoptable
Which is more useful for designing
and managing programs?
“The shelters are full
of nothing but pit
bulls.”
Pit Bull Type dogs make up 20% of
total dog intake. On average, pit
bull type dogs are held for 35 days
prior to an outcome versus 14 days
for other dogs.
Ways to Use Data Analysis to Drive Impact
• Identify new program opportunities
• Geo target S/N or TNR programs
• Monitor compliance with policy
• Determine Length of Stay
• Communicate with community
• Design adoption incentives
• Identify best/worst kennels
• Monitor program effectiveness
• Incentivize and reward staff
• Motivate volunteers and donors
• Identify opportunities for better process
Save Time
Save Money
Save Lives
Salt Lake County Animal Services
• 10K sq ft
• 88 dogs kennels – all
indoor
• 65 cat cages –
stainless steel
• 9700 animals
• LRR – 58.5%
• Statistics available
through public records
request
Required Statistics for $
The Evolution
Analyzing both
numbers and
percentages
Evaluating hold
times based on
outcome
Historical Data
Continued Evolution
Today’s Agenda
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
10
Your Brain – Incredibly Powerful, Slightly Dangerous
11
Cognitive Bias
12
Cognitive bias describes the inherent thinking errors that humans
make in processing information. These thinking errors can prevent
us from accurately understanding reality, even when we have all the
needed data and evidence to form an accurate view.
Said another way, cognitive bias is the common tendency to
acquire and process information by filtering it through one's
own likes, dislikes, and experiences.
Negativity Bias
13
Negativity Bias is a tendency to notice, pay more attention, or give
more weight to negative experiences or information over positive.
What is an animal welfare example
of negativity bias?
Frequency Illusion
Frequency Illusion is the phenomenon in which people who just
learn or notice something start seeing it everywhere.
What is an animal welfare example
of frequency illusion?
Today’s Agenda
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
15
Steps in Data Analysis Process
7
Collect & Validate
Analyze
Interpret & Act
Accurate Data Collection is Key: Beware GIGO!
16
Garbage IN Garbage OUT
If input data are not complete, accurate, and timely, then
the resulting output is unreliable and of no useful value
GIGO Example: Cat Intake
17
Zip codes with
shelter locations
A Few Words on Shelter Software
18
• Different features and
functions mean no one
size fits all approach
• Data you input, you
should be able to extract
and analyze
• Standard and custom
reports usually available
• Use .csv or .xls exports
• Well worth having staff
and/or volunteer
member(s) with expertise
on your system
Google Sheets or Microsoft Excel?
18
• Free
• Cloud based
• Multiple users can be working
on or looking at
simultaneously
• Best for small data sets
• Will timeout if working on a
sheet with many tabs or more
than a few thousand rows
• Lots of add-ons for analysis
• Pivot tables are easier for
beginners
• Need a license
• Saved on your computer
• Only one user can be working
on, have to email versions
around
• Best for any data sets
• Can process lots of data
quickly
• No add-ons available, pretty
much everything you need is
built in
• Pivot tables are better
Demographic Data can be a Helpful Overlay
Demographics – The characteristics of
human populations and population
segments, especially when used to
identify consumer markets.
• Age, Race/Ethnicity, Income,
Education, Employment, Language
Spoken, Housing Type, Access to
Vehicle, etc.
Data Sources
• Zip code summary
www.esri.com/data/esri_data/ziptapestry
• Detailed zip data www.factfinder.census.gov
• City/County data www.quickfacts.census.gov
All Data by Species (Cat/Dog) and
age (</> 5 mo)
• Annual beginning and ending shelter count
• Intake
–Stray/At Large, Relinquished by Owner, Owner Intended
Euthanasia, Transferred in from Agency, Other
• Outcomes
–Adoption, Returned to Owner, Transferred to another Agency,
Returned to Field, Other Live Outcome
–Died in Care, Lost in Care, Shelter Euthanasia, Owner Intended
Euthanasia
Shelter Animals Count Basic Data
22
Resources
• Data Matrix & Definitions
• What’s Your Rate? Calculation Guidance
Additional Data is Very Helpful
23
Shelter Animals Count minimum data will allow you to calculate
release rates and do basic intake and outcome analysis.
Additional data will allow you to do deeper analysis that will benefit
your organization so we strongly suggest adding:
• Breed/Type
• Intake Address
• Length of Stay
Geographic Information Systems Also Powerful
Resourceswww.Mapline.com
www.OpenHeatMap.com
Tools Available via ASPCA X Maps Spot Project
http://aspcapro.org/resource/saving-lives-research-data/x-maps-spot-tools-gis
Data Collection & Validation Tips
21
Things to do BEFORE you start an analysis
1. Identify what data you want to analyze and why
2. Verify how that data is currently being captured in your
system (i.e. who, when, how)
3. Consider whether the data is processed or modified after it
is entered (i.e. intact status)
4. Validate that data values are within realm of reality (i.e.
negative LOS)
5. Look for anomalies or outliers that need to be explored
Don’t Despair – Help is Available!
42
Where to look for help:
• Your local government
• Area universities
–Professors
–Class projects
–Interns
• Your volunteers!
Today’s Agenda
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
28
Data Analysis - OR – Lies, Damn Lies, and Statistics
25
Shelter A Shelter B Shelter C
Population Served 600,000 500,000 100,000
Intake 10,000 25,000 5,000
Euthanasia 5,000 6,000 1,500
Euthanasia as % of Intake 50% 24% 30%
Euthanasia per 1,000 Pop. 8.3 12 15
Which organization has the “best” life saving impact?
Reliance on any single metric will not give
a full picture of performance
Kitten
Rescue
Fluffy Dog
Recsue
Medical
Rescue
Foster Homes 10 15 20
Avg. LOS (Days to Adopt)
Annual Animals Placed
Annual Budget
Cost Per Animal
Rescue Groups Should Look At Metrics Too!
25
Which rescue group has the most life saving impact?
Kitten
Rescue
Fluffy Dog
Recsue
Medical
Rescue
Foster Homes 10 15 20
Avg. LOS (Days to Adopt) 45 20 60
Annual Animals Placed 240 270 120
Annual Budget
Cost Per Animal
Kitten
Rescue
Fluffy Dog
Recsue
Medical
Rescue
Foster Homes 10 15 20
Avg. LOS (Days to Adopt) 45 20 60
Annual Animals Placed 240 270 120
Annual Budget $36,000 $81,000 $120,000
Cost Per Animal $150 $300 $1,000
How does YOUR organization’s impact match with
identified community needs?
Let Your Data Speak – Good Graphs Part 1
26
8%
38%
48% 12%
38%
70%
12%
Population Served: 1,065,897
Intake per 1,000: 9.7
Euthanasia per 1,000: 4.6
Population Served: 669,561
Intake per 1,000: 6.7
Euthanasia per 1,000: 0.8
Good Graphs Part 2 – “Hygeine”
28
38%
Start data
axis at zero
To
compare,
keep same
axis scale
Use Data
Labels to
show #s
“Waterfall” style
graphs give great
data visualization
Pivot Tables are Your Friend!!
29
A pivot table is a tool in Google Sheets & Excel that
allows you to explore large sets of data
interactively. Once you create a pivot table, you can
quickly transform huge amounts of data into a
meaningful summary.
Pivot Table Example – Cat Intake & Outcomes
30
Nearly 50,000 rows of outcome data becomes. . . .
Pivot Table Tutorials
Microsoft https://youtu.be/qMGILHiLqr0
Google https://youtube.com/watch
Data Analysis Tips & Tricks
33
1. Look at numbers (#) and percentages (%)
2. Use graphs to visualize data; “Waterfall” style graphs are
especially useful
3. Use good graph hygiene; always start graph axis at zero to
avoid distortion
4. For comparison between graphs, keep same axis maximum
and scale
5. For comparison between geography, normalize per 1,000
human population
6. Two words: PIVOT TABLES!!
Beware Data Analysis Pitfalls
34
Garbage IN =
Garbage OUT
“Boiling the Ocean”
Analysis
Paralysis
Keep the “Big Picture” in Mind
Goal: Maximize Lifesaving Impact
Decrease
Intake
Increase
Live Release
Spay Neuter
Trap Neuter Return
Surrender Prevention
Adoption
Kitten Foster
Return to Field
Know your INTAKE
What pets are coming into
shelter? Why?
Know your OUTCOMES
What happens to pets in
shelter? Why?
37
Today’s Agenda
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
38
Correlation is NOT Causation
38
• Studies have shown that people who eat yogurt regularly have a
healthier body weight than those who do not eat yogurt regularly
• Can we therefore say that if you eat yogurt you will have a healthier
body weight? YOGURT CAUSES HEALTHY WEIGHT
• Or, might it be that people who make healthier diet choices overall
tend to eat more yogurt? HEALTHY PEOPLE EAT YOGURT
• We can say that eating yogurt is CORRELATED with healthy
weight, we cannot say that eating yogurt CAUSES healthy weight
• Most results have multiple contributing causes
• Resist the urge to oversimplify cause and effect
Does Seasonal Kitten Intake Cause Higher Euthanasia?
40
0
500
1000
1500
2000
2500
3000
Ja
n
Fe
b
Mar
Ap
r
May
Ju
n
Ju
l
Au
g
Sep
Oct
No
v
Dec
2014 Feline Intake
Cat Intake Kitten Intake
0
500
1000
1500
2000
2500
3000
Ja
n
Fe
b
Mar
Ap
r
May
Ju
n
Ju
l
Au
g
Sep
Oct
No
v
Dec
2014 Feline Euthanasia
Kitten Euthanasia Cat Euthanasia
Data Interpretation Tips & Tricks
41
1. Don’t get too attached to your initial hypothesis, look for
other explanations
2. Remember that correlation does not imply causation
3. Recall that most outcomes are the result of multiple different
variables – resist the urge to over simplify!
4. Always ask yourself questions “What else could be going on
here?” “What else has changed?”
Today’s Agenda
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
42
CAGE TYPE CASE STUDY
Does Cage Type Impact LOS for Cats at Adopt & Shop?
What data to analyze, and why?
• Adult cats, In a single kennel for entire stay, after Nov
Check data for errors and anomalies
• Eliminate records with negative LOS
Calculate metrics and graph
• Average LOS by kennel type, range
Interpret and act. . . .all grate front kennels?
Pull data from system and process
• Date of outcome – DOB = Age
Cage Type Data Collection & Analysis Process
Cage Type Data Collection & Analysis Process
Cats in
Sample19 11 9
What can we
conclude?
CITY OF LOS ANGELES CASE STUDY
2010 Los Angeles Intake & Outcomes - Numbers
58
33
1
15
90
10
20
30
40
50
60
70
Inta
ke
Dogs
Pup
pie
s
Ca
ts
Kitte
ns
Th
ou
san
ds
2010 Intake
22 90
7
70
10
20
30
40
50
60
Euth
ana
sia
Dogs
Pup
pie
s
Ca
ts
Kitte
ns
Th
ou
san
ds
2010 Euthanasia
62% Save Rate
58
33
1
15
90
10
20
30
40
50
60
70In
take
Do
gs
Pup
pie
s
Cats
Kitte
ns
Th
ou
san
ds
2010 Intake
22 90
7
70
10
20
30
40
50
60
Euth
ana
sia
Dogs
Pup
pie
s
Cats
Kitte
ns
Th
ou
san
ds
2010 Euthanasia
38 27 23 44 72% of
Intake
Biggest Opportunities for Lifesaving in LA in 2010?
2010 LA Outcomes by Animal Type - Percentages
NKLA Program Started 2011
Used 2010 data analysis and community input to design several
programs with aim to make Los Angeles No Kill by 2017
Initial programs included:
• Grants for zip code targeted spay neuter (low income)
• Adoption subsidies for rescue groups
• Kitten nursery/foster programs
• New adoption facilities
2010 vs 2014 Los Angeles Intake - Numbers
58
33
1
15
90
10
20
30
40
50
60
70
Inta
ke
Dogs
Pup
pie
s
Ca
ts
Kitte
ns
Th
ou
san
ds
2010 Intake
52
29
2
13
90
10
20
30
40
50
60
70
Inta
ke
Dogs
Pup
pie
s
Ca
ts
Kitte
ns
Th
ou
san
ds
2014 Intake
2010 vs 2014 Los Angeles Euthanasia - Numbers
22
9
0
7
7
0
5
10
15
20
25
Euth
ana
sia
Dogs
Pup
pie
s
Cats
Kitte
ns
Th
ou
san
ds
2010 Euthanasia
12 40
4
40
5
10
15
20
25
Euth
ana
sia
Dogs
Pup
pie
s
Ca
ts
Kitte
ns
Th
ou
san
ds
2014 Euthanasia
77% Save Rate
2014 Outcomes by Animal Type - Percentages
Next Steps for NKLA
CATS, CATS, CATS!!!
• Shifting S/N grant
focus to cats
• More kitten nursery/
foster capacity
• Cat adoption
promotions
• Targeting TNR to
areas with high kitten
intake
LENGTH OF STAY CASE STUDY
Do different transfer models impact LOS at Adopt &
Shop?
3 different transfer models1. A&S Staff select animals
2. Third Party select animals
3. Source shelter staff selects animals
Our assumptions• LOS will be lowest when the A&S staff
select the animals
• LOS will be highest when shelter staff select
animals since they may be sending those
that they cannot adopt
What data to analyze, and why?
• All animals by source shelter
Check data for errors and anomalies
• Eliminate records with negative LOS, eliminate
anomalies
Calculate metrics and graph
• Average LOS by source shelter, grouped by transfer
model
Interpret and act. . . .source more animals from shelters
with a lower LOS?
Pull data from system and process
• Date of outcome – Date of intake = LOS
• Total LOS / total number of entries = AVG LOS
LOS Collection & Analysis Process
LOS Collection & Analysis Process
Were our
assumptions
correct?
LOS Collection & Analysis Process
Is the
correlation
between how
the animal is
selected?
Or LRR?
What about
causation?
Shelter Published Live Release Rate
49
FOSTER CAPACITY CASE STUDY
What is it really going to take to save 1800 kittens?
2015 Statistics• ~200 foster homes
• <1000 kittens/mothers went
through program
• Minimal foster bottleneck
Assumptions• We are going to need A LOT of
new foster homes
Historical data to analyze
• AVG # litters per foster annually, AVG # kittens in a litter,
AVG LOS in a foster home
Pull data from system and process
• Total # kittens fostered/total # litters = AVG # kittens per
litter
• Total # litters/total # foster homes = AVG # of litters per
foster home
How many fosters homes are needed to save 1800
kittens in 12 months?
Historical data to analyze
• AVG # litters per foster annually, AVG # kittens in a litter,
AVG LOS in a foster home
Historical data to analyze
• AVG # litters per foster annually, AVG # kittens in a litter,
AVG LOS in a foster home
Pull data from system and process
• Total # kittens fostered/total # litters = AVG # kittens per
litter
• Total # litters/total # foster homes = AVG # of litters per
foster home
Check data for errors and anomalies
Calculate metrics and graph
Interpret and act
• Do we have enough foster homes?
LOS Collection & Analysis Process
Raw Data Processed Data
Charting the Results
65%22%
7%
3% 2%
1 Litter 2 Litters 3 Litters 4 Litters 6 Litters
0
20
40
60
80
100
120
140
160
1 Litter 2 Litters 3 Litters 4 Litters 5 Litters 6 Litters
Chart both numbers and percentages to see the full picture.
Findings• 87% of our foster homes (FH) take 1-2 litters annually
• AVG litter size is 2 kittens
• 226 active foster homes
Calculations• 1800 kittens/2 kittens per litter = 900 litters
• 900 litters/2 litters per FH = 450 FHs needed
Optional Paths • Recruit an additional 224 FHs
• Increase # of kittens per litter (1800 kittens/3 kittens per litter = 600 litters/2
litters per FH = 300 FHs needed)
• Increase # of litters per FH (1800 kittens/2 kittens per litter = 900 litters/3 litters
per FH = 300 FHs needed)
• Increase # of kittens per litter and litters per FH (1800/3 kittens per litter = 600
litters/3 litters per FH = 200 FHs needed)
Results
49
ADOPTER DISTANCE CASE STUDY
How far are adopters willing to drive in L.A. traffic?
Historical data to analyze
• AVG # litters per foster annually, AVG # kittens in a litter,
AVG LOS in a foster home
Pull data from system and process
• Total # kittens fostered/total # litters = AVG # kittens per
litter
• Total # litters/total # foster homes = AVG # of litters per
foster home
Where should we spend our marketing dollars?
Historical data to analyze
• AVG # litters per foster annually, AVG # kittens in a litter,
AVG LOS in a foster home
Historical data to analyze
• All adopters with addresses
Pull data from system and process
• Standard report available with adopter information
• Remove foster and employee adoptions
Check data for errors and anomalies
• Out of state
Calculate metrics and heat map
Interpret and act
• How far from the store locations should we be marketing
to potential adopters?
Heat Mapping