know how a data management project can help: improve program design demonstrate effectiveness ...
TRANSCRIPT
Getting to Outcomes: Next Steps
Know how a data management project can help:
Improve program design Demonstrate effectiveness Highlight the best work being done Compete for funding, and Mobilize public support.
Workshop Objectives
Learn about: Identifying expected outcomes, and defining objectives and incremental
indicators of success consistent with your mission.
Workshop Objectives
Understand the elements and stages of a data management planning process
Be familiar with common barriers and costs associated with data management
Workshop Objectives
Learn how quality data can influence and inform the strategic planning process.
Explore options for tracking and using data efficiently at reasonable cost.
Workshop Objectives
City of Pittsfield Neighborhood evaluation◦ Combining & analyzing data from multiple sources
DIAL/SELF (Greenfield, MA) Transitional Living Program housing outcomes◦ Sorting and interpreting data from a single
collection source
(Lets go to visit source tables in Excel then come back to PowerPoint to review graphs)
Examples of Data Compilation:
Building permits over 20K by Pittsfield neighborhoods:
All Pitt
sfiel
d City
Mor
ning
side
Wes
tsid
e
Oth
er Q
ualifi
ed N
eigh
borh
oods
Oth
er P
ittsfi
eld
Neigh
borh
oods
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
All years: Permits ≥20K (percentage of permits taken out in district)
Permits >20K in 2008
Permits ≥20K in 2009
Intake by age range
Any Intake Standard Intake Brief intake Both 0
50
100
150
200
250
300
350
400
450
500
All YouthOver 18Under 18
Issues at intake by age
0
50
100
150
200
250
300
All YouthOver 18Under 18
Housing Outcome Data
Any Intake Identifies a housing problem
Any Posiitive Housing Outcome
0
50
100
150
200
250
300
350
400
450
All YouthOver 18Under 18
Bring key people together at each stage of planning process
Administration, program directors and supervisor participation is critical in early stages, but direct care staff can be helpful too (ask questions!)
Project Planning 101:
Initial planning stages require a deep understanding of the resources (funding, technology/equipment, and staff time) required to plan, implement and maintain a data management project/data driven culture.
Project Planning 101:
It may be worthwhile to invest in a consultant or devote substantial administrative time to produce useful estimates of the time and cost involved in implementing and maintaining a data driven culture
Project Planning 101:
Understand the purpose of your project - what will this data do for your organization?
Identify data priorities
Plan to start small and efficiently – you can grow as you learn and achieve - look for the intersection of what data you can easily obtain and what you would want to know in an ideal world! (go to flip chart )
Planning Guidelines
As you move into more detailed planning, direct care staff input becomes extremely important.
Involve staff in a formal way and carefully assess what support they will need to succeed!
Design formal systems for Training, Support and Accountability
Staff Commitment & Support
Reports/data you already need for funders
Identify information for internal evaluation and improvement (even if it isn’t currently required by funders)
Develop a functional draft of outcomes, objectives and indicators (your dataset) prior to shopping for a database or building a data collection system
Gather the Information You Need to Get Started
ImProve OutcomesSM
A Brief Summary
Objectives = desired participant changes or achievements
Indicators = measurable events
Outcomes = level of achievement
ImProve OutcomesSM Language
ImProve OutcomesSM
& Logic Models
Inputsresources
Outputsactions
Outcomesachievements
Basic Logic Model
Inputsresources
Outputsactions
Indicatorsevents
Objectivesexpectations
Outcomesachievements
ImProve OutcomesSM Model
*Identify tracking method
ImProve OutcomesSM is…
Extension of logic models
Based on incremental change
Means of prioritizing information
Method of categorizing information
Levels of Learning Mastery
◦Knowledge/Comprehension (learn about it)
◦Application (use it, try it out)
◦Synthesis (integrate with other knowledge)
Bloom’s Taxonomy
Specific Measurable Achievable Relevant Timely
Indicators Should be SMART
Use active verbs to describe indicators
Look for achievement opportunities at levels that are relevant to the services, time frame or intervention level of your program
Indicators reflect participant capacity for positive change and choices that indicate forward movement
Indicators are Activity or Behavior Based (observable)
Web/Cloud Based◦ Require reliable, high speed internet connection(s)◦ Each user has own license – can access from anywhere◦ Easy to monitor data entry◦ Evaluate capability and cost of compilation, sorting and
reporting◦ Carefully evaluate ownership of data and “worst-case
scenarios” (e.g., you or the provider go out of business?)
PC-Based◦ You own software and data that is on your computer◦ Speed depends on speed of machine◦ May require additional software to run the database◦ Can be difficult to synchronize data from multiple
sources.◦ Ease of data retrieval depends a lot on initial design and
software used.
Database Options
1. Surveys: Useful to capture information from participants You have to ask the right question(s). That takes planning
and some experimentation to gather aggregateable data. Results can be compiled in Excel – but consider using
Survey Monkey where you can get reports and export to excel.
2. Microsoft Access: Good for demographic data and tracking objectives and
indicator completion – data that changes or needs to be cross-referenced.
Inexpensive, but requires expertise to develop functional applications
Easy to retrieve data through queries
Tools you can use – now!
3. Daily Logs (paper or software)◦ Most useful if data is aggregated and entered into
a database or spreadsheet regularly (daily, weekly or monthly)
◦ Like surveys, the right questions have to be asked to get useful, accessible information
◦ With proper planning, could be used to track a variety of participant achievements.
4. Exit interviews!◦ Build some of the questions to have
aggregateable answers (e.g., multiple choice, name at least one xxx, etc.)
More Tools you can use – now!
