![Page 1: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/1.jpg)
Building a Blueprint for Using Data more Effectively
Shawn Bergman Appalachian State and Vela InstituteJennifer Wilson-Kearse Vela Institute
1
![Page 2: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/2.jpg)
Session Overview1. Session Objectives2. Foundational Concepts3. Analytics Method 4. Wrap-Up and Q&A
2
![Page 3: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/3.jpg)
1Differentiate
between metrics and analytics
2Examine
relationship between data
analytics problem-solving
3Look at analytics as a resource to
improve decision-making
4Identify key concepts or
data needed to deliver
actionable results
Session Objectives
3
![Page 4: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/4.jpg)
Promise of Analytics
4
![Page 5: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/5.jpg)
Great!What could possibly go wrong?
5
![Page 6: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/6.jpg)
Reasons Analytic Initiatives Fail
6
![Page 7: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/7.jpg)
Session Handout
![Page 8: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/8.jpg)
Analytics: What is it?
Explore
Scientific process of transforming data into insights for making better decisions
Visualize Discover Communicate
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences; Schniederjans, M. J., Schniederjans, D. G., & Starkey, C. M. (2015). Business analytics principles, concepts, and applications with SAS: What, why, and how. Upper Saddle River, New Jersey: Pearson Education, Inc.
![Page 9: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/9.jpg)
Four Analytic Applications
Prescriptive
Futu
re
Predictive
Diagnostic
Descriptive Past
9
![Page 10: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/10.jpg)
Evidence-Based Practices
Barends, E., Rousseau, D.M., & Briner, R.B. (2014). Evidence-Based Management: The Basic Principles. Amsterdam: Center for Evidence-Based Management. 10
![Page 11: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/11.jpg)
Five Components of Data Analytics
Bergman, S.M. (2016, December). Applied data analytics: It is not all about the numbers. Presentation made to the Appalachian State University Walker College of Business, Boone, NC. 11
![Page 12: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/12.jpg)
Data Analytics Process
Deploy Results
Problem Framing
Analytics Problem Framing
DataProtocols
Method Selection
Build Analytics
Model
Project Life Cycle Management
• Provides a way to systematically think about how to use data and analytics • Helps to ensure projects produce the desired outcomes and results • Is an iterative process that requires a team working together in collaboration
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 12
![Page 13: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/13.jpg)
• The most important, because it sets expectations and limitations• Establishes a firm foundation on which analytics can be implemented
Deploy Results
Problem Framing
Analytics Problem Framing
DataProtocols
Method Selection
Build Analytics
Model
Project Life Cycle Management
Step 1
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 13
![Page 14: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/14.jpg)
What is the question?
Who are the stakeholders?
Can the problem be addressed with
analytics?
What are the potential issues or
constraints?
What are the benefits?
Agree on problem statement or
question
Problem Framing: Identify & Refine
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 14
![Page 15: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/15.jpg)
What issues could disrupt the project?
Who should have access to information?
Who should be involved and
when?
Establish communication
strategies
How can negative impact
be reduced?
Consider interests of all stakeholders who may affect or be affected by project
Stakeholder Analysis
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences
Problem Framing
15
![Page 16: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/16.jpg)
• Can our goals, methods, and measurable outcomes be explicitly stated in our strategic plan and get support from key personnel?
• What would it take to make this happen?
• What will we do with the answer or solution once we get it?
• Have we considered the unintended consequences that analytics may introduce?
Problem Framing: Making the Case
Problem Framing
16
![Page 17: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/17.jpg)
What do you want to know?Activity 1: Pair and share
Question and Action • What questions do you have about work in
your community?• What actions would you take if you know the
answer to those questions?
Stakeholders• Who should be involved in helping answer
those questions?• Who should be involved in taking action after
you get the answers to those questions?
• Respond to the questions and write down responses.5-10 minutes
• Share responses with partners at your table.
• Share out some questions with group.5 minutes
Problem Framing
17
![Page 18: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/18.jpg)
• Dialogue between the stakeholders who have a problem and the analytics personnel who can provide a solution to that problem
Deploy Results
Problem Framing
Analytics Problem Framing
DataProtocols
Method Selection
Build Analytics
Model
Project Life Cycle Management
Step 2
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 18
![Page 19: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/19.jpg)
Analytics Problem Framing
Reframe QuestionReformulate problem statement into analytics
problem
Develop a proposed set of drivers and relationships to outputs
Set the set of assumptions
Define key metrics of success
Obtain agreement on analytics solution
Framing the problem the right way isimperative for clear thinking, proper
planning, and effective communication
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 19
![Page 20: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/20.jpg)
Define the input/output functions of the problem
• Communicate them to stakeholders
• Sketches & diagrams
Gather a sense of their possible effects and how to
communicate them
• Simple table• Black box sketch
Analytic Model• Diagramming methods can be helpful
• Conceptualize relationships between variables and create a predictive model
• Based on experience, knowledge, and logical relationships • Connect desired outcome to input variables and constants
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences
Analytics Problem Framing
20
![Page 21: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/21.jpg)
Model Example
21
Analytics Problem Framing
![Page 22: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/22.jpg)
Build a ModelActivity 2: Build a model
• Identify the outcome(s) for the question
• What are the drivers (factors) that influence the outcome?
• What is the relationship(s) between the drivers and outcomes?
• Can you provide rationale or justification for those relationships?
• For one problem in Activity 1: create a analytic-model diagram or table 5 minutes
• Share your model with members of group 5 minutes
• Discuss the ease and/or difficulty of putting an analytic-model together 5 minutes Analytics
Problem Framing
22
![Page 23: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/23.jpg)
Causal Validity
• Cause related to effect• Statistical validity
• Cause before effect • Temporal precedence
• Control extraneous variables• Factors that influence the
relationship between the cause and effect
Cause Effect
Extraneous
Analytics Problem Framing
23
![Page 24: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/24.jpg)
Internal Validity:Third Variable Problem Examples
Analytics Problem Framing
24
![Page 25: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/25.jpg)
Internal Validity:Third Variable Problem Examples
Analytics Problem Framing
25
![Page 26: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/26.jpg)
Internal Validity:Third Variable Problem Examples
Analytics Problem Framing
26
![Page 27: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/27.jpg)
Number of books in
the home
Child reading
level
Extraneous Variables:Common Cause Variables
Analytics Problem Framing
27
X
Parent Attributes
![Page 28: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/28.jpg)
GPAUsing Face-
to-Face Tutoring
Extraneous Variables:Common Cause Variables
Analytics Problem Framing
28
X
Student Attributes
![Page 29: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/29.jpg)
What Am I Missing? Activity 3: Think About It
For the analytic-model created in Activity 2:• Re-examine the relationships between the
input (driver) factor• Should other inputs be added to the model?
• Re-examine the relationships between the model’s inputs and outcomes• Are there any unmeasured (third) variables
that could change the nature of those relationships?
• Respond to the questions by thinking about it and write down responses.5 minutes
• Report back the ease and/or difficulty of finding unmeasured variables10 minutes
Analytics Problem Framing
29
![Page 30: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/30.jpg)
• Determine how data collection and management will support analytic framework• Make sure data can answer the question and provide actionable results
Deploy Results
Problem Framing
Analytics Problem Framing
DataProtocols Method
Selection
Build Analytics
Model
Project Life Cycle Management
Step 3
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 30
![Page 31: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/31.jpg)
Identify and prioritize data needs and resources
Identify means of data collection and acquisition
Combine, rescale, clean, and share data
Determine documentation and report
“If you can't measure it, you can't manage it.”-Peter Drucker
Data Protocols
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences
DataProtocols
31
![Page 32: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/32.jpg)
• Examining the analytics problems • Do you have the data (measured variables) needed?• Effort to pull those data elements together? • Is the data of sufficient quality to answer the question?
• This examination should also loop back to problem • What is the most important outcome? • What is the best use of organizational resources?
Data Protocols: Determine Needs
DataProtocols
32
![Page 33: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/33.jpg)
Data Audit Example
![Page 34: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/34.jpg)
34
History Lesson:Importance of Sampling
• Who was the only president from the state of Missouri?
![Page 35: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/35.jpg)
35
History Lesson:Importance of Sampling
![Page 36: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/36.jpg)
Got Data?Activity 4: DATA!!!
• Name of outcome or driver variable
• Source or potential source of data
• Do you currently have access to the data for this variable?
• Is the data for this variable representative of target population?
• Data issues with this variable (e.g., missing or incomplete data, have the right level of data)?
• For analytic-model in Activity 2: determine data needs and data quality5 minutes
• Discuss comments and feedback regarding the ease and/or difficulty of getting good data10 minutes
DataProtocols
36
![Page 37: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/37.jpg)
Steps 1, 2, & 3
• Continual interplay between the data you have, the data you can collect, and the questions you want to answer.
Deploy Results
Problem Framing
Analytics Problem Framing
DataProtocols Method
Selection
Build Analytics
Model
Project Life Cycle Management
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 37
![Page 38: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/38.jpg)
Steps 4 & 5
Deploy Results
Problem Framing
Analytics Problem Framing
DataProtocols
Method Selection
Build Analytics
Model
Project Life Cycle Management
• Steps are typically completed by analytics personnel
• Communication still needed, but to a lesser extent
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 38
![Page 39: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/39.jpg)
Data Analysis: Analytics Models
Method Selection
Build Analytics
Model
Test relationships among predictors and outcomes
Identify available problem-solving approaches
Select model testing approaches
Examine and evaluate model fit
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 39
![Page 40: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/40.jpg)
• Deploying the model involves interacting with the partners and users who will use the model directly or the results of the model
• Requires careful planning so everyone knows their roles and how to apply results
Deploy Results
Problem Framing
Analytics Problem Framing
DataProtocols
Method Selection
Build Analytics
Model
Project Life Cycle Management
Step 6
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 40
![Page 41: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/41.jpg)
Deploy Results
Appl
ying
Mod
el R
esul
ts
to M
ake
a Di
ffere
nce
Ensure results are usable
Create an infrastructure for implementation
Deliver report with the findings
Support dissemination
Support ongoing data use
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences
Deploy Results
41
![Page 42: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/42.jpg)
Is the answer tied to the original question?
Has the context changed?
Is there integrity in the reporting?
Has a peer-review been conducted?
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences
Deploy Results
42
Double Check Solution
![Page 43: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/43.jpg)
• Analytical models will go through a lifecycle from conception and justification through the model building and deployment
• A good lifecycle process helps to keep this process orderly, minimizes the cost and efforts, and provides the users with clear roles within the lifecycle
Problem Framing
Analytics Problem Framing
DataProtocols
Method Selection
Build Analytics
Model
Project Life Cycle Management
Step 7
Deploy Results
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 43
![Page 44: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/44.jpg)
Project Life Cycle Management
Document the Process
Ensure Results are Usable
Recalibrate, Update, Maintain
Provide End-User Training
Continually Evaluate Outcomes
Certified Analytics Professional. (2014). INFORMS study guide. Catonsville, MD: Institute for Operations Research and Management Sciences 44
![Page 45: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/45.jpg)
Project Life Cycle Management: Write it Down
• For the model to be trusted it has to be repeatable
• Document: • Key assumptions made about the organizational context • Assumptions about the analytics problem• Data sources and data schema• Methods used to clean and harmonize the data• Model approach and selection choices • Documentation for any software code written• Recommendations for future improvements to the model Project Life
Cycle Management
45
![Page 46: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/46.jpg)
Follow-Up on Solutions
• Communication should occur throughout the project lifecycle (before, during, and after) to evaluate:
• Progress toward objectives
• Changes to objectives and business
• Value of findings
• Updates to the data, model, or deployment
Project Life Cycle
Management
46
![Page 47: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/47.jpg)
1Differentiate
between metrics and analytics
2Examine
relationship between data
analytics problem-solving
3Look at analytics as a resource to
improve decision-making
4Identify key concepts or
data needed to deliver
actionable results
Session Objectives
47
![Page 48: Building a Blueprint for Using Data more Effectively](https://reader033.vdocument.in/reader033/viewer/2022042201/6258666b2da33a6a24098879/html5/thumbnails/48.jpg)
Jennifer Wilson-KearseVela Institute [email protected]
Shawn Bergman, Ph.D.Appalachian State: [email protected] Institute: [email protected]
48