Improvement Coach Professional Development ProgramVirtual Workshop 2
February 22, 201712-3 PM Eastern
Agenda2
Time Item Who
12:00 PM Welcome and warm up Linson
12:10 PM Tools spotlight: Tree diagrams Phyllis
12:35 PM Transition to WebEx breakouts All
12:40 PM Breakouts: WIP presentations Bill, Phyllis,
Christina, Karen
facilitate
1:40 PM Break and transition back to main room All
1:55 PM Tools spotlight: Scatter plots and surveys Phyllis, Karen
2:50 PM Next steps and evaluation Linson
3:00 PM Adjourn All
WebEx quick reference
• Please use chat to “All Participants” for questions
• For technology issues only, please chat to “Host”
• WebEx Technical Support: 866-569-3239
• Dial-in Info: Communicate, Join Teleconference (in menu)
3
Raise your hand
Select Chat recipient
Enter Text
You should see a small phone icon or a
headset icon next to your name.
WebEx connection
When chatting…
Please send your message to
All Participants
5
Please mute your line
• Please keep your line
on mute when you are
speaking to minimize
background noise
• You can mute or
unmute using *6 on
your phone
6
Who’s on the line with us today…
Phyllis Virgil Bill Peters Karen Baldoza
Christina
Gunther-Murphy
Linson Naval
Warm-up chat
• In the chat box, please chat to All
Participants...
Your favorite Spring activity!
8
Tools Spotlight: Tree DiagramsPhyllis
What is a tree diagram?
Tool can be used to organize and understand the structure and content of any problem, issue or objective
Diagram resembles a tree when completed
Helps teams think systematically, logically and comprehensively
Each branch is exploded and explained
Allows an easy to see graphic view of increasing detail
Moves thinking step by step from the general to the specific
Can be developed horizontally or vertically
Also called: systematic diagram, tree analysis, analytical tree, or hierarchy diagram
10
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement, API by Phyllis M. Virgil
1. Problem Solving Tree (Why)
2. Planning Tree (How)
3. Analysis Tree (What)
4. Mixed
Types of Tree Diagrams11
Phyllis M. Virgil as adapted From: http://asq.org/learn-about-quality/new-management-planning-tools/overview/tree-diagram.html
12Readmissions Problem Solving (Why)
Source: Phyllis M. Virgil
Strategic Planning for Healthy Weight (How)
Source: Phyllis M. Virgil
Work Breakdown Structure Tree (How)
Source: http://www.criticaltools.com/wbschartprosoftware.htm
Used Extensively by Project Management Professionals
What does that mean?
What does that mean?
What does that mean?
What does that mean?
Customer Requirement (VOC)
Good Cup of Coffee
Hot
Temp between 91° and 96°
Tasteful
Flavorful
Perfect Brew
Strength
Golden Ratio 17.42 units of
water to 1 unit of coffee
Not Bitter
PH of 5
Quality Characteristic Analysis Tree (What)
Source: Phyllis M. Virgil, Strategic Quality Planning Workshop
Six Sigma’s DMAIC
Critical To Quality Tree (What)
Source: http://isoconsultantpune.com/voc-data-collecting-tools/
Driver Diagram is a Tree that Maps Out Your Theory (Mixed)
Source http://www.ihi.org/resources/Pages/Publications/WhatsYourTheoryDriverDiagrams.aspx
Whiteboard Video: Intro to Driver Diagrams18
http://www.ihi.org/education/IHIOpenSchool/resources/Pages/AudioandVideo/Whiteboard9.aspx
New Driver Diagram Paper
http://www.ihi.org/resources/Pages/Publications/WhatsYourT
heoryDriverDiagrams.aspx
Benefits of Tree Diagrams
• Makes it less likely essential items will be left out
• Encourages discussion and dialogue
• Improves communication and presentations
• Versatile and easy to use
20
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
Use of Tree Diagrams
A tree diagram can be used in all phases of
improvement work:
Break project down into sub-objectives
Uncover the causes of problems
Understand customers needs
Develop our measure of change
Identify change ideas
Detail out action plans
Adapted by Phyllis M. Virgil from: The Improvement Handbook: Model, Methods, and Tools for Improvement
What are we trying toaccomplish?
How will we know that achange is an improvement?
What change can we make thatwill result in improvement?
Model for Improvement
PlanAct
DoStudy
API
How to Develop a Tree Diagram
1. Clearly state the problem, opportunity, objective or issue of interest. Place
this as the first box at the far right/left or top of your work space (page, flip
chart, white board)
2. Ask questions of Why, What or How to develop the next level of detail.
Place each answer in a box (or on a line) at next level and link them by
drawing a line.
Problem Solving -- Why
Planning -- How
Analysis -- What
3. Repeat step 2 process until a useful level of detail is achieved.
Adapted by Phyllis M. Virgil from Minnesota Department of Health QI Tool Box http://www.health.state.mn.us/divs/opi/qi/toolbox/print/tree.pdf
Questions to Ask
– Planning Tree: “How can this be accomplished?” or
“What Tasks must be done to accomplish this?”
– Problem Solving Tree: “Why does this happen?” or
“What causes this?”
– Analysis Tree: “What are the components?” “What
goes into it?” “What drives this?” “What comes next?”
– For unclassified tree diagrams ask whatever
question is appropriate, What, Why, How...
Adapted by Phyllis M. Virgil from: http://asq.org/learn-about-quality/new-management-planning-tools/overview/tree-diagram.html
23
Tips for Constructing Tree Diagrams
Sticky notes are often useful to use when developing tree diagrams.
Best when done by a small team who has detailed knowledge
Develop by brainstorming, clumping, labeling and sequencing ideas
Another way is to logically and sequentially move from level to level
Measures, dates and responsibility can also be added diagrams
Typically taken to third level of detail
Use nested diagrams for more detail
Do a “necessary, valid and sufficient” check
Adapted from ASQ and Minnesota Department of Health
Use SmartArt to Create in MS Word or PowerPoint
• Go to Insert Tab > SmartArt > Hierarchy
• Use Design and Format tabs to create and format.
Content Design: Phyllis M. Virgil
25
Summary
Tree Diagrams are versatile and easy to use.
They help you to think systematically, logically and
comprehensively about any problem, issue or objective
They can be used in any phase of improvement work.
Content Design: Phyllis M. Virgil
26
Appendix: Six More Examples
1. CRX Problem Solving Tree (Problem Solving)
2. Healthy Weight Action Plan Tree (Planning)
3. Customer Service CTQ Tree (Analysis)
4. Decreasing Fuel Costs (Driver Diagram)
5. Behavior Health Primary Care Integration (Driver Diagram)
6. Grocery Store Customer Satisfaction (Driver Diagram)
28
1991 CRX will not start
Dead Battery
Fluid Levels Low
did not fill Did not check
Would not charge
Generator not working
Solenoid broken
Left Lights onUsed lights during day
Did not drive in 3 months
Too busy to drive it
No Gas
Fuel Line broken
Rubber hose worn
Rubber hose old
Gas filter clogged
Not changed in 3 years
Not on maintenance
schedule
Gas tank empty
Fuel gage broken
Forgot to reset
odometer
Starter malfunction
Wires not connected
Vibration caused
disconnect
Parts WornFriction and
Age
Engine Malfunction
CarburetorChoke
Malfunction
Distributor Timing Off
Problem
Solving
Tree (Why)
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement, API
HW Action Plan Tree (How) nested diagramSee HW Strategy Tree
Source: Phyllis M. Virgil
DMAIC Critical to Quality Tree (What)
Source: http://isoconsultantpune.com/voc-data-collecting-tools/
Basic
Driver
Diagram
Source: Quality Institute, NHS, UK
Complex
Collaborative
Driver Diagram
Source: CCRMC BH/PC Collaborative, 2015
Change Idea Tree (aka Driver Diagram)
WIP PresentationsKaren, Linson
WIP presentations: Purpose
The purpose of the Works in Progress (WIP) sharing is to provide an environment that enables Improvement Coaches to:
• Lead effective two-way communication about coaching and improvement efforts
• Elicit useful feedback to accelerate personal skill building and project gains, and in turn, provide coaching to fellow Improvement Coaches
• Stimulate peer-to-peer learning and creative ideas about coaching and improvement among Improvement Coaches and faculty.
35
WIP presentations: Format
• For each presentation, the presenter-coach will have
12 minutes as follows:
– 7 minute uninterrupted presentation
– 5 minute discussion that the presenter-coach facilitates
• All should speak up or use chat to provide feedback,
ask questions, etc.
• We will capture the chat so presenter-coaches have
the notes and can concentrate on the discussion
• Faculty will keep time and send time checks via the
chat
36
WIP presentations: Timing
Time (Eastern) Presentation
12:35 PM-12:40 PM Transition and get started
12:40 PM-12:55 PM Presentation 1
12:55 PM-1:10 PM Presentation 2
1:10 PM-1:25 PM Presentation 3
1:25 PM-1:40 PM Presentation 4
1:40 PM-1:55 PM Break and transition back to the main room
37
WebEx breakouts
• It is imperative that everyone be connected correctly to
fully participate in WebEx Breakouts for the Project
Presentations
• When you join WebEx, wait until the Pop Up window
comes up and gives you the number to dial, session
number, and attendee ID#.
• You need to dial all of the numbers to connect
completely.
You should see a small phone icon or a
headset icon next to your name.
Today we will use WebEx Breakouts for the
Project Presentations
• When we are ready to start the
Breakouts, you will see a pop-up
window that invites you to join.
Please click YES when it pops up.
• You will then see a pop-up
window that tells you that your
audio line will automatically be
transferred to the Breakout. Click
YES again.
Breakouts
• Once you are in your virtual breakout room, Karen, Bill, Christina, or Phyllis will help facilitate the project presentations. Linson will load the slides for each breakout room.
• You will be able to speak, raise your virtual WebEx hand, and use the chat box while you are in the breakout room.
• Linson will send message to provide a 5-minute warning when we are nearing the end of our breakout sessions.
• Five minutes later, Linson will send a message that requests that you return to the main session.
• Do not click refresh or back- sometimes it takes a few seconds for the breakouts to begin and end- please be patient!
Transition to Project Presentation Breakouts
15 minute break
Tools Spotlight: Scatter Plots
A tool to understand relationshipsPhyllis Virgil
47
What is a Scatter Plot?
A scatter plot is a graphic representation of the association between
two variables (X and Y)
It is a dot plot of paired data
Which helps to test the direction and strength of possible
relationships
Also known as Scatter Diagram, or X-Y Graph
Content Design: Phy llis M. Virgil
48
When do you use a Scatter Plot?
Use to test hunches and theories about relationships
In improvement work they can help us
1. Select and focus improvement projects
2. Verify relationship of process and
outcome measures
3. Target areas for change
Content Design Phy llis M. Virgil
49
What are we trying toaccomplish?
How will we know that achange is an improvement?
What change can we make thatwill result in improvement?
Model for Improvement
PlanAct
DoStudy
API
Independent Variable X Y Dependent Variable
Positive relationship > upward slope > as X increases Y increase
Negative relationship > downward slope > as X increases Y decreases
No relationship > no slope up or down
X = Ave. Temp.
Y=
Ele
ctr
ic B
ill
Positive Relationship
X = Weight of Car
Y=
Mile
s p
er
Gallo
n
NegativeRelationship
X = Shoe SizeY
= IQ
Te
st S
co
re
No Relationship
Direction of the Relationship
Positive or Negative
upward slope > as
Content Design: Phyllis M. Virgil
Images courtesy of http://sixsigmatraining.com
50
X Y
Strong relationship > tight clustering of dots
Mild or Weak relationship > loose clustering of dots
X = Miles to Destination
Y=
Air
Fare
Mild Relationship
X = Waiting Time
Y=
Cu
sto
me
r
Sati
sfa
cti
on
Strong Relationship
Strength of the Relationship
Strong or Weak
51
Content Design: Phyllis M. Virgil
Images courtesy of http://sixsigmatraining.com
More Complex Non Linear Scatter Plots
Stopping Distance by Speed
Fuel Used by SpeedReading Score by
Hours of Sleep
Examples of Curvilinear & Quadratic (Parabola) Scatter Plots
Stopping Distance by Speed : http://athometuition.com/modelling-bivariate.aspxReading Scores and Sleep: https://www.aea267.k12.ia.us/assessment/statistics/descriptive-statistics/correlation/curvilinear-relationship/ Anxiety and Performance : https://webcourses.ucf.edu/courses/950845/pages/the-correlational-research-strategyFuel Used by Speed: http://bolt.mph.ufl.edu/6050-6052/unit-1/case-q-q/linear-relationships/
How to Construct a Scatter Plot
Gather paired data from the same observation unit (time, person, place, thing)
By hand = scale and label x and y axis, plot dot of intersection for each paired valued, circle duplicates
By Excel = Highlight data > Insert Tab > Select Scatter Plot
X Y
Date# Patients
Late
Ave. Waiting
Time
4-Apr 3 15
5-Apr 4 17
6-Apr 5 20
7-Apr 6 29
8-Apr 6 32
11-Apr 6 36
12-Apr 4 25
13-Apr 3 10
14-Apr 2 7
15-Apr 5 30
18-Apr 5 30
0
5
10
15
20
25
30
35
40
0 1 2 3 4 5 6 7
Ave. Waiting Time
# Patients Late
Ave. Waiting Time by # Patients Late
53
Source: Phyllis M. Virgil
Trend lines can be added...
Eyeball Method = draw line in middle of dots (half above, half below)
Excel = Grab Graph > Layout Tab > Select Trendline.
0
5
10
15
20
25
30
35
40
2 3 4 5 6 7
Ave Waiting Time
# Patients Late
Ave Waiting Time by # Patients Late
54
Source: Phyllis M. Virgil
Select and Focus Improvement Projects
1.0
2.0
3.0
4.0
5.0
6.0
7.0
0 20 40 60
Ave. Pat Satisfaction
Ave. Weekly Wait Time
Patient Satisfaction by Wait Time
Finding: Fairly Strong negative relationship between wait time and patient satisfaction.
Conclusion: Working on decreasing wait time makes sense.
55
Source: Phyllis M. Virgil
Use after C/E Brainstorming
To Confirm Change Ideas Before Testing
Finding: Fairly Strong positive relationship between wait time and # of patient late.
Conclusion: Testing change ideas aimed reducing # late patients makes sense.
0
5
10
15
20
25
30
35
40
2 3 4 5 6 7
Ave Waiting
Time
# Patients Late
Ave Waiting Time by # Patients Late
C/E Image Credit: https://www.nursingtimes.net/Journals/2013/04/12/k/x/z/Using-fishbone-analysis--to-investigate-problems-160413.pdf
Content Source: Phyllis M. Virgil
Beware of Spurious (false) Scatter Plots57
Spurious scatter plot is often a result of a third factor that is not apparent at the
time of examination. Spurious comes from the Latin word spurious, which
means illegitimate or false.
Killer Ice Cream?
Content Design: Phyllis M. Virgil
Scatter Plot Source: http://tellingthestory.typepad.com/telling_the_story/2010/06/
Study shows killers love ice cream!
Conclusion: Think before you believe.
Scatter Plots are about relationships!
Scatter plots do not prove anything!
Rather they provide “evidence”
They help you:
Understand relationships
Understand the direction and strength of the relationships
Scatter Plots in Action
59
Physical Function by Steps per Day
Content Design and Conclusion: Phyllis M. Virgil
Content and Image Source http://www.rehab.research.va.gov/jour/04/41/4/smith.html
Shows No Relationship
Conclusion: Think twice about focusing on increasing steps per day
as a means to improve physical function.
60
Service Quality Rating and Patient Safety
Content Design and Conclusion by Phyllis M. Virgil
Content and Image Source by http://www.psqh.com/enews/1107feature.html
Shows a Very Moderate Positive Relationship
Conclusion: Patient Safety is not a driver of satisfaction,
but likely a Prerequisite. More study is required.
61
Patient Satisfaction by Nurse Contact Time
62
Content Design and Conclusion by Phyllis M. Virgil
Content and Image Source http://www.qihub.scot.nhs.uk/knowledge-centre/quality-improvement-tools/scatter-plot.aspx
Shows a Moderate Positive Relationship
Conclusion: Looks like 10-20 minutes is a sweet spot.
Do more investigation.
Sick Days to Case Load
Scatter sick days to case load
Scattergram
30 35 40 45 50 55 60 65
Case Load (X)
0
2
4
6
8
10
12
14
16
Days S
ick (
Y)
Scatter sick days to case load
Scattergram
30 35 40 45 50 55 60 65
Case Load (X)
0
2
4
6
8
10
12
14
16
Days S
ick (
Y)
Sick Days to Case Load
Shows Moderate, Curvilinear
Positive Relationship
63
Source: The Health Care Data Guide: Learning from Data for Improv ement By Lloyd P. Provost, Sandra Murray
Sick Days to Case Load
by Department
64
Case Load Vs Sick Days Total
30 35 40 45 50 55 60 65Case Load (X)
0
2
4
6
8
10
12
14
16
Days S
ick (
Y)
Case Load Vs Sick Days Dept A
40 45 50 55 60 65Case Load (X)
0
2
4
6
8
10
12
14
16
Days S
ick (
Y)
Case Load Vs Sick Days Dept B
30 35 40 45 50 55 60 65Case Load (X)
0
2
4
6
8
10
12
14
16
Days S
ick (
Y)
Case Load Vs Sick Days Dept C
35 40 45 50 55 60Case Load (X)
0
2
4
6
8
10
12
14
16
Days S
ick (
Y)
Dept. A
Shows a Positive, Moderate
Curvilinear Relationship
Dept. C
Shows No Relationship
Dept. B
Shows No Relationship
Conclusion, focus improvement effort only on Department A
and do more investigation
Total
Shows a Positive, Moderate
Curvilinear Relationship
Source: The Health Care Data Guide: Learning from Data for Improv ement By Lloyd P. Provost, Sandra Murray
Nonsense (or spurious) Scatter Plots65
450
500
550
600
650
700
750
9 9.5 10 10.5 11 11.5
Civil Engineering
Degrees Awarded
Mozzarella Cheeze Consumption Per Capata in Pounds
9.2
9.4
9.6
9.8
10
10.2
10.4
10.6
10.8
11
11.2
450 550 650 750
US Per Capita
Mozzarella Cheeze
Consumption in Pounds
Civil Engineering Doctorate Degrees US
Content Design and Conclusion by Phyllis M. Virgil
Content and Image Source by http://tylervigen.com/old-version.html
Conclusion: just because any two variables can be
put on a scatter plot, does not mean it makes sense.
Use rational judgment when discerning associations.
Mozzarella Cheese Consumption and Civil Engineering Degrees
Show a Strong Positive Relationship.
Correlation Coefficient and Regression Lines
More advanced applications of the Scatter Plot include the use of correlation coefficients (R) and regression lines
R Indicates how closely observed values fall around regression line – both the clustering about line & direction of association.
R standardizes the degree of association, regardless of units of measurement
Ranges between -1 and 1
Direction of the relationship indicated by a + or -
positive r = positive relationship
negative r = negative relationship
0 = no relationship
General guide for interpreting strength of the relationship R (absolute value)
0 - .2 = weak, slight
.2 - .4 = mild
.4 - .6 = moderate
.6 - .8 = strong
.8 - 1.0= very strong
In Excel select graph > Layout Tab > Trendline > More Options > Display R squared value on chart
Content Design: Phy llis M. Virgil
66
Review of Scatter Plot Patterns67
No Relationship
Scattered
No Relationship
Horizontal Cluster
Quadratic
Relationship
Curvilinear
Relationship
Strong Positive
Relationship
Strong Negative
Relationship
Weak Positive
relationship
Weak Negative
relationship
Images from: http://slideplayer.com/slide/4682680/, as published by Justin Jacobs
Summary
Scatter plots test association between two variables
They tell us if there is “evidence” of a relationship
Direction: is it positive, negative, curvilinear, or quadratic?
Strength: is it strong or weak (tight or loose clustering of dots)?
Can use them in first three questions MFI
Focus aim
Select measures
Target areas for change
Content Design: Phy llis M. Virgil
68
Tools spotlight: SurveysTools and methods to gather informationKaren
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
Reference: The Improvement Guide, page 417
Session objective
• Summarize how surveys can be used to gather
information and for data collection to enhance
your improvement effort
Session agenda
Topic Time
Overview 5 minutes
Written surveys 5 minutes
Developing and using surveys, analyzing the results 5 minutes
Kano method 5 minutes
Chat in
• Think about the last survey you participated in.
• Chat in your answers to the following:
– What was it?
– What did you like about the survey?
– What didn’t you like about it?
72
Surveys (as we’ll define it here)
• A method of collecting information directly from people about their feelings, motivations, plans, beliefs, experiences, and backgrounds
• In relation to the Model for Improvement and PDSA cycles, a survey is a type of data collection process to answer a question(s) posed in Question 2 and the planning phase of the cycle
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
73
Reasons to survey
• To build the knowledge of those obtaining the
information (duh!)
• To obtain a desired effect on the relationship
between those obtaining the information and
those providing the information
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
74
Methods for obtaining information
• Informal conversation
• Written surveys
• Personal interviews
• Group interviews
• Observations
• Trading places
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
75
Considerations:
• Purpose
• Reliability and validity
• Usefulness and credibility of results
• Cost
• Anonymity
• Convenience
• Complexity of information
• Time – to administer and respond
• What you will do with the information
Written survey example76
Thank you for participating in our workshop today. Please help us improve our service to you by completing this survey.
Please list three ways we can improve our service to you.
1. _____________________________________
2. _____________________________________
3. _____________________________________
Please rate your experience with our services today (circle one)
Poor Fair Good Very Good Excellent
Other suggestions:
_______________________________________________________________
_______________________________________________________________
Written surveys
• Delivery: intentional or opportunistic?
• Design considerations:
– Order of the questions
– Selection of response options
– Wording
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
77
Written surveys
Advantages Disadvantages
Beneficial for those who are better at
written than verbal communication
Respondent has no say about what is
covered
Can be administered to preserve
confidentiality
Usually a large percent do not
complete
Less expensive than interviews Questioner cannot build on answers
Useful in reaching a geographically
diverse group of peopleRespondents have to interpret the
questions
Respondent assumes most of the
work
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
78
Steps for conducting a survey
1. Clarify the purpose of the
survey
2. Consider why a survey is the
best method for obtaining
the desired information
3. What questions are to be
answered by the survey?
4. Decide on the type of survey
(e.g., written, phone,
interview)
5. Select the survey design,
content, questions, and test
the form
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
79
6. Decide who should
participate and when (will
sampling be used?)
7. Will there be follow-up
surveys?
8. Attend to the survey
administer (e.g., collecting
forms, conducting
interviews) (Do!)
9. Analyze, interpret, and
report results (Study!)
10. Act (if appropriate) on the
results
Survey questions
• Consider the respondents’ point of view – how complicated should the questions be and at what level should they be written?
• Open-ended– Offers insight as to why people believe what they do
– Interpretation can be difficult
• Closed-ended– Easy to use, score, and code
– Reliable to the extent they provide uniform data
– Validity usually requires extensive testing
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
80
Closed-ended questions
• Each question should be meaningful to the respondents
• Use standard, simple language
• Make questions concrete
• Avoid biased words and phrases
• Check your own biases
• Don’t get too personal
• Each question should have just one thought (no double-barreled questions)
• Make sure the response options match the question
• Tip: State questions/statements in the extreme
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
81
82Written survey examples
Learning in real time
Hayes, H., Scott, V., Scaccia, J.P., Abraszinskas, M., Stout, S., & Wandersman, A. (under review). A Multi-Method Approach to Evaluating Training
83
Methods for analyzing survey data
• Depends on the type of questions asked
• Closed-ended questions:– Frequency plots (possibly
with stratification)
– If collected over time, run and control charts
• Open-ended questions:– Clustering techniques with
written summaries of the clusters/themes – e.g., affinity diagram or Pareto chart
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
84
0
1
2
3
4
5
6
7
8
9
Poor Fair Good Very good Excellent
Nu
mb
er o
f re
spo
nse
s
Response
Customer Satisfaction
Kano survey method
• Useful for testing new ideas and innovations with
customers to determine if the ideas might
change their expectations (future needs and
wants)
• Incorporates the concept of “attractive” and
“must-be” quality elements
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
85
Kano two-dimensional recognition
mode
86
Customer
Satisfied
Attractive One-dimensional
Not Fulfilled Fulfilled
Must-be
Customer
Dissatisfied
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
Three classifications of quality-characteristics of a produce or service:• One-dimensional Element – gives satisfaction when fulfilled and results in dissatisfaction when
not fulfilled• Attractive Quality Element – give satisfaction when fulfilled but is acceptable when not fulfilled• Must-be Quality Element – take for granted when fulfilled but results in dissatisfaction when
not fulfilled
Method
• Ask questions in pairs
– How would you feel if you had coffee at the
workshop?
– Delighted, I expect it and like it, No feeling, Live with it, Do
not like, Other
– How do you feel when you do not have coffee at the
workshop?
– Delighted, I expect it and like it, No feeling, Live with it, Do
not like, Other
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
87
Two-dimensional chart of survey results for each
quality element
88
Response to “Negative Wording” of Question
Re
spo
nse
to “
Po
sitiv
e
Wo
rdin
g” o
f Q
ue
stio
n
Delighted I expect and like it
No feeling
Live with it
Do not like
Other
Delighted Skeptical Attractive Attractive Attractive One-
dimensional
Other
I expect and like it
Reverse Indifferent Indifferent Indifferent Must-be Other
No feeling Reverse Indifferent Indifferent Indifferent Must-be Other
Live with it Reverse Indifferent Indifferent Indifferent Must-be Other
Do not like Reverse Reverse Reverse Reverse Skeptical Other
Other Other Other Other Other Other Other
• One-dimensional: Satisfied when fulfilled, dissatisfied when not fulfilled
• Attractive: Satisfied when fulfilled, no feeling when not fulfilled
• Must-be: No feeling when fulfilled, dissatisfied when not fulfilled
• Indifferent: No feeling of satisfaction or dissatisfaction regardless of fulfillment
• Reverse: Not expected, dissatisfied when fulfilled or satisfied when not fulfilled
• Skeptical: Doubtful if question is understood; difficult question, poor expression
• Other: None of the responses apply; other consideration
89
Summary Survey Results Example: Trends in Quality Elements
Survey Form Example: Kano
Questionnaire at a Hotel
Next steps and evaluation
• Continue working with your team
• Our next Virtual Workshop is on:
March 2, 2017
12:00 PM – 3:00 PM Eastern
• Please give us your feedback! Chat in:– What you think “went well”
– Improvement suggestions
– Questions you still have that you would like us to address
We really appreciate your feedback and questions!
Thank you!
SurveysAppendix
92
Personal interviews
• Discussion between an interviewer and a respondent to obtain information on a pre-arranged set of topics
• Can be specific questions or set of topics
• Emphasis on a particular topic depends on value of the information being obtained and effect we hope to get
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
93
Personal interviews
Advantages Disadvantages
Respondent can provide input into what will
be discussed
Difficult to administer to people in varied
locations
Interviewer can:
• Check for understanding and clarify
• Change which topics are emphasized
• Pursue fruitful unforeseen areas
• Observe body language
Variation is introduced by multiple
interviewers
Presents an opportunity to build or enhance
relationships
Lacks confidentiality
Less work for interview than a written
survey
Time-consuming for interviewers
Expensive compared to written surveys
Format is not standard
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
94
Group interviews or focus groups
• Like personal interviews
• Discussion between an interviewer and a group
of respondents to obtain information on a pre-
arranged set of topics
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
95
Group interviews
Advantages Disadvantages
Mutual stimulation of the group can
result in more information than
individual interviews
Respondents may be reluctant to
discuss sensitive issues in a group
Fruitful unforeseen areas can be
pursued by the entire group
Interviewer must be more skilled than
for a personal interview
Can help to establish or enhance
relationships between members of the
group
Interactions between members of the
group may suppress or bias
information from one or more
members
Less time consuming than personal
interviews
Expensive if the group is
geographically diverse
May create other agendas for some
members of the group
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
96
Observation
• Observe users of your
product, process,
equipment, performing a
function
• Can be used as the basis for
innovation
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
97
Observation
Advantages Disadvantages
Information obtained under realistic
conditions
The presence of an observer may
create artificial conditions
Observer needs less skill to obtain the
information than an interviewer
Limited to the conditions under which
the observations were made
Useful for generating ideas for
innovation
The use of many products and
services is difficult to observe
Simple May miss important but infrequently
occurring events
Useful for validation of personal or
group interviews
If not careful to be objective, observers
can sometimes see what they want to
see
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
98
Trading places
• Experience the product
or service for yourself
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
99
Trading places
Advantages Disadvantages
Allows the person trading places to
experience feelings that could not be
expressed by respondents in an
interview
Perceptions are obtained from only
one or a few people who assume the
role
Very useful for building or enhancing a
relationship
Presence of the person assuming the
role may create artificial conditions
Important, but minute, level of detail
can be experienced
May need certain skills to assume
someone’s role
May be too short to form a useful
perception
Familiarity with the product or service
may make it impossible to assume the
position of a first time user
Adapted from: The Improvement Handbook: Model, Methods, and Tools for Improvement
100