the couples satisfaction index (csi) ronald d. rogge asst. professor of psychology university of...

The Couples Satisfaction Index (CSI)

Ronald D. RoggeAsst. Professor of Psychology

University of Rochester

rogge@psych.rochester.eduwww.couples-research.com

Overview

PART 1: Development of CSI Existing scales Development of new scale Cross-sectional validation Longitudinal validation

PART 2: Use & Interpretation of CSI Administration Scoring Interpretation Norms

PART 1: Existing Scales

Strengths20-30 years of converging results Clearly measure satisfaction

Limitations20-30 years oldHeterogeneous contentUnknown noise

Existing Scales

Scale Items Name Cit. Cit./Yr

DAS 32 Dyadic Adjustment Scale 2,191 77.1

MAT 15* Marital Adjustment Test 1,489 32.1

QMI 6 Quality of Marriage Index 221 9.9

RAS 7 Relationship Assessment Scale 156 8.8

Evaluating Scales

Item Response Theory

Used to create SAT, GRE, MCAT

Item by item analysis• If happy, higher responses?• If unhappy, lower responses?

Requires large samples• Estimates parameters for each item• Estimates parameter for each subject

Sample-Independent Results

DAS-31 (Please indicate the degree of happiness, all things considered, of your relationship.)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

babilit

y o

f each r

esponse .

1 - Extremely Unhappy

2 - Fairly Unhappy

3 - A little unhappy

4 - Happy

5 - Very Happy

6 - Extremely Happy

7 - Perfect

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-3 -2 -1 0 1 2 3

Satisfaction

Info

rmati

on (th

eta

) .

DAS/MAT 5Agreement on: FRIENDS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

babilit

y o

f each r

esponse . 1 - Always Disagree

2 - Almost Always Disagree

3 - Frequently Disagree

4 - Occasionally Disagree

5 - Almost Always Agree

6 - Always Agree

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-3 -2 -1 0 1 2 3

Satisfaction

Info

rmati

on (th

eta

) .

Study 1: Goals

Evaluate current scales

DAS, MAT, QMI, RAS IRT in large sample

Develop CSI

Large item pool Factor analysis IRT

Study 1: Method

Online survey (N = 5,315)

Contents141 satisfaction items

Items from DAS, MAT, QMI, RAS 71 additional items

7 anchor scales e.g., neuroticism, hostile conflict, stress

2 validity scales

Study 1: Sample

Avg 26yo (SD=10yr)26% High School or less83% Female76% CaucasianRelationships

24% Married (avg 6.3yrs)16% Engaged60% Committed dating

Relationship Quality

Sample Size(N)

Length of relationship

Satisfaction (DAS)

Married 1254 9.0 yrs 108

Engaged 866 3.1 yrs 117

Dating 3194 1.7 yrs 113

Evaluating Previous Scales

IRT results

Evaluated 66 items of existing scales

Some very informative items

Many poor items

DAS-31 (Please indicate the degree of happiness, all things considered, of your relationship.)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

babilit

y o

f each r

esponse .

1 - Extremely Unhappy

2 - Fairly Unhappy

3 - A little unhappy

4 - Happy

5 - Very Happy

6 - Extremely Happy

7 - Perfect

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-3 -2 -1 0 1 2 3

Satisfaction

Info

rmati

on (th

eta

) .

Standard Deviation (SD) Units Standard Deviation (SD) Units

QMI-1We have a good relationship

0

0.5

1

1.5

2

2.5

3

3.5

4

-3 -2 -1 0 1 2 3

Satisfaction

Info

rma

tio

n (

the

ta)

.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

ba

bil

ity o

f e

ac

h r

esp

on

se

.

1 Very Strong Disagreement

2

3

4

5

6

7 Very Strong Agreement

SMD-2BAD 1 – 2 – 3 – 4 – 5 – 6 GOOD

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

babilit

y o

f each r

esponse .

1 - BAD

2

3

4

5

6 - GOOD

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-3 -2 -1 0 1 2 3

Satisfaction

Info

rmati

on (th

eta

) .

DAS/MAT 5Agreement on: FRIENDS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

babilit

y o

f each r

esponse . 1 - Always Disagree

2 - Almost Always Disagree

3 - Frequently Disagree

4 - Occasionally Disagree

5 - Almost Always Agree

6 - Always Agree

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-3 -2 -1 0 1 2 3

Satisfaction

Info

rmati

on (th

eta

) .

DAS/MAT 6Agreement on: SEX RELATIONS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

babilit

y o

f each r

esponse .

1 - Always Disagree

2 - Almost Always Disagree

3 - Frequently Disagree

4 - Occasionally Disagree

5 - Almost Always Agree

6 - Always Agree

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-3 -2 -1 0 1 2 3

Satisfaction

Info

rmati

on (th

eta

) .

DAS/MAT 9 Agreement on: WAYS OF DEALING WITH PARENTS OR IN-LAWS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

babilit

y o

f each r

esponse .

1 - Always Disagree

2 - Almost Always Disagree

3 - Frequently Disagree

4 - Occasionally Disagree

5 - Almost Always Agree

6 - Always Agree

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-3 -2 -1 0 1 2 3

Satisfaction

Info

rmati

on (th

eta

) .

MAT 12In leisure time, do you (and does your mate) prefer to be “on the go” or to stay at home?

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-3 -2 -1 0 1 2 3

Satisfaction

Pro

babilit

y o

f each r

esponse . 1 - Mismatch

2 - Both on-the-go

3 - Both stay-at-home

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-3 -2 -1 0 1 2 3

Satisfaction

Info

rmati

on (th

eta

) .

From Items to Scales

A scale’s information

= sum of information from each item

How informative

Across different levels of happiness

Scale Information

0

10

20

30

40

50

60

-3 -2 -1 0 1 2 3

Satisfaction

Sc

ale

In

form

ati

on

DAS (32)

QMI (6)

RAS (7)

MAT (16)

DAS-4 (4)

Summary

MAT and DAS have poor items

Increases NOISE

MAT-15 no better than 4-item scale

DAS-32 little better than 6-item scale

Assess satisfaction, but not very efficiently

Poor thermometers

Creating the CSI

141 item pool

Screen for contaminating items

Screen for redundant items

IRT on remaining 66 items

Select 32 most effective

Parameter Invariance

RANDOM SAMPLE HALVES

-5

-4

-3

-2

-1

0

1

2

3

4

5

-5 -4 -3 -2 -1 0 1 2 3 4 5

b's est in even-row subjects

b's

est

in o

dd-r

ow

subje

cts

.

r = 0.998

MALE vs. FEMALE

-7

-6

-5

-4

-3

-2

-1

0

1

2

3

4

5

6

7

-7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7

b's est in MALE respondents

b's

est

in F

EM

ALE r

espondents

.

r = .991

Basic Psychometrics

AlphaDistress

Cut Score

Correlations

1 2 3 4 5 6

1. DAS .94 97.5 -- -- -- -- -- --

2. MAT .84 95.5 .90 -- -- -- -- --

3. QMI .96 24.5 .85 .87 -- -- -- --

4. RAS .92 23.5 .86 .87 .91 -- -- --

5. CSI-32 .98 104.5 .91 .91 .94 .96 -- --

6. CSI-16 .98 51.5 .89 .90 .96 .95 .98 --

7. CSI-4 .94 13.5 .87 .88 .93 .94 .97 .97

Correlations with Anchors

Thoughts of Breakup

Positive Communication

StressHostile Conflict

Sexual Chemistry

Neuroticism

DAS -.74 .73 -.53 -.54 .42 -.40

MAT -.74 .69 -.49 -.49 .41 -.38

CSI-32 -.78 .71 -.52 -.48 .45 -.38

CSI-16 -.78 .71 -.53 -.49 .43 -.38

CSI-4 -.75 .69 -.52 -.47 .41 -.36

Criterion Validity

DAS Distress groups Current gold-standard

DAS score < 97.5 1027 DAS distressed P’s

ROC’s to identify CSI cut scores Identified CSI distressed P’s

91% agreement w/ DAS

Summary

Operate similar across Male vs. Female Older vs. Younger Married vs. Engaged vs. Dating

CSI measures same construct Nearly identical correlations Highly similar screen for distress

Evaluating Possible Improvement CSI-32 vs. DAS-32 CSI-16 vs. MAT-15 CSI-4 vs. DAS-4

More information? Less noise? Better thermometer?

Scale Information

0

10

20

30

40

50

60

-3 -2 -1 0 1 2 3Satisfaction

Info

rma

tio

n

CSI-32

CSI-16

CSI-4

DAS-32

MAT-15

DAS-4

Relative Efficacies

0

1

2

3

4

5

6

7

-3 -2 -1 0 1 2 3

Satisfaction (SD's)

CSI-16 vs MAT

CSI-32 vs DAS

CSI-4 vs DAS-4

Eff

ecti

ve L

eng

th

Satisfaction Groups

IRT satisfaction estimates For each subject Based on MAT, DAS, & CSI items

(equivalent of SAT scores)

Created satisfaction groups N = 265 in each group Levels of sat. HIGHLY similar within each group

MAT, DAS & CSI scores also similar?

Precision: CSI-32 vs. DAS

Precision: CSI-16 vs. MAT

Effect Size

Ability to detect difference Between groups Pre – Post

Effect Size = M1 – M2 . pooled SD

Difference in SD units

Power for detecting ’s in SAT groups

Power: CSI-32 vs. DAS

0

0.5

1

1.5

2

2.5

Effe

ct S

izes

(C

ohen

's d

)

.

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Adjacent Satisfaction Group Contrasts

DAS

CSI(32)

Power: CSI-16 vs. MAT

0

0.5

1

1.5

2

2.5

Effec

t S

izes

(C

ohen

's d

) .

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Adjacent Satisfaction Group Contrasts

MAT

CSI(16)

Conclusions

CSI scales More information Less noise More power

Better thermometers

NEXT STEP True over time? Better at detecting change?

Studies 2, 3, 4: Method

Study 2 596 online respondents 1 and 2 week follow ups (n = 267) CSI, MAT, DAS

Study 3 398 online respondents 1, 2, 3, 4, 6 and 12 mo follow ups (n = 156) CSI, MAT, DAS

Study 4 1,062 online respondents 1, 2, 3, 4, 6 and 12 mo follow ups (n = 545) CSI, MAT

Studies 2-4: Demographics

SAMPLE N = 2,056 initial respondents N = 968 (47%) respondents with longitudinal data

AGE M = 27.7yo (9.3yrs)

GENDER 71% Female 29% Male

RACE 83% Caucasian 5% Asian 4% African American 4% Latino

SES 10% High school diploma or less 25K avg yearly income

Studies 2-4: Relationships

Relationship Types 37% Married: 7.9 yrs (7.9 yrs) 13% Engaged: 3.2 yrs (2.4 yrs) 50% Dating: 1.8 yrs (1.9 yrs)

Relationship Satisfaction (MAT) Married: 108 (32) Engaged: 122 (24) Dating: 116 (24)

Dissatisfied Respondents 24% (n = 487)

Change Criterion

How much has each of these changed? Overall happiness in the relationship Feeling close and connected Stability of the relationship

Averaged responses Alpha = .92

Agree with MAT, DAS, & CSI scores?

MuchWORSE

SomewhatWORSE

A littleWORSE

Stayed the SAME

A little BETTER

Somewhat BETTER

Much BETTER

-3 -2 -1 0 +1 +2 +3

Noise over time (SERM)

Score scatter in “no change” group

238 “no change” at 1st assessment

Repeated Measures MANOVA

Scatter (noise) in scale scores across time

SERM = 2*MSE

MuchWORSE

SomewhatWORSE

A littleWORSE

Stayed the SAME

A little BETTER

Somewhat BETTER

Much BETTER

-3 -2 -1 0 +1 +2 +3

Detecting Individual Change

Can we detect individual change? Minimal Detectible Change (MDC95)

• RCI: Jacobson & Truax (1991)

• MDC95: Stratford et al. (1996)

Pre-Post score change• In one individual• Necessary to exceed noise

MDC95 (SD units) = 1.96*SERM .

SD

Minimum Detectible Change

How much must an individual’s score shift to show significant change?

00.10.20.30.40.50.60.70.8

MD

C's

in

SD

Un

its

CSI-32 CSI-16 CSI-4 DAS MAT

AB

C*C

C*

Detecting Individual Change

CSI scales more sensitiveRequired smaller pre-post score shifts

Longer scales more sensitiveCSI-32 > CSI-16 > CSI-4

MAT & DAS not as sensitiveOperated no better than CSI-4

Detecting Group Differences

Can we detect clinically distinct groups? Improved vs. No-change Deteriorated vs. No-change

Minimal Clinically Important Difference (MCID)• Guyatt, Walter & Norman (1987)

MCID Effect Size = M(improved) – M(no change)

Noise over time (SERM)

HLM framework• Global change predicting scores on scales• 2,475 points of change from 968 respondents

• Improved vs. Deteriorated• Satisfied vs. Dissatisfied• Gender effects

MCID Effect Sizes

How well can we detect naturally occurring change?

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Eff

ec

t S

ize

to

De

tec

t 1

Po

int

of

Glo

ba

l C

ha

ng

e

Deterioration Improvement Deterioration Improvement

CSI-32

CSI-16

CSI-4

DAS

MAT

Dissatisfied Respondents Satisfied Respondents

A A

B

CD

A B

C C C

A B

C

D

E

A A B B B

Differences by Gender

Scales showed slightly smaller effect sizes in men

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

Re

du

ctio

n in

Effe

ct S

ize

s

CSI-32 CSI-16 CSI-4 DAS MAT

* *

**

Detecting Group Differences

CSI-32 & CSI-16 Out performed DAS & MAT

• Improvement / Deterioration• Satisfied / Dissatisfied

CSI-4 Deterioration: Out performed DAS & MAT Improvement: Equivalent to DAS & MAT

Weak gender effect All scales slightly less responsive in males

Summary of Development

CSI scales represent improved thermometers

Developed with IRT / FA No contaminating items Non-redundant items Most informative items

Still measure satisfaction Consistent with MAT / DAS

Offer greater power More information Less noise

More sensitive cross-sectionally Detecting group differences

More responsive over time Detecting change in a single individual Detecting differences between clinical groups

PART 2: Administration

See CSI handout

Spouses complete separately No discussion during administration Want unique perspectives

Inform of confidentiality limits Feedback given? Dyadic or individual feedback? Normative data

Should take 3-4 minutes

Scoring

See CSI scoring handout

Sum the item responses10 reverse scored items

• High sat options offered first (items 2-5)• Reversed wording (items 10, 15…)

Total scores Range from 0-161

Interpretation

Box Plots

Dissatisfaction Cut ScoreScores below 104.5

MedianLowest 25% of scores 2nd quartile

of scores3rd quartile of scores

Highest 25% of scores

Norms in Dating Individuals

N = 1477

N = 2191

N = 415

Norms in Engaged Individuals

N = 551

N = 141

Norms in Married Individuals

N = 1129

N = 735

Norms in Married Individuals

N = 271

N = 321