individual differences in replicated multi-product 2-afc...

Individual differences in replicated multi-product 2-AFCdata with and without supplementary difference scoring:

Comparing Thurstonian mixed regression models forbinary and ordinal data with linear mixed models

Christine Borgen Linander1,∗, Rune Haubo Bojesen Christensen1,Rebecca Evans2, Graham Cleaver2, Per Bruun Brockhoff1

1Department of Informatics and Mathematical Modeling, Section for Statistics, TechnicalUniversity of Denmark

2Unilever Research and Development Port Sunlight, UK*Contact author: [email protected]

| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 1/30 |

Outline

1 Introduction

2 Results

3 Final remarks


Introduction

Experiment

Data:

12 Products - denoted A, B, C, . . . , L

24 Assessors

8 Attributes

5 different attributes - denoted A, B, C, D, E3 of those evaluated again after 5 minutes - denoted C.5m, D.5m, E.5m

2 Sessions leading to replications

Each Product was compared to a control.

Not all combinations of Assessor and Product are present. Henceincomplete data. (Have included Assessors with at least 50% of theobservations)

For each Attribute a maximum of 24 observations for each Assessor.


Introduction

Experiment

Test protocol:

2-AFC protocol

With an additional task to score the difference from ’not different’ to’extremely different’ with 5 categories

The data from the additional task is transformed into a scale from -4 to 4where:

0 corresponds to no difference

negative values are in favor of the control

positive values are in favor of the Product

Thus possibility to consider the response in 3 ways:

Binary with values 0, 1

Quantitative with values -4, 3, . . . , 3, 4

Ordinal with values -4, 3, . . . , 3, 4


Introduction

Aims for the analysis

Analytical aims:

We want to illustrate that it is possible to fit a Thurstonian mixedmodel with the binary response

We want to have the capability to take account of other sources ofvariation when analysing data from the 2-AFC protocol

To be more specific - we want to model the individual assessors

We want to analyse the results from a set of products in one analysisrather than one product comparison at a time

Business oriented aim:

We want to investigate if we gain valuable extra information addingthe additional task to the usual 2-AFC protocol.


Introduction

Models

We consider 2 types of models:

Naive aggregation model - ignoring the replicated structure

Mixed model - including Assessor as a random effect, modelling thereplicated structure

Where the naive model will be fitted with the binary response.

And the mixed model will be fitted with the 3 different responses.


Results

F test - mixed model - Quantitative response

The PanelCheck way by the R-package lmerTest

A B C D E C.5m D.5m E.5m

Assessor−by−Product interaction effects

Attributes

F s

tatis

tic

010

2030

40 0.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001


Results



Assessor main effects

Attributes

F s

tatis

tic

010

2030

4050

0.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001


Results



Product main effects

Attributes

F s

tatis

tic

010

2030

40



Results

Likelihood Ratio test - mixed model - Binary response



Attributes

Like

lihoo

d R

atio

sta

tistic

0.0

0.4

0.8

1.2



Results




Attributes

Like

lihoo

d R

atio

sta

tistic

010

2030

40



Results



Product main effects − mixed model

Attributes

Like

lihoo

d R

atio

sta

tistic

050

100

200



Results

Likelihood Ratio test - naive model - Binary response


Product main effects − naive model

Attributes

Like

lihoo

d R

atio

sta

tistic

050

100

200



Results

Likelihood Ratio test - comparison of Product - Binaryresponse


Mixed modelNaive model

Comparing test statistics for the mixed and naive model

Attributes

Like

lihoo

d R

atio

sta

tistic

050

100

150

200


Results

Comparing the two models for Product - Binary response

Not a big difference in values between the methods. This fits nicelywith the Assessor-by-Product interaction being non-significant

In general you would expect the values from the mixed model to beless than the values from the naive model

For some attributes this is not the case. These are the ones with thehighest values of the test of Assessor main effect

Looking at the plot of the Assessor main effect you would expect thatthis was also the case for for the D.5m attribute.But looking at the plot of the Assessor-by-Product interaction this isthe attribute with the highest value


Results

Post hoc - Product differences in terms of d-prime

A B C D E F G H I J K L

Product estimates for the attribute E.5m

Products

d−pr

ime(

95%

con

fiden

ce in

terv

al)

−4

−3

−2

−1

01

2


Results

Post hoc - Product differences in terms of d-prime

A D G J

A

Products

d−pr

ime

−2

02

4

A D G J

B

Products

d−pr

ime

−2

02

A D G J

C

Products

d−pr

ime

−4

02

A D G J

D

Products

d−pr

ime

−3

−1

A D G J

E

Products

d−pr

ime

−30

00

300

A D G J

C.5m

Products

d−pr

ime

−5

−2

02

A D G J

D.5m

Products

d−pr

ime

−3

−1

1

A D G J

E.5m

Products

d−pr

ime

−4

−2

02


Results

Post hoc - PCA of product d-prime values

−6 −4 −2 0 2 4 6

−1

01

23

4

Bi plot

Principal component 1 (74.4%)

Prin

cipa

l com

pone

nt 2

(11

.4%

)

A

B

C

D

EC.5m

D.5mE.5mA

B

C

D

EF

GH

I

J

K

L


Results

Post hoc - Assessor performance in terms of d-prime

Assessor estimates for the attribute E.5m

d−pr

ime(

95%

con

fiden

ce in

terv

al)

−2

−1

01

2


Results

Post hoc - Assessor performance in terms of d-prime

2 6 23 9

A

Assessors

d−pr

ime

−1.

00.

01.

0

20 8 14 6 1

B

Assessors

d−pr

ime

−1.

00.

01.

0

1 7 14 21 27

C

Assessors

d−pr

ime

−1.

50.

01.

0

5 8 21 6

D

Assessors

d−pr

ime

−1

01

2

20 15 24 8 7

E

Assessors

d−pr

ime

−1.

50.

01.

5

11 9 1 23 10

C.5m

Assessors

d−pr

ime

−1

01

2

2 7 3 13 20

D.5m

Assessors

d−pr

ime

−1.

50.

01.

5

20 24 21 5 7

E.5m

Assessors

d−pr

ime

−2

02


Results

Likelihood Ratio test - mixed model - Ordinal response

Using the R-package ordinal



Attributes

Like

lihoo

d R

atio

sta

tistic

05

1015

20



Results




Attributes

Like

lihoo

d R

atio

sta

tistic

010

2030

4050



Results



Product main effects

Attributes

Like

lihoo

d R

atio

sta

tistic

050

100

150

200

250



Results

Likelihood Ratio test - comparison Assessor

1 2 3 4 5 6 7 8

010

2030

4050

Likelihood Ratio test − Assessor main effects

Attributes

Like

lihoo

d ra

tio s

tatis

tic

BinaryOrdinal


Results

Likelihood Ratio test - comparison Product

1 2 3 4 5 6 7 8

050

100

150

200

Likelihood Ratio test − Product main effects

Attributes

Like

lihoo

d ra

tio s

tatis

tic

BinaryOrdinal


Results

Likelihood Ratio test - comparison Assessor-by-Product

1 2 3 4 5 6 7 8

02

46

810

12

Likelihood Ratio test − Assessor−by−Product interaction effects

Attributes

Like

lihoo

d ra

tio s

tatis

tic

BinaryOrdinal


Final remarks

Final remarks

We can handle incomplete observations

We can fit the Thurstonian mixed model with binary data

The Thurstonian mixed model provides additional information (comparedto the naive aggregation model)

d-prime interpretations of Product estimates

d-prime interpretations of Assessor estimates

These preliminary results indicate that the ordinal information is moresensitive than the binary information


Final remarks

Future work

At some point in the future:

Investigation of how much binary data is needed to find a significantAssessor-by-Product interaction

Come up with a proper Thurstonian interpretation of the ordinalapproach

General modelling - for instance how to handle order effects


Final remarks

Thank you for your attention!


Final remarks

References

PanelCheck: Open source software for sensory profile data,www.panelcheck.com

Christensen, R. H. B. (2012). Ordinal - Regression Models for OrdinalData. R package version 2012.01-19http://www.cran.r-project.org/package=ordinal/

Christensen, R. H. B. and P. B. Brockhoff (2012). Analysis ofreplicated categorical ratings data from sensory experiments. Workingpaper.

Alexandra Kuznetsova, Per Bruun Brockhoff and Rune HauboBojesen Christensen (2012). lmerTest: Tests for random and fixedeffects for linear mixed effect models (lmer objects of lme4 package)..R package version 1.0.

Brockhoff, P.B. and Christensen, R.H.B. (2010). Thurstonian modelsfor sensory discrimination tests as generalized linear models. FoodQuality and Preference 21(3), 330-338


individual differences in replicated multi-product 2-afc...

Documents