CARLETON UNIVERSITY

SCHOOL OF

MATHEMATICS AND STATISTICS

HONOURS PROJECT

TITLE: Analyzing Pixel Intensity Data Using

Linear Regression Model and Linear Mixed Model

AUTHOR: Yumin Park

SUPERVISOR: Song Cai

DATE: January 4, 2019

2

Table of Contents

1. Introduction ............................................................................................................... 3

2. Data Description........................................................................................................ 4

3. Models Analysis ......................................................................................................... 5

3.1 Linear Regression Model ................................................................................... 5

3.2 Linear Mixed-Effect Model ............................................................................... 6

3.2.1 Random Effects ........................................................................................... 7

4. Exploratory Analysis ................................................................................................ 7

4.1 Basic Structure of Data ...................................................................................... 8

4.2 Relationship Between Response and Time....................................................... 9

4.3 Summary of Exploratory Analysis ................................................................. 11

5. Data Analysis Using Regression Models ............................................................... 11

5.1 Analysis Using Linear Regression .................................................................. 11

5.1.1 Model Building and Variable Selection .................................................. 12

5.1.2 Diagnostic for Linear Regression Model ................................................ 13

5.1.3 Analysis and Interpretation ..................................................................... 15

5.2 Analysis Using Linear Mixed-Effect Model ................................................... 16

5.2.1 Choosing Structure for Linear Mixed-Effect Model ............................. 16

5.2.2 Building Linear Mixed-Effect Model ...................................................... 17

5.2.3 Diagnostic for Linear Mixed-Effect Model............................................. 19

5.3 Analysis, Comparison and Interpretation ..................................................... 21

6. Conclusion and Discussion ..................................................................................... 24

7. Appendix .................................................................................................................. 26

8. References ................................................................................................................ 45

3

1. Introduction

Over the last decade, the literature on model selection in linear mixed-effect models

has been growing exponentially. The problem became more complicated than in linear

regression, because the selection on the covariance structure was not simple, due to the

computational issues and boundary problems arising from positive semi-definite

constraints on covariance matrices. Hence, the Linear Mixed-Effects Model (LMMs) was

proposed in the classical paper by Laird and Ware (1982).

The linear mixed-effect models, also known as “multi-level models” or

“hierarchical models”, are a type of regression model that contemplates both (1) variation

that is explained by the independent variables of interest – fixed effects, and (2) variation

that is not explained by the independent variables of interest – random effects. Due to the

aspect of this model having a mixture of fixed and random effects, it is classified as a mixed

model. In terms of random effects, it gives an error term ϵ to the structure, and this

represents the deviation from the predictions due to “random” factors that cannot be

controlled experimentally.

The general linear mixed model provides a useful approach for analysing a wide

variety of data structures which practising statisticians often encounter. Such data

structures which can be problematic, are imbalanced repetitive measured data and

longitudinal data. The model is similar in many aspects to ordinary multiple regression, but

because it allows correlation between the observations, it requires an additional work to

specify models and assess goodness-of-fit.

In linear regression models, the responses are independent, whereas, in linear

mixed-effect models, they are typically dependent. This dependence impacts on model

selection by reducing the effective sample size, a quantity that affects the theoretical

properties of procedures and is used explicitly in some model selection procedures such as

the Bayesian Information Criteria (BIC; Schwarz, 1978). The dependence also means that

linear mixed-effect models have both regression parameters—which describe the mean

structure, and variance parameters – that describe the sources of variability and the

dependence structure. These parameters have a different relative importance in the analysis.

Thence, this should be reflected in model selection.

4

From this research project, an exploratory analysis was performed to discover the

basic structure of data based on the X-ray pixel intensity dataset acquired from an

experiment. Thus, the linear regression analysis was implemented to find the relations

between the response and covariates. Both exploratory analysis and linear regression

analysis indicated that there was a two-level nested group effect in the data. Based on this

analysis, a linear mixed-effect model was created to fit the data. The construction of this

model referenced the information from the linear regression model. Consequently, the

linear mixed-effect model built was optimal in terms of trade-off between fitting and

robustness.

2. Data Description

The dataset used for this research project is chosen from R ‘MEMSS’ dataset package,

‘Pixel’, and it was described in Pinheiro and Bates (2000). The dataset is from an

experiment conducted by Deborah Darien, Department of Medical Sciences, School of

Veterinary Medicine, University of Wisconsin, Madison. The experimenters injected ten

individual dogs with a dye contrast, then recorded the mean X-ray pixel intensities from

CT scans of the right and left lymph nodes in the axillary region of each dog on several

occasions up to 21 days post injection. The main interest of this experiment was to study

the change of the mean pixel intensity over the time

This ‘Pixel’ data frame has 102 rows and 4 columns of data based on the above

information and data frame contains the following columns:

Dog: a factor with levels A to J designating the dog on which the scan was made

Side: a factor with levels L and R designating the side of the dog being scanned

Day: a numeric vector giving the day post injection of the contrast on which the

scan was made

Pixel: a numeric vector of pixel intensities

5

Figure 1: Pixel dataset (98 rows are omitted)

Because a dog is a group factor and the side are a factor nested within the dog, this paper

refers the “dog level” to the individual effect of the dog, and the “side level” or “side within

dog level” represents the effect of each side of a dog. Also, the “within group” factor refers

to the effect of each observation.

Since the data was obtained from an experiment, there were few assumptions to

proceed the research:

1. Assume that the experiment is well randomized.

2. Assume that the scans are carried out independent within the side level

3. The pixel observations are numerical values obtained from a continuous variable, and

we assume that each observation is obtained from a normal distribution.

3. Models Analysis

From this section, linear regression model and linear mixed effect model will be

explained theoretically before the implementation of concepts to the dataset.

3.1 Linear Regression Model

The linear regression model,

6

𝑦𝑖 = 𝛽1𝑥1𝑖 + 𝛽2𝑥2𝑖 + ⋯+ 𝛽𝑝𝑥𝑝𝑖 + 𝜖𝑖

𝜖𝑖 ~ 𝑁𝐼𝐷(0, 𝜎2)

has one random effect, the random error term, denoted 𝜖𝑖. The parameters of the model are

the regression coefficients, 𝛽1, 𝛽2, … , 𝛽𝑝, and the error variance, 𝜎2. In most cases, 𝑥1𝑖 =

1, and so 𝛽1 is a constant or intercept. Therefore, the linear model in matrix form has the

following structure;

𝑌 = 𝑋𝛽 + 𝜖

𝜖 ~ 𝑁𝑛(0, 𝜎2𝐼𝑛)

Where 𝒀 = (𝑦1, 𝑦2, … , 𝑦𝑛)′ is the response vector; 𝚾 is the model matrix, with typical row

𝒙𝑖′ = (𝑥1𝑖, 𝑥2𝑖 , … , 𝑥𝑝𝑖); 𝜷 = (𝛽1, 𝛽2, … 𝛽𝑝)

′ is the vector of regression coefficients; 𝜖 =

(𝜖1, 𝜖2, … , 𝜖𝑛)′ is the vector of random errors; 𝑵𝑛 represents the n-variable multivariate-

normal distribution; 0 is an 𝑛 × 1 vector of zeroes; and 𝚰𝑛is the order-𝑛 identity matrix.

3.2 Linear Mixed-Effect Model

The mixed-effect models include additional random-effect terms and are often

appropriate for representing clusters. Therefore, they are claimed to be independent. For

instance, when data are collected hierarchically, when observations are taken on related

individuals, or when data are gathered over time on the same individuals. For hierarchical

data with a single level of grouping, we can formulate the classical LMM at a given level

of a grouping factor as follows:

𝒚𝒊 = Χ𝑖𝛽 + 𝑍𝑖𝑏𝑖 + 𝜖𝑖

𝒃𝒊 ~ Nq(0,𝝍)

𝝐𝒊 ~ 𝑁𝑛(0, 𝜎2 ∧𝑖)

Where;

𝒚𝒊 is the 𝑛𝑖 × 1 response vector for observations in the 𝑖th group.

𝚾𝒊 is the 𝑛𝑖 × 𝑝 model matrix for the fixed effects for observations in group 𝑖.

𝜷 is the 𝑝 × 1 vector of fixed-effect coefficients.

7

𝒁𝒊 is the 𝑛𝑖 × 1 model matrix for the random effects for observations in group 𝑖.

𝒃𝒊 is the 𝑞 × 1 vector of random-effect coefficients for group 𝑖 and independent.

𝝐𝒊 is the 𝑛𝑖 × 1 vector of errors for observations in group 𝑖 and independent.

𝝍 is the 𝑞 × 𝑞 covariance matrix for the random effects.

𝝈𝟐 ∧𝒊 is the 𝑛𝑖 × 𝑛𝑖 covariance matrix for the errors in group 𝑖.

Specifically, where 𝒚𝒊, 𝚾𝒊, 𝜷, and 𝝐𝒊 are the vectos, the design matrix, and the vector of

residual errors for group 𝑖, respectively, while 𝒁𝒊 and 𝒃𝒊 are the matrix of covariates and

the corresponding vector of random effects:

𝒁𝒊 =

[ 𝑧𝑖1

(1)𝑧𝑖1

(2)

𝑧𝑖2(1)

𝑧𝑖2(2)

⋯ 𝑧𝑖1(𝑞)

⋯ 𝑧𝑖2(𝑞)

⋮ ⋮

𝑧𝑖𝑛𝑖

(1)𝑧𝑖𝑛𝑖

(2)⋱ ⋮

⋯ 𝑧𝑖𝑛𝑖

(𝑞)]

= (𝑧𝑖(1)

𝑧𝑖(2) ⋯ 𝑧𝑖

(𝑞)), 𝒃𝒊 = (

𝑏𝑖1

𝑏𝑖2

⋮𝑏𝑖𝑞

) .

3.2.1 Random Effects

The core of mixed models is incorporation of fixed and random effects. A fixed effect

is a parameter that does not vary. For example, it can be assumed that there are some true

regression lines in the population, 𝛽, and estimation of it, ��. In contrast, random effects are

parameters that are themselves random variables. For instance, assume 𝛽 as a distributed

random normal variate with mean value of 𝜇 and standard deviation of σ, or equation form

𝛽~ 𝑁(𝜇, 𝜎). This is essentially equivalent to linear regression, where the data are random

variables. However, the parameters are fixed effects. Then, the data are random variables

and the parameters are random variables at one level but fixed at the highest level.

4. Exploratory Analysis

In order to investigate the change of the pixel over the time, the model should be

built with pixel in terms of response and time (day) as covariate factors. Hence, to build

the previously described model, following aspects can be used: the type of relationship

between pixels and time, the applicability of relationship applied for different dogs or even

8

each side of a dog, and finally, according to the data, the possible assumptions to be made

for building a regression model.

4.1 Basic Structure of Data

As mentioned in Section 2, Data Description, the dataset contains 102 observations

in total, and the number of observations for each dog is shown in Table 1, where N

represents the total number of observations for each dog. Additionally, basic statistics for

pixels and time are shown in Table 2.

Table 1: Number of observations for each dog

Dog 1 2 3 4 5 6 7 8 9 10

N 13 13 14 14 9 10 8 8 4 6

Table 2: Basic Statistics for Pixel and Dog

Min 1st Qu. Median Mean 3rd Qu. Max

Pixel 1034 1054 1088 1087 1110 1161

Dog 1.000 3.000 5.000 4.902 7.000 10.000

Prior to the further analysis, there are few things to point out from Table 1 and

Table 2. First of all, the number of observations for each dog is different. Thus, the

distributions of response pixel and covariate time is not seriously skewed. To check if there

are any differences among the groups at dog level and side level, the structure of pixel for

different dogs and both sides of each dog’s lymph nodes should be studied as the data was

collected from different dogs, and for each dog, response values of two sides of lymph

nodes were always recorded. From this statement, the “groups” represents the groups of

most inside level (i.e. side level).

Additionally, box plots are drawn at the dog level and side level. Referring to the

left diagram in Figure 2, at dog level, the differences among mean values were obvious and

the variation of the spreads among the different dogs is shown. Roughly speaking, dogs

with smaller mean pixel intensities have smaller variances. However, dog 10 is an

exception, because it has a medium mean value, but the values are widely spread out, which

represents the dog 10 as having large range of values. Also, there are two outliers with very

9

small pixel values for dog 4 and one outlier each for dog 2 and dog 3. Referring to the

diagram on the right side of Figure 2, at side level which was nested within dog, mean

values as well as variances of two sides of each dog’s lymph nodes are different especially

for dog 6, 7, 8, 9 and 10. At both dog and side levels, dog 10 is sufficient to be call as an

exception.

4.2 Relationship Between Response and Time

In section 4.1, the analysis is based on the relationship between Pixel and Dogs.

Moreover, from these results, it is ambiguous to state that these differences are caused by

group effects or by the non-homogeneity of time for different groups. For instance,

different groups observed the pixel at different days to figure out the clear cause of the

difference. Thus, the change of response over time was studied.

Figure 2: Pixel for each dog (left) and Pixel for each side of every dog (right)

10

Table 3: Basic Statistics for Pixel and Day

To study the change of response over time, the scatter plot of pixel versus time was

drawn for the entire data and a smoothed curve was fit using Local Polynomial Regression

Fitting (LOESS), shown in Figure 3. This fitted curve showed a quadratic pattern for the

relationship between pixel and day and this indicated that for each day, the variability of

pixel was high. Due to the fact that different observations were made from different dogs

and different sides, the variabilities were present. Hence, the effect of individual group was

investigated with days at dog level and side level versus pixel intensity scatter plots (Figure

4). There were few details to point out from Figure 4.

First, for each group at side level, the relationship between pixel and day (time)

were non-linear and for most dogs, they had a shape of quadratic. Especially for dog 4,

pixel on both sides of lymph nodes showed greater variability than pixel on both sides of

other dogs’ lymph nodes. This indicated heteroscedasticity of within group error.

Furthermore, within dog level, curvatures were very similar for both sides of lymph

nodes for most dogs, except that there was a constant drift. For all dogs, except 9 and 10,

even though they had different left and right lymph node values, the patterns were similar

as described above. However, the graphs of dog 9 and dog 10 showed that left and right

Min 1st Qu. Median Mean 3rd Qu. Max

Pixel 1034 1054 1088 1087 1110 1161

Time (day) 0.00 4.00 6.00 7.49 10.00 21.00

Figure 3: Scatter Plot of pixel vs time of entire data with

fitted curve

Figure 4: Scatter Plots of pixel vs time at dog level

and side level

11

lymph node values were different, as well as the patterns. Consequently, there were

differences between slopes of two sides of lymph nodes, but this may have been caused by

the small sample sizes. As shown in Table 1 in section 4.1, dog 9 and dog 10 had only four

and six observations respectively.

Ultimately, from the left diagram on Figure 2, obvious contrasts were seen between

dogs, sample means and sample variances. Although, the pattern of relationship between

pixel and time for most of dogs was quadratic, the shapes varied.

4.3 Summary of Exploratory Analysis

Referring back to Section 4.1 and 4.2, the following preliminary conclusions can be

made: the time on pixel density is different for each dog and each side of lymph node. At

dog level, there were difference between means, variances and the curvature. At side level,

the curvatures were similar but different intercepts. Within groups at side level, there was

a possibility of heteroscedasticity, especially for dog 4. Also, the relationship between pixel

density and day (time) was nearly quadratic for most dogs.

5. Data Analysis Using Regression Models

From this section, linear regression model and linear mixed effect model was

analyzed and compared technically for the corresponding dataset described in the Section

2 and 4.

5.1 Analysis Using Linear Regression

Prior to the usage of linear regression models, recall three assumptions made at the

end of Section 2.

1. Assume that the experiment is well randomized.

2. Assume that the scans are carried out independent within the side level

3. The pixel observations are numerical values obtained from a continuous variable,

hence assume that each observation is obtained from a normal distribution.

12

In consideration of the three assumptions above, especially with the last assumption, the

linear regression model can be built to study the relationship between pixel and day (time).

The linear regression model used in this section is providing very useful guide for building

a linear mixed-effect model. Furthermore, the result from this model was used as reference

to compare with a mixed model.

5.1.1 Model Building and Variable Selection

Preliminary analysis showed that the relationship between pixel and day(time) is

nearly quadratic. Therefore, the term 𝑑𝑎𝑦2 was used for the model. Additionally, the dog

and side both had few effects on the response. Consequently, there were used as

covariates. Then, the simple model was defined as:

𝑌𝑖 = 𝛽0 + 𝛽1𝑑𝑎𝑦𝑖 + 𝛽2𝑑𝑎𝑦𝑖2 + 𝛽3𝐷𝑜𝑔𝑖 + 𝛽4𝑆𝑖𝑑𝑒𝑖 + 𝜖𝑖

= β′ 𝑋𝑖 + ϵi (1)

where ϵi 𝑖. 𝑖. 𝑑.~ 𝑁(0, 𝜎2), β′ = (𝛽0, ⋯ , 𝛽4) and 𝑋𝑖

= (𝑑𝑎𝑦𝑖 , 𝑑𝑎𝑦𝑖2, 𝐷𝑜𝑔𝑖 , 𝑆𝑖𝑑𝑒𝑖)′. Then, the

step-wise selection on this model was performed and concluded with three covariates: day,

day2 and dog. The variable Side was excluded, since it did not pass the significant test with

p-value of 0.1 for t-test. Furthermore, the R-squared statistics for the reduced model was

0.7541 where the higher R-squared value represents a better result with 1 being the best.

Therefore, this was not a good fit for the model. (Refer to Appendix A for the outputs)

With the previous analysis, for each dogs and sides, the relationship between pixel

and time may be different. Consequently, the cross effect terms, such as “day:Dog”,

“day:Dog:Side” and etc., was implemented to the model, in order to achieve the desired

results. Prior to the selection of hyperparameters, all possible terms including intercept,

main effects and interaction between cross effects terms were taken into consideration.

Then, the following procedure was used for performing variable selection method:

1. Fit models to each of the variables individually, then choose the qualifying

variables that pass the t-test.

13

2. Then, a model is generated with the selected variables from step 1. From this

model, t-test is performed once again for each individual variable, to remove

the variables that may not pass the test. At each removal, F-test is used to

compare the nested model to see if the removal leads to the worst fit. If it leads

to the worst fit, keep the variable, otherwise remove it.

3. Variables removed from step 1 may become significant variables for the

presence of others. Thus, those variables are added one at a time to the model

from step 2 whether it leads to the better fit. Similar to step 2, F-test is used to

compare fits.

4. The last step is to verify the model from step 3, to ensure if this is the best model

by checking that no variable should be either removed from or added to the

current model using F-test for the best fit.

Note that Appendix B contains the outputs. After going through this selection procedure,

the final model is found, and this model is called lmfit.

𝑙𝑚𝑓𝑖𝑡 ∶ pixel ~ day2 + Dog + day: Dog + day: Side + day2: Dog + Dog: Side

The model above was labeled as la2 in the codes and the Residual standard error is 8.63 on

63 degrees of freedom. Also, the R-Squared statistic of this model is 0.9292. Compared

with previous simple linear model, R-Squared statistic was 0.3465. This model explained

more variability of the data. Also, this provide better fit as p-value of F-test for comparing

those two models is 1.520𝑒−13.

5.1.2 Diagnostic for Linear Regression Model

Seven residual plots were generated to test the corresponding model assumptions:

independence, normality, constant variance and linearity. Since there were no patterns

shown for the top left plot (Residuals values versus Fitted values), and the second top left

plot (Scale-Location), which represents the standardized residual values versus fitted

values, indicated that the model was in a reasonably good fit. Thus, it justified the constant

variance assumption and indicated that the errors were independent.

14

However, with these plots, several outliers were identified, such as 24, 27, 28 and

74. Furthermore, the Q-Q plot showed that the normality assumption was suitable, and it

also showed outliers, such as 27, 28 and 74. The Residual versus Leverage plot and Half

Normal Plot of Leverage plot suggested that there were several influential points, such as

28, 47, 49, 98 and 100. The partial residual plot of main effect of day plot showed a good

linear relationship. The Partial Residual Plot of day^2 plot which is the main effect of

day^2, is not strictly linear and this may have been occurred by the higher order nature of

this variable. Overall, the model fits the data reasonably well.

Figure 5: Residual plots for linear regression model: lmfit

15

5.1.3 Analysis and Interpretation

Clearly, from the model construction, several cross-effect terms were considered

significant in the model. At dog level, the cross-effect terms day:Dog and day^2:Dog were

present. With the main effect terms of day, day^2 and Dog, was included in the final model.

This can be justified as day and day^2 have interactions with Dog level and this agreed

with the exploratory analysis with Figure 4 that for each dog, the mean value of pixels is

different, and the curvature of the relationships are also different.

At side level, the cross-effect terms day:Side and Dog:Side were present but the

main effect of side was not present in the model. This indicated that there was a nested

effect of side on the pixel values. Furthermore, the cross-effect term day2: Side was not

included in the final model and this implies that no significant difference between the

quadratic terms of two sides of lymph nodes for each dog were shown. Moreover, this

agreed with the exploratory analysis, Section 4.3, within dog level, the curvature for two

sides are nearly parallel with a constant drift, and for dog 9 and 10, there were differences

between slopes.

Even though, the linear regression model, lmfit, is an excellent fit, there were some

issues to be discussed:

1. There were only 102 observed data, however, there were 50 parameters present in

the model due to the complicated group effects. As a consequence, the average

magnitude of sample size was 5 for each group at side level, since for each side of

a dog, the biggest size was 7 and the smallest size was 2. In some cases, the model

was easily fitted to few samples in the group, but it missed the true relationship

from the population, such as overfitting.

2. In Section 5.1.2, several outliers and influential points were identified. It is not

surprising that all the outliers came from dog 4. According to Figure 1 and 4, and

exploratory analysis, pixels on both sides of lymph nodes of dog 4 showed large

variability compared to other dogs. Additionally, this may had been occurred by

the heteroscedasticity of within group error. Therefore, it cannot be reduced with a

constant variance assumption. On the other hand, four of the influential points

16

originated from dog 9 and 10. Hence, according to Table 2, Dog 9 had only 4

observations and Dog 10 had only 6 observations. Since both sides of lymph nodes

were observed individually, the number 4 and 6 represented that dog 9 was

observed only twice for each side and dog 10 was observed for 3 times for each

side. Considering that the sample sizes of those two dogs were very small, it can be

concluded that the inference is actually made toward the few points. Due to the fact

that these few points were used to generate the inference, a small estimated variance

occurred, hence these points were indeed influential.

Table 4: List of Outliers Table 5: List of Influential Points

5.2 Analysis Using Linear Mixed-Effect Model

A linear mixed-effect model would be an ideal candidate to build and compare with

the linear regression model lmfit. This is because the linear mixed-effect model would help

avoid issues that are listed at the end of Section 5.1 and help incorporate the effects of

different groups. The effects of different groups are assumed to be a random sample from

a common population with common covariance structure in the linear mixed-effect model,

and this covariance structure reflects the variability between groups. Therefore, the effects

of groups and its correlations are incorporated whilst the numbers of parameter are visibly

reduced.

5.2.1 Choosing Structure for Linear Mixed-Effect Model

For forming the linear mixed-effect model for data, there are two questions that must

be considered:

Dog Side Day pixel

24 4 R 6 1126.4

27 4 R 14 1100.7

28 4 R 21 1102.2

74 4 L 4 1114.1

Dog Side Day pixel

28 4 R 21 1102.2

47 9 R 4 1068.2

49 10 R 4 1095.0

98 9 L 4 1097.2

100 10 L 4 1132.3

17

1. Which variables should be considered for having random effects?

2. What is the covariance structure of the random effects at each level?

For the first question, the linear regression model lmfit has answered partially. All variables

appeared in the lmfit’s cross effect terms at dog level and side level could be considered as

a random effect. More specifically, intercept, day and day2 from the dog level, intercept

and day from the side level are considered as random effects. Furthermore, this agrees with

the exploratory analysis as shown in the previous sections.

To discover the covariance structure, two paired plots can be drawn for the

coefficients of the cross-effect terms from lmfit at dog level, Figure 6, and at side level

Figure 7. With these two plots, it is easy to see if there is any correlation existing for cross-

effect terms. Hence, the full variance-covariance structure will be used in the mixed model

as it appears that the variables are correlated at each level.

Figure 6: Correlation of coefficients of cross-effect

terms at dog level

Figure 7: Correlation of coefficients of cross-effect

terms at side level

5.2.2 Building Linear Mixed-Effect Model

According to above analysis for the linear mixed-effect model, the model can be

built as follow:

𝑌𝑖𝑗𝑘 = 𝛽0 + 𝛽1𝑑𝑎𝑦𝑖𝑗𝑘 + 𝛽2𝑑𝑎𝑦𝑖𝑗𝑘2 + 𝜂𝑖𝐷𝑜𝑔𝑖𝑗𝑘 + 𝜂𝑖𝑑𝑎𝑦𝑖𝕝(𝑖≠1) + 𝛾𝑖0 + 𝛾𝑖1𝑑𝑎𝑦𝑖𝑗𝑘

+ 𝛾𝑖2𝑑𝑎𝑦𝑖𝑗𝑘2 + 𝛼𝑖𝑗0 + 𝛼𝑖𝑗1𝑑𝑎𝑦𝑖𝑗𝑘 + 𝜖𝑖𝑗𝑘

18

Where 𝑖 = 1,… , 10 is the label for dogs, 𝑗 = 1,2 is the label for sides, 𝑘 = 1,… 𝑛𝑖𝑗 is the

label for observations for side j of dog I and 𝕝(𝑖≠1) is the indicator function which takes

value 1 if 𝑖 = 𝑘 and 0 otherwise, and 𝕝(𝑖≠𝑘) ∶= 1 − 𝕝(𝑖=𝑘). In the model, 𝛽𝑖 𝑎𝑛𝑑 𝜂𝑖 are

fixed effects, and 𝛾𝑖𝑗 and 𝛼𝑖𝑗𝑘 are random effects.

Define,

�� 𝑖𝑗 = (𝑌𝑖𝑗1, ⋯ , 𝑌𝑖𝑗𝜂𝑖𝑗)𝑇

𝑋𝑖𝑗 = (

1 𝑑𝑎𝑦𝑖𝑗1 𝑑𝑎𝑦𝑖𝑗12

⋮ ⋮ ⋮1 𝑑𝑎𝑦𝑖𝑗𝜂𝑖𝑗

𝑑𝑎𝑦𝑖𝑗𝜂𝑖𝑗

2

𝕝(𝑖=2) 𝕝(𝑖=3) …

⋮ ⋮ 𝕝(𝑖=2) 𝕝(𝑖=3) …

𝕝(𝑖=10)

⋮𝕝(𝑖=10)

)

And

𝛽 = (𝛽0, 𝛽1, 𝛽2, 𝜂2,, ⋯ , 𝜂10)𝑇

Note that “dog” is a categorical variable of 10 levels.

In addition, let

𝑍𝑖𝑗 = (

1 𝑑𝑎𝑦𝑖𝑗1 𝑑𝑎𝑦𝑖𝑗12

⋮ ⋮ ⋮1 𝑑𝑎𝑦𝑖𝑗𝜂𝑖𝑗

𝑑𝑎𝑦𝑖𝑗𝜂𝑖𝑗

2)

𝑊𝑖𝑗 = (

1 𝑑𝑎𝑦𝑖𝑗1

⋮ ⋮1 𝑑𝑎𝑦𝑖𝑗𝜂𝑖𝑗

)

𝛾 𝑖 = (𝛾𝑖0, 𝛾𝑖1, 𝛾𝑖2)𝑇 𝑎𝑛𝑑 𝛼 𝑖𝑗 = (𝛼𝑖𝑗1, 𝛼𝑖𝑗2)

𝑇

Then the model can be represented as following:

�� 𝑖𝑗 = 𝑋𝑖𝑗𝛽 + 𝑍𝑖𝑗𝛾 𝑖 + 𝑊𝑖𝑗𝛼 𝑖𝑗 + 𝜖 𝑖𝑗 𝑤ℎ𝑒𝑟𝑒 𝜖 𝑖𝑗 = (𝜖𝑖𝑗1, ⋯ , 𝜖𝑖𝑗𝜂𝑖𝑗)𝑇

We assume that 𝛾 𝑖 𝑖. 𝑖. 𝑑.~ 𝑁(0,𝜓1) , 𝛼 𝑖𝑗

𝑖. 𝑖. 𝑑.~ 𝑁(0,𝜓2) and 𝜖 𝑖𝑗

𝑖. 𝑖. 𝑑.~ 𝑁(0, 𝜎2𝐼) , where

𝜓1, 𝜓2 and 𝜎2 are unknown parameters. Additionally, assume that 𝛾 𝑖 , 𝛼 𝑖𝑗, and 𝜖 𝑖𝑗 are

independent and this model can be called as lmefit. Results of lmefit showed that all fixed

effects are significant in this model. To compare models, covariance matrices should be

19

modified to diagonal matrices. After that, chi-square test can be used to compare models.

According to R, lmefit has a p-value as 00.000133 and when this model is compared to the

reduced model, it is significantly a better model.

As mentioned in Section 5.1.1, the chi-square test is used instead of the F-test to

perform the same variable selection method for this model. The reason behind this is that

the chi-square test is intended to test how likely it is that an observed distribution is due to

chance. It is also called a “goodness of fit” (All outputs can be found in Appendix B)

because of its ability to measure how well the observed distribution of data fits with the

distribution that is expected if the variables are independent. According to Appendix B, the

model Imefit19 representing the above linear mixed-effect model is the best output as it

has the smallest chi-squared value, 0.01491.

5.2.3 Diagnostic for Linear Mixed-Effect Model

For linear mixed-effect model, following two types of model assumptions were examined:

1. Within-group errors are independent and identically normally distributed with

mean zero and constant variance, and they are independent of the random effects.

2. The random effects are normally distributed.

For within-group error, the residual Q-Q plot, figure 8, and residual scatter plot,

figure 9, are draw. For the Q-Q plot, it shows that the points in a plot are straight therefore,

the normality assumption is reasonably satisfied. Additionally, the residual scatter plot has

no pattern. A residual plot shows the residuals on the vertical axis and the independent

Figure 8: Q-Q plot of residual of lmefit

Figure 9: Residual plot of lmefit

20

variable on the horizontal axis. If the points in a residual plot are randomly dispersed

around the horizontal axis, a linear regression model is appropriate for the data. Since the

residual plot has no pattern, the model fits well and the constant variance assumption is

satisfied.

Figure 10 represents the scatter plots of residual for each side and this indicates that

the right-side points of plot are more clustered than the left side. This indicates that the

residual of the right side of lymph nodes has a slightly larger variability than the residual

of the left side of lymph nodes. However, they are basically identical. Moreover, Figure 11

represents the residual plot of lmefit by group and points for plots of dog 3, dog 4, dog 6,

dog 7 and dog 10 are spread especially dog 4. This shows that dog 4 has the greatest

variability compare to other dogs. However, for other groups, scatter plots show no

relationship between within-group error and effects of groups. This justifies assumption of

within-group errors are independent of random effects.

For random effects, Q-Q plots of estimated random effects at dog level, Figure 12, and

side level, Figure 13, can be draw. As mentioned earlier, points of each plot are shown as

straight therefore, the normal assumption for random effects are well satisfied.

Figure 10: Residual plot of lmefit by side

Figure 11: Residual plot of lmefit by group

21

Figure 12: Q-Q plots of random effect at dog level

Figure 13: Q-Q plot of random effect at side level

5.3 Analysis, Comparison and Interpretation

From beginning to end of the perfect fit model finding procedure, there are few major

points to be reviewed:

1. Why is the mixed model needed even though a linear regression can fit the data

quite well?

2. how does the number of parameters effect the test result?

3. Outliers.

To answer the first question, two models, lmfit and lmefit, should be compared. Recall that,

lmfit is the normal linear regression model and lmefit is the linear mixed-effect model.

Table 6 represents some goodness-of-fit statistics of lmfit and lmefit.

For AIC, two models have analogous values but lmfit has slightly smaller value

than lmefit. This can be concluded as the model lmfit is slightly better than lmefit. For BIC,

lmefit has smaller value than lmfit. For AIC, the difference between lmfit and lmefit was

only 9.9333 however, for BIC, the difference is 37.3162 which is much bigger than AIC.

there are 50 parameters in the model lmfit but, there are only 21 parameters in lmefit. Since

BIC penalizes more on number of parameters than AIC does, lmefit has much better value

on BIC. Therefore, this can be concluded as the model lmefit is a better fit. For log-

likelihood values, lmfit has lower log likelihood value than lmefit because lmfit contains

more parameters.

22

Table 6: Statistics for lmfit and lmefit

The analysis described above leads to the second question of the key points and to

answer this question, fitted values of two models for different groups at each level should

be compared. Figure 14 and 15 represent Fitted Values by Linear regression model and

Linear Mixed-Effect Model accordingly. Those two figures look similar but when each

fitted curve is compared, the curves fitted by linear mixed-effect model is “flatter” than the

curves fitted by linear regression model, especially for the groups with small samples, Dog

5, 6, 7, 8 and 10 for both sides. This indicates that the linear mixed-effect model is less

“plagued” by the small sample size problem, i.e. provides more robust estimates of

parameters than linear regression. It is because, the normal linear regression model lmfit

has parameters for day and day2 for each group at every level. Hence, the sample points

used to estimate those parameters in each group are very small. However, for linear mixed-

effect model, the group effects share a few sample parameters at each level and this

increased the number of observations used to estimate each parameter efficiently.

Additionally, the last key point outliers can be considered as a problem. As shown

in Figure 11, dog 4 has some extra variability that is not explained by the model lmefit and

this was also observed in exploratory analysis section. There are two possible following

reasons:

1. The sample size for dog 4 is too small.

2. Dog 4 is from a distribution that has different variance with other groups.

Number of

parameters

AIC BIC Log Likelihood

lmfit 50 759.9864 864.9853 -339.9932

(df = 40)

lmefit 21 769.9197 827.6691 -362.9599

(df = 22)

Difference 29 9.9333 37.3162 22.9667

23

For this case, it is more likely to be caused by the second reason. It is because for

dog 4, there are total of 14 observations, 7 for each side, which is the largest sample size

in entire dogs, so the first reason doesn’t count. Also, beginning of this research, there were

three assumptions in Section 2 and one of those was, we assume that at each level, the

random effects come from a same normal distribution. If the second reason is the case, the

model lmefit cannot capture this variability since it comes from different normal

distribution.

Figure 14: Fitted Values by Normal Linear Regression

Model

Figure 15: Fitted Values by Linear Mixed-Effect Model

Finally, the last key point, outliers can be considered as a problem. As shown in

Figure 11, dog 4 has some extra variability that is not explained by the model lmefit and

this was also observed in exploratory analysis section. There are two possible following

reasons:

1. The sample size for dog 4 is too small.

2. Dog 4 is from a distribution that has different variance with other groups.

For this case, it is more likely to be caused by the second reason. It is because for

dog 4, there are total of 14 observations, 7 for each side, which is the largest sample size

24

in entire dogs, so the first reason doesn’t count. Also, beginning of this research, there were

three assumptions in Section 2 and one of those was, we assume that at each level, the

random effects come from a same normal distribution. If the second reason is the case, the

model lmefit cannot capture this variability since it comes from different normal

distribution.

6. Conclusion and Discussion

For the X-ray pixel intensities data, both models from normal linear regression lmfit

and linear mixed-effect model lmefit were adequate. Comprehensively, both models agreed

with the results from the following exploratory analysis: the relationship between pixel

density and day was nearly quadratic for most dogs, the group effects on the relationship,

at dog level. Moreover, there were differences between means, variance, and curvature and

at side level, the curvatures were similar but had different intercepts.

Through out the model construction procedure, the conclusion was leaning towards

the linear mixed-effect model for grouped data, because it has capabilities of modeling the

differences between groups and the correlation within groups. Thus, at the same time, at

each level, shared parameters were used, and this introduced “robustness” to the model.

On the other hand, Normal linear regression model was adequate, but to use normal linear

regression model, usually more parameters were required, and this could possibly cause

“overfitting” issue. Once overfitting happens to the model, it fails to generalize, then

malfunctions. Overfitting usually occurs when sample size at each group is too small.

Both exploratory analysis and linear regression analysis was available to provide

some useful guidance, especially on the choice of variables for random effects and the

covariance structure of random effects at each level.

From the analysis and interpretation of the dataset, dog 4 showed much greater variability

than all other dogs, where it can be easily confirmed with Figure 4 and 11. Furthermore,

both linear regression model and mixed effect model had few points of dog 4, that were

outliers and this can be checked with Figure 2 and Table 4. Additionally, for mixed model,

there were few assumptions made and the most important assumption was common

covariance structures of the random effects for each dog group. However, it was extremely

25

likely that the effect of dog 4 came from a population that had different variance from other

groups. One possible way to deal with this problem was to “push” variability into the

within-group error, then create a model that is capable of dealing with heteroscedasticity

errors. Nevertheless, neither of these two models incorporated this problem and this can be

studied further to improve the models.

26

7. Appendix

Code

# install.packages("rpart")

# install.packages("nlme")

# install.packages("lattice")

# install.packages("faraway")

# install.packages("lme4")

# install.packages("xtable")

# install.packages("Hmisc")

# install.packages("partykit")

# install.packages("robustHD")

# install.packages("sqldf")

# Random Effects Models

rm(list=ls())

## Pixel

require(nlme)

require(lattice)

require(faraway)

require(lme4)

require(xtable)

##require(Hmisc)

data(Pixel)

attach(Pixel)

print(Pixel)

View(Pixel)

class(Pixel)

getGroupsFormula(Pixel)

attach(Pixel)

## Preliminary Analysis

str(Pixel)

summ1 <- summary(pixel)

print(summ1)

summ2 <- summary(Pixel$Dog)

print(summ2)

summ3<- summary(Pixel)

print(summ3)

bwplot(pixel~Dog, xlab='Dog',varwidth=T)

bwplot(pixel~Side|Dog, xlab='Side in Dog')

27

## scatter plot

scatter.smooth(day, pixel)

## Pixel~day|Dog/Side

plot(Pixel, display = "Dog", inner = ~Side)

## Linearity and transformation

dogsize <- 1:10

sidesize <- 1:2

meand <- rep(0,length(dogsize))

sdd <- rep(0,length(dogsize))

means <- rep(0, length(sidesize)*length(dogsize))

sds <- rep(0, length(sidesize)*length(dogsize))

for (i in 1:length(dogsize)) {

meand[i] <- mean(pixel[Dog==dogsize[i]])

sdd[i] <- sd(pixel[Dog==dogsize[i]])

means[i] <- mean(pixel[Dog==dogsize[i] & Side=='L'])

means[length(dogsize)+i] <- mean(pixel[Dog==dogsize[i] & Side=='R'])

sds[i] <- sd(pixel[Dog==dogsize[i] & Side=='R'])

sds[length(dogsize)+i] <- sd(pixel[Dog==dogsize[i] & Side=='R'])

}

Lfit <- lm(log(sdd)~log(meand))

summary(Lfit)

plot(log(meand), log(sdd))

abline(Lfit)

Lfit <- lm(log(sds)~log(means))

summary(Lfit)

plot(log(means), log(sds))

abline(Lfit)

## lm fit

lmfit0 <- lm(pixel~day+I(day^2))

lmfit1 <- lm(pixel~day+I(day^2)+Dog)

lmfit2 <- lm(pixel~day+I(day^2)+Dog+day:Dog)

lmfit3 <- lm(pixel~day+I(day^2)+Dog+day:Dog+I(day^2):Dog)

lmfit4 <- lm(pixel~day+I(day^2)+Dog+Side)

summary(lmfit4)

anova(lmfit4)

lmfit5 <- lm(pixel~day+I(day^2)+Dog+Side+day:Dog+day:Side)

lmfitfull0 <-

lm(pixel~day+I(day^2)+Dog+Side+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side)

lmfitfull1 <-

lm(pixel~day+I(day^2)+Dog+Side+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side

+Dog:Side) ## the Best

lmfitfull2 <-

lm(pixel~day+I(day^2)+Dog+Side+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side

28

+Dog:Side+day:Dog:Side)

lmfitfull3 <-

lm(pixel~day+I(day^2)+Dog+Side+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side

+Dog:Side+day:Dog:Side+I(day^2):Dog:Side)

anova(lmfitfull1, lmfitfull0)

## lmfitfull1 best so far

lmfitfull1a <-

lm(pixel~day+I(day^2)+Dog+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side

+Dog:Side)

anova(lmfitfull1a, lmfitfull1) ## 1a better

lmfitfull1a1 <-

lm(pixel~day+I(day^2)+Dog+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side)

anova(lmfitfull1a, lmfitfull1a1) ## 1a better

lmfitfull1a2 <-

lm(pixel~day+I(day^2)+Dog+day:Dog+day:Side+I(day^2):Dog+Dog:Side)

anova(lmfitfull1a, lmfitfull1a2) ## 1a2 better

lmfitfull1a3 <-

lm(pixel~day+I(day^2)+Dog+day:Dog+I(day^2):Dog+Dog:Side)

anova(lmfitfull1a, lmfitfull1a3) ## 1a2 better

anova(lmfitfull1a2, lmfitfull1a3) ## 1a2 better

lmfitfull1b <-

lm(pixel~day+I(day^2)+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side

+Dog:Side)

anova(lmfitfull1a, lmfitfull1b) ## 1a better

lmfitfull1c <-

lm(pixel~day+I(day^2)+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side)

anova(lmfitfull1a, lmfitfull1b) ## 1a better

lmfitfull1d <-

lm(pixel~day+I(day^2)+day:Dog+day:Side+I(day^2):Dog)

anova(lmfitfull1a, lmfitfull1d) ## 1a better

lmfitfull1e <-

lm(pixel~day+I(day^2)+day:Dog+day:Side+I(day^2):Dog)

anova(lmfitfull1a, lmfitfull1d) ## 1a better

lmFinal <-

lm(pixel~day+I(day^2)+Dog+day:Dog+day:Side+I(day^2):Dog+Dog:Side) ## 1a2

length(coef(lmFinal))

summary(lmFinal)

anova(lmFinal)

graphics.off()

par(mfrow=c(4,2))

plot(lmFinal)

termplot(lmFinal, partial=T, term=1, main='Partial Residual Plot of day',

col.res='black')

29

termplot(lmFinal, partial=T, term=2, main='Partial Residual Plot of day^2',

col.res='black')

halfnorm(influence(lmFinal)$hat,main='Half Normal Plot of Leverage')

print(RatPupWeight[influence(lmFinal)$hat>0.2,])

print(influence(lmFinal)$hat>0.2)

graphics.off()

## lmList

detach(Pixel)

pixel <- Pixel$pixel

day <- Pixel$day

Dog <- Pixel$Dog

Side <- Pixel$Side

Side_in_Dog<-c(rep(1,7),rep(2,7),rep(3,7),rep(4,7), rep(5,7), rep(6,7),

rep(7,7),rep(8,7), rep(9,5),rep(10,5), rep(11, 5), rep(12,5),

rep(13, 4),rep(14,4),rep(15,4),rep(16,4),rep(17,2),

rep(18,2),rep(19,3),rep(20,3))

tmpdata <- data.frame(pixel, day, Dog, Side, Side_in_Dog)

lmListfit1 <- lmList(pixel~day+I(day^2) | Dog, tmpdata)

lmListfit2 <- lmList(pixel~day+I(day^2) | Side_in_Dog, tmpdata)

plot(lmListfit1)

plot(lmListfit2)

pairs(coef(lmListfit1))

pairs(coef(lmListfit2))

#install.packages("psych")

library(psych)

pairs.panels(coef(lmListfit1),

method = "pearson", # correlation method

hist.col = "#00AFBB",

density = TRUE, # show density plots

ellipses = TRUE # show correlation ellipses

)

dev.copy2pdf(file='Pairs graph 1.pdf')

pairs.panels(coef(lmListfit2),

method = "pearson", # correlation method

hist.col = "#00AFBB",

density = TRUE, # show density plots

ellipses = TRUE # show correlation ellipses

)

graphics.off()

## lme

require(lme4)

30

attach(Pixel)

lmefit1 <- lme(pixel~day+I(day^2), data=Pixel, random=list(Dog=~day+I(day^2),

Side=~day+I(day^2)))

lmefit1 <- lme(pixel~day+I(day^2), data=Pixel, random=list(Dog=~day+I(day^2),

Side=~day+I(day^2)))

lmefit2 <- lme(pixel~day+I(day^2), data=Pixel, random=list(Dog=~day,

Side=~1))

lmefit0 <- lmer(pixel~day+I(day^2)+(1|Dog)+(0+day|Dog), data=Pixel)

lmefit01 <- lmer(pixel~day+I(day^2)+(day|Dog), data=Pixel)

lmefit1 <- lmer(pixel~day+I(day^2)+(day+I(day^2)|Dog)+(day+day^2|Side:Dog),

data=Pixel)

lmefit2 <- lmer(pixel~day+I(day^2)+(day+I(day^2)|Dog)+(day|Side:Dog),

data=Pixel)

lmefit3 <- lmer(pixel~day+I(day^2)+(day|Dog)+(day|Side:Dog),

data=Pixel)

lmefit4 <- lmer(pixel~day+I(day^2)+(day|Dog)+(1|Side:Dog),

data=Pixel)

lmefit5 <- lmer(pixel~day+I(day^2)+(1|Dog)+(0+day|Dog)+(1|Side:Dog), data=Pixel)

lmefit6 <- lmer(pixel~day+I(day^2)+(day+I(day^2)|Dog)+(1|Side:Dog),

data=Pixel)

lmefit7 <-

lmer(pixel~day+I(day^2)+(1|Dog)+(0+day|Dog)+(0+I(day^2)|Dog)+(1|Side:Dog),

data=Pixel)

lmefit8 <- lmer(pixel~day+I(day^2)+Dog+Side+(day+I(day^2)|Dog)+(1|Side:Dog),

data=Pixel)

lmefit9 <- lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(1|Side:Dog),

data=Pixel)

lmefit10 <- lmer(pixel~day+I(day^2)+Side+(day+I(day^2)|Dog)+(1|Side:Dog),

data=Pixel)

## only lmefit9 beat lmefit6

lmefit19 <- lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(day|Side:Dog),

data=Pixel)

summary(lmefit19)

anova(lmefit19)

lmefit29 <-

lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(day+I(day^2)|Side:Dog),

data=Pixel) ## put this one in

anova(lmefit9, lmefit19)

anova(lmefit9, lmefit29)

anova(lmefit19, lmefit29)

## So far 19 is the best one

lmefit39 <-

lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(day|Side:Dog)+(0+I(day^2)|Side:D

og), data=Pixel)

31

lmefit49 <-

lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(1|Side:Dog)+(0+day|Side:Dog),

data=Pixel) ## put this one in

anova(lmefit19, lmefit49)

## Finally the best one is model lmefit19

lmeFinal <- lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(day|Side:Dog),

data=Pixel)

lmeFinal1 <- lmer(pixel~day+I(day^2)+(day+I(day^2)|Dog)+(day|Side:Dog),

data=Pixel) ##### worse than lmefitFinal

lmeFinal2 <- lmer(pixel~day+I(day^2)+Dog+(day|Dog)+(1|Side:Dog),

data=Pixel)

anova(lmeFinal, lmeFinal2)

lmeFinal3 <- lmer(pixel~day+I(day^2)+Dog+(1|Dog)+(0+day|Dog) +

(0+I(day^2)|Dog) + (1|Side:Dog) + (0+day|Side:Dog),

data=Pixel)

anova(lmeFinal, lmeFinal3)

lmeFinal4 <- lmer(pixel~day+I(day^2)+Dog+(1|Dog)+(0+day|Dog) +

(1|Side:Dog),

data=Pixel)

anova(lmeFinal, lmeFinal4)

## lmeFinal is always the best

tvalues <- c(129.91, 3,26, -5.29, 3.32, 0.19, -0.32, 0.57, 5.18, 4.03, 7.85, 1.83, 1.75)

print(pvalues <- (1-pt(abs(tvalues),60))*2)

## Diagnostic

## within-group

xyplot(resid(lmeFinal)~Side|Dog,xlab='Sides in Dog', ylab='Residual')

xyplot(resid(lmeFinal)~Dog, xlab = 'Dog', ylab='Residual')

(1:102)[resid(lmeFinal) > 20]

(1:102)[resid(lmeFinal) < -20]

qqnorm(resid(lmeFinal))

xyplot(resid(lmeFinal)~fitted(lmeFinal), group=Side, pch=c(1,2), xlab='Fitted Value',

ylab='Residual Plot', main='Residual Plot of lmefit')

xyplot(resid(lmeFinal)~fitted(lmeFinal)|Dog, xlab='Fitted Value',

ylab='Residual', main='Residual Plots for each Dog group')

xyplot(resid(lmeFinal)~fitted(lmeFinal)|Side, xlab='Fitted Value',

ylab='Residual plot for each Side group')

plot(fitted(lmeFinal)~pixel, xlab='Observed Pixel', ylab='Fitted Pixel')

abline(lm(fitted(lmeFinal)~pixel))

graphics.off()

## Diagnostic for random effects

par(mfrow=c(2,2))

32

qqnorm(ranef(lmeFinal)$Dog[,1], main='Normal Q-Q Plot for Intercept at Dog

level')

qqnorm(ranef(lmeFinal)$Dog[,2], main='Normal Q-Q Plot for Day at Dog

level')

qqnorm(ranef(lmeFinal)$Dog[,3], main='Normal Q-Q Plot for Day^2 at Dog

level')

par(mfrow=c(1,2))

qqnorm(ranef(lmeFinal)$`Side:Dog`[,1], main='Normal Q-Q plot for Intercept at

Side within Dog level')

qqnorm(ranef(lmeFinal)$`Side:Dog`[,2], main='Normal Q-Q plot for Day at

Side within Dog level')

graphics.off()

## Compare

AIC(lmeFinal)

BIC(lmeFinal)

logLik(lmeFinal)

AIC(lmFinal)

BIC(logLik(lmFinal))

logLik(lmFinal)

print(llratio <- -2*(-362.9599-(-339.9932)))

print(ddf <- 40-22)

print(pchiok <- 1-pchisq(llratio, ddf))

## fitted value plot

par(mfrow=c(1,2))

xyplot(fitted(lmeFinal)+pixel~day|Side:Dog, panel=panel.superpose,

distribute.type=TRUE, pch=c(4,20), type=c('o','p'), ylab='pixel',

main='Fitted Values by Linear Mixed Model')

xyplot(fitted(lmFinal)+pixel~day|Side:Dog, panel=panel.superpose,

distribute.type=TRUE, pch=c(4,20), type=c('o','p'), ylab='pixel',

main='Fitted Values by Linear Regression')

graphics.off()

Appendix A

Analysis of Variance Table Model 1: pixel ~ day + I(day^2) + Dog + Side + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + Dog + Side + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side Res.Df RSS Df Sum of Sq F Pr(>F) 1 61 4372.8

33

2 70 13242.2 -9 -8869.4 13.748 9.441e-12 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > ## lmfitfull1 best so far > lmfitfull1a <- + lm(pixel~day+I(day^2)+Dog+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side + +Dog:Side) > anova(lmfitfull1a, lmfitfull1) ## 1a better Analysis of Variance Table Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + Dog + Side + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Res.Df RSS Df Sum of Sq F Pr(>F) 1 62 4455.2 2 61 4372.8 1 82.427 1.1499 0.2878 > lmfitfull1a1 <- + lm(pixel~day+I(day^2)+Dog+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side) > anova(lmfitfull1a, lmfitfull1a1) ## 1a better Analysis of Variance Table Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side Res.Df RSS Df Sum of Sq F Pr(>F) 1 62 4455.2 2 71 13347.0 -9 -8891.8 13.749 7.845e-12 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > lmfitfull1a2 <- + lm(pixel~day+I(day^2)+Dog+day:Dog+day:Side+I(day^2):Dog+Dog:Side) > anova(lmfitfull1a, lmfitfull1a2) ## 1a2 better Analysis of Variance Table Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + Dog:Side Res.Df RSS Df Sum of Sq F Pr(>F) 1 62 4455.2 2 63 4692.1 -1 -236.88 3.2965 0.07426 . --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > lmfitfull1a3 <- + lm(pixel~day+I(day^2)+Dog+day:Dog+I(day^2):Dog+Dog:Side) > anova(lmfitfull1a, lmfitfull1a3) ## 1a2 better Analysis of Variance Table

34

Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + Dog + day:Dog + I(day^2):Dog + Dog:Side Res.Df RSS Df Sum of Sq F Pr(>F) 1 62 4455.2 2 63 4617.6 -1 -162.43 2.2605 0.1378 > anova(lmfitfull1a2, lmfitfull1a3) ## 1a2 better Analysis of Variance Table Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + Dog:Side Model 2: pixel ~ day + I(day^2) + Dog + day:Dog + I(day^2):Dog + Dog:Side Res.Df RSS Df Sum of Sq F Pr(>F) 1 63 4692.1 2 63 4617.6 0 74.446 > lmfitfull1b <- + lm(pixel~day+I(day^2)+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side + +Dog:Side) > anova(lmfitfull1a, lmfitfull1b) ## 1a better Analysis of Variance Table Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Res.Df RSS Df Sum of Sq F Pr(>F) 1 62 4455.2 2 70 6654.9 -8 -2199.7 3.8265 0.001039 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > lmfitfull1c <- + lm(pixel~day+I(day^2)+day:Dog+day:Side+I(day^2):Dog+I(day^2):Side) > anova(lmfitfull1a, lmfitfull1b) ## 1a better Analysis of Variance Table Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Res.Df RSS Df Sum of Sq F Pr(>F) 1 62 4455.2 2 70 6654.9 -8 -2199.7 3.8265 0.001039 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > lmfitfull1d <- + lm(pixel~day+I(day^2)+day:Dog+day:Side+I(day^2):Dog) > anova(lmfitfull1a, lmfitfull1d) ## 1a better Analysis of Variance Table

35

Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + day:Dog + day:Side + I(day^2):Dog Res.Df RSS Df Sum of Sq F Pr(>F) 1 62 4455.2 2 80 17507.5 -18 -13052 10.091 2.409e-12 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > lmfitfull1e <- + lm(pixel~day+I(day^2)+day:Dog+day:Side+I(day^2):Dog) > anova(lmfitfull1a, lmfitfull1d) ## 1a better Analysis of Variance Table Model 1: pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + I(day^2):Side + Dog:Side Model 2: pixel ~ day + I(day^2) + day:Dog + day:Side + I(day^2):Dog Res.Df RSS Df Sum of Sq F Pr(>F) 1 62 4455.2 2 80 17507.5 -18 -13052 10.091 2.409e-12 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > lmFinal <- + lm(pixel~day+I(day^2)+Dog+day:Dog+day:Side+I(day^2):Dog+Dog:Side) ## 1a2 > length(coef(lmFinal)) [1] 40 > summary(lmFinal) Call: lm(formula = pixel ~ day + I(day^2) + Dog + day:Dog + day:Side + I(day^2):Dog + Dog:Side) Residuals: Min 1Q Median 3Q Max -22.028 -4.121 -0.369 3.699 24.287 Coefficients: (1 not defined because of singularities) Estimate Std. Error t value Pr(>|t|) (Intercept) 1045.94343 4.45298 234.886 < 2e-16 *** day 1.07784 1.80718 0.596 0.553032 I(day^2) -0.10783 0.12585 -0.857 0.394793 Dog10 33.22150 63.97076 0.519 0.605355 Dog2 4.06366 6.76442 0.601 0.550168 Dog3 -3.91261 7.54531 -0.519 0.605892 Dog4 3.20157 7.54531 0.424 0.672783 Dog5 42.59467 19.83029 2.148 0.035570 * Dog6 57.61432 19.83029 2.905 0.005055 ** Dog7 88.94735 21.64211 4.110 0.000116 *** Dog8 22.02461 21.64211 1.018 0.312725 Dog9 51.22918 14.80626 3.460 0.000975 *** day:Dog10 22.13078 22.59678 0.979 0.331139 day:Dog2 2.35105 2.54480 0.924 0.359082 day:Dog3 6.09947 2.14900 2.838 0.006096 ** day:Dog4 11.11590 2.14900 5.173 2.55e-06 *** day:Dog5 8.30475 4.97841 1.668 0.100249

36

day:Dog6 2.38877 4.97841 0.480 0.633013 day:Dog7 -1.46827 5.65921 -0.259 0.796135 day:Dog8 7.87183 5.65921 1.391 0.169125 day:Dog9 -0.16272 2.22278 -0.073 0.941874 day:SideR 0.28276 0.33405 0.846 0.400509 I(day^2):Dog10 -1.67967 1.87269 -0.897 0.373172 I(day^2):Dog2 -0.11035 0.17798 -0.620 0.537491 I(day^2):Dog3 -0.20821 0.13627 -1.528 0.131526 I(day^2):Dog4 -0.39887 0.13627 -2.927 0.004754 ** I(day^2):Dog5 -0.48333 0.28152 -1.717 0.090914 . I(day^2):Dog6 -0.18922 0.28152 -0.672 0.503945 I(day^2):Dog7 -0.05765 0.31952 -0.180 0.857395 I(day^2):Dog8 -0.35757 0.31952 -1.119 0.267351 I(day^2):Dog9 NA NA NA NA Dog10:SideR -60.62987 7.32590 -8.276 1.17e-11 *** Dog2:SideR -7.48028 4.93932 -1.514 0.134917 Dog3:SideR 9.64091 5.50619 1.751 0.084827 . Dog4:SideR -1.77338 5.50619 -0.322 0.748466 Dog5:SideR 1.64485 6.13716 0.268 0.789564 Dog6:SideR -24.37515 6.13716 -3.972 0.000186 *** Dog7:SideR 26.54657 6.73061 3.944 0.000204 *** Dog8:SideR -19.05343 6.73061 -2.831 0.006223 ** Dog9:SideR -33.94654 8.85971 -3.832 0.000296 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 8.63 on 63 degrees of freedom Multiple R-squared: 0.9558, Adjusted R-squared: 0.9292 F-statistic: 35.88 on 38 and 63 DF, p-value: < 2.2e-16 > anova(lmFinal) Analysis of Variance Table Response: pixel Df Sum Sq Mean Sq F value Pr(>F) day 1 4562 4561.9 61.2517 7.181e-11 *** I(day^2) 1 20617 20617.0 276.8224 < 2.2e-16 *** Dog 9 57294 6366.0 85.4753 < 2.2e-16 *** day:Dog 9 7469 829.9 11.1433 3.488e-10 *** day:Side 1 80 80.0 1.0736 0.30409 I(day^2):Dog 8 1169 146.1 1.9617 0.06612 . Dog:Side 9 10351 1150.2 15.4431 6.271e-13 *** Residuals 63 4692 74.5 --- Signif . codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Appendix B

> summary(lmefit1) Linear mixed model fit by REML ['lmerMod'] Formula: pixel ~ day + I(day^2) + (day + I(day^2) | Dog) + (day + day^2 | Side:Dog) Data: Pixel

37

REML criterion at convergence: 810.3 Scaled residuals: Min 1Q Median 3Q Max -2.6865 -0.4857 0.0239 0.4637 3.1629 Random effects: Groups Name Variance Std.Dev. Corr Side:Dog (Intercept) 108.69536 10.4257 day 0.60441 0.7774 1.00 Dog (Intercept) 829.16140 28.7952 day 9.80357 3.1311 -0.25 I(day^2) 0.01406 0.1186 -0.31 -0.84 Residual 63.37445 7.9608 Number of obs: 102, groups: Side:Dog, 20; Dog, 10 Fixed effects: Estimate Std. Error t value (Intercept) 1073.8458 9.8984 108.487 day 5.7130 1.2410 4.604 I(day^2) -0.3335 0.0537 -6.210 Correlation of Fixed Effects: (Intr) day day -0.315 I(day^2) -0.035 -0.861 convergence code: 1 > anova(lmefit1) Analysis of Variance Table Df Sum Sq Mean Sq F value day 1 134.22 134.22 2.1179 I(day^2) 1 2443.67 2443.67 38.5593 > > summary(lmefit1) Linear mixed model fit by REML ['lmerMod'] Formula: pixel ~ day + I(day^2) + (day + I(day^2) | Dog) + (day + day^2 | Side:Dog) Data: Pixel REML criterion at convergence: 810.3 Scaled residuals: Min 1Q Median 3Q Max -2.6865 -0.4857 0.0239 0.4637 3.1629 Random effects: Groups Name Variance Std.Dev. Corr Side:Dog (Intercept) 108.69536 10.4257 day 0.60441 0.7774 1.00 Dog (Intercept) 829.16140 28.7952 day 9.80357 3.1311 -0.25 I(day^2) 0.01406 0.1186 -0.31 -0.84 Residual 63.37445 7.9608 Number of obs: 102, groups: Side:Dog, 20; Dog, 10 Fixed effects:

38

Estimate Std. Error t value (Intercept) 1073.8458 9.8984 108.487 day 5.7130 1.2410 4.604 I(day^2) -0.3335 0.0537 -6.210 Correlation of Fixed Effects: (Intr) day day -0.315 I(day^2) -0.035 -0.861 convergence code: 1 > anova(lmefit1) Analysis of Variance Table Df Sum Sq Mean Sq F value day 1 134.22 134.22 2.1179 I(day^2) 1 2443.67 2443.67 38.5593 > > lmefit2 <- lme(pixel~day+I(day^2), data=Pixel, random=list(Dog=~day, + Side=~1)) > > summary(lmefit2) Linear mixed-effects model fit by REML Data: Pixel AIC BIC logLik 841.2102 861.9712 -412.6051 Random effects: Formula: ~day | Dog Structure: General positive-definite, Log-Cholesky parametrization StdDev Corr (Intercept) 28.36990 (Intr) day 1.84375 -0.555 Formula: ~1 | Side %in% Dog (Intercept) Residual StdDev: 16.82431 8.989606 Fixed effects: pixel ~ day + I(day^2) Value Std.Error DF t-value p-value (Intercept) 1073.3391 10.171686 80 105.52225 0 day 6.1296 0.879321 80 6.97083 0 I(day^2) -0.3674 0.033945 80 -10.82179 0 Correlation: (Intr) day day -0.517 I(day^2) 0.186 -0.668 Standardized Within-Group Residuals: Min Q1 Med Q3 Max -2.8290572 -0.4491811 0.0255493 0.5572163 2.7519651 Number of Observations: 102 Number of Groups: Dog Side %in% Dog 10 20 > anova(lmefit2) numDF denDF F-value p-value (Intercept) 1 80 16771.715 <.0001

39

day 1 80 0.116 0.7345 I(day^2) 1 80 117.111 <.0001 > lmefit0 <- lmer(pixel~day+I(day^2)+(1|Dog)+(0+day|Dog), data=Pixel) > lmefit01 <- lmer(pixel~day+I(day^2)+(day|Dog), data=Pixel) > lmefit1 <- lmer(pixel~day+I(day^2)+(day+I(day^2)|Dog)+(day+day^2|Side:Dog), + data=Pixel) > lmefit2 <- lmer(pixel~day+I(day^2)+(day+I(day^2)|Dog)+(day|Side:Dog), + data=Pixel) > lmefit3 <- lmer(pixel~day+I(day^2)+(day|Dog)+(day|Side:Dog), + data=Pixel) > lmefit4 <- lmer(pixel~day+I(day^2)+(day|Dog)+(1|Side:Dog), + data=Pixel) ## !!!!!!!!! > lmefit5 <- lmer(pixel~day+I(day^2)+(1|Dog)+(0+day|Dog)+(1|Side:Dog), data=Pixel) > lmefit6 <- lmer(pixel~day+I(day^2)+(day+I(day^2)|Dog)+(1|Side:Dog), + data=Pixel) ## !!!!!!!!! > summary(lmefit6) Linear mixed model fit by REML ['lmerMod'] Formula: pixel ~ day + I(day^2) + (day + I(day^2) | Dog) + (1 | Side:Dog) Data: Pixel REML criterion at convergence: 813.3 Scaled residuals: Min 1Q Median 3Q Max -2.43538 -0.47622 0.03253 0.43589 2.97456 Random effects: Groups Name Variance Std.Dev. Corr Side:Dog (Intercept) 259.38092 16.1053 Dog (Intercept) 779.62551 27.9218 day 10.42998 3.2295 -0.26 I(day^2) 0.01329 0.1153 -0.28 -0.86 Residual 67.70714 8.2284 Number of obs: 102, groups: Side:Dog, 20; Dog, 10 Fixed effects: Estimate Std. Error t value (Intercept) 1074.52882 10.02075 107.230 day 5.49232 1.23552 4.445 I(day^2) -0.31904 0.05041 -6.328 Correlation of Fixed Effects: (Intr) day day -0.344 I(day^2) -0.021 -0.873 convergence code: 1 > anova(lmefit6) Analysis of Variance Table Df Sum Sq Mean Sq F value day 1 330.55 330.55 4.882 I(day^2) 1 2711.56 2711.56 40.048 > > lmefit7 <-

40

+ lmer(pixel~day+I(day^2)+(1|Dog)+(0+day|Dog)+(0+I(day^2)|Dog)+(1|Side:Dog), + data=Pixel) ## !!!!!!!!!! > summary(lmefit7) Linear mixed model fit by REML ['lmerMod'] Formula: pixel ~ day + I(day^2) + (1 | Dog) + (0 + day | Dog) + (0 + I(day^2) | Dog) + (1 | Side:Dog) Data: Pixel REML criterion at convergence: 826.7 Scaled residuals: Min 1Q Median 3Q Max -2.63975 -0.45962 0.02743 0.55890 2.73078 Random effects: Groups Name Variance Std.Dev. Side.Dog (Intercept) 2.846e+02 16.87041 Dog I(day^2) 2.276e-03 0.04771 Dog.1 day 3.746e+00 1.93545 Dog.2 (Intercept) 7.042e+02 26.53650 Residual 7.716e+01 8.78385 Number of obs: 102, groups: Side:Dog, 20; Dog, 10 Fixed effects: Estimate Std. Error t value (Intercept) 1072.40903 9.68675 110.709 day 6.31417 0.93656 6.742 I(day^2) -0.37512 0.04198 -8.937 Correlation of Fixed Effects: (Intr) day day -0.210 I(day^2) 0.193 -0.638 > anova(lmefit7) Analysis of Variance Table Df Sum Sq Mean Sq F value day 1 140.5 140.5 1.8211 I(day^2) 1 6162.0 6162.0 79.8644 > > > lmefit8 <- lmer(pixel~day+I(day^2)+Dog+Side+(day+I(day^2)|Dog)+(1|Side:Dog), + data=Pixel) > lmefit9 <- lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(1|Side:Dog), + data=Pixel) ## !!!!!!!!! > summary(lmefit9) Linear mixed model fit by REML ['lmerMod'] Formula: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (1 | Side:Dog) Data: Pixel REML criterion at convergence: 729.8

41

Scaled residuals: Min 1Q Median 3Q Max -2.3868 -0.5134 0.0696 0.4737 2.9572 Random effects: Groups Name Variance Std.Dev. Corr Side:Dog (Intercept) 287.39707 16.9528 Dog (Intercept) 77.40022 8.7977 day 14.43036 3.7987 0.18 I(day^2) 0.01294 0.1137 -0.05 -0.99 Residual 70.25703 8.3819 Number of obs: 102, groups: Side:Dog, 20; Dog, 10 Fixed effects: Estimate Std. Error t value (Intercept) 1043.94237 14.93062 69.920 day 4.42683 1.40205 3.157 I(day^2) -0.25833 0.04994 -5.173 Dog10 56.16444 24.99952 2.247 Dog2 1.79265 21.06698 0.085 Dog3 -4.54461 20.99398 -0.216 Dog4 5.13051 20.99398 0.244 Dog5 70.69182 22.68475 3.116 Dog6 55.22997 22.68475 2.435 Dog7 108.16314 22.71789 4.761 Dog8 24.65224 22.71789 1.085 Dog9 29.62569 25.07613 1.181 Correlation of Fixed Effects: (Intr) day I(d^2) Dog10 Dog2 Dog3 Dog4 Dog5 Dog6 Dog7 Dog8 day 0.016 I(day^2) 0.006 -0.945 Dog10 -0.598 -0.110 0.095 Dog2 -0.705 0.000 0.000 0.421 Dog3 -0.700 -0.054 0.095 0.425 0.502 Dog4 -0.700 -0.054 0.095 0.425 0.502 0.525 Dog5 -0.653 -0.066 0.078 0.397 0.464 0.476 0.476 Dog6 -0.653 -0.066 0.078 0.397 0.464 0.476 0.476 0.438 Dog7 -0.652 -0.063 0.075 0.397 0.464 0.475 0.475 0.437 0.437 Dog8 -0.652 -0.063 0.075 0.397 0.464 0.475 0.475 0.437 0.437 0.436 Dog9 -0.596 -0.109 0.094 0.367 0.420 0.424 0.424 0.396 0.396 0.395 0.395 convergence code: 0 unable to evaluate scaled gradient Model failed to converge: degenerate Hessian with 1 negative eigenvalues > anova(lmefit9) Analysis of Variance Table Df Sum Sq Mean Sq F value day 1 1758.0 1757.96 25.0218 I(day^2) 1 2253.1 2253.09 32.0692 Dog 9 3275.6 363.96 5.1804 > >

42

> lmefit10 <- lmer(pixel~day+I(day^2)+Side+(day+I(day^2)|Dog)+(1|Side:Dog), + data=Pixel) > ## only lmefit9 beat lmefit6 > lmefit19 <- lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(day|Side:Dog), + data=Pixel) > summary(lmefit19) Linear mixed model fit by REML ['lmerMod'] Formula: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day | Side:Dog) Data: Pixel REML criterion at convergence: 725.9 Scaled residuals: Min 1Q Median 3Q Max -2.6844 -0.4211 -0.0060 0.4649 3.1645 Random effects: Groups Name Variance Std.Dev. Corr Side:Dog (Intercept) 106.9086 10.3397 day 1.0187 1.0093 1.00 Dog (Intercept) 92.7085 9.6285 day 13.5799 3.6851 0.38 I(day^2) 0.0126 0.1122 -0.38 -1.00 Residual 64.8810 8.0549 Number of obs: 102, groups: Side:Dog, 20; Dog, 10 Fixed effects: Estimate Std. Error t value (Intercept) 1045.53113 11.30284 92.502 day 4.51704 1.38676 3.257 I(day^2) -0.26435 0.05001 -5.286 Dog10 53.76256 21.84830 2.461 Dog2 0.52257 15.76778 0.033 Dog3 -8.37906 15.83115 -0.529 Dog4 -3.35252 15.83115 -0.212 Dog5 69.47243 18.33215 3.790 Dog6 55.32561 18.33215 3.018 Dog7 110.67758 18.35956 6.028 Dog8 20.98427 18.35956 1.143 Dog9 29.30346 21.92173 1.337 Correlation of Fixed Effects: (Intr) day I(d^2) Dog10 Dog2 Dog3 Dog4 Dog5 Dog6 Dog7 Dog8 day 0.114 I(day^2) -0.065 -0.938 Dog10 -0.523 -0.137 0.116 Dog2 -0.698 0.000 0.000 0.361 Dog3 -0.684 -0.066 0.114 0.363 0.498 Dog4 -0.684 -0.066 0.114 0.363 0.498 0.523 Dog5 -0.602 -0.092 0.111 0.320 0.430 0.445 0.445 Dog6 -0.602 -0.092 0.111 0.320 0.430 0.445 0.445 0.384 Dog7 -0.601 -0.089 0.107 0.320 0.429 0.444 0.444 0.382 0.382

43

Dog8 -0.601 -0.089 0.107 0.320 0.429 0.444 0.444 0.382 0.382 0.381 Dog9 -0.522 -0.136 0.114 0.280 0.360 0.361 0.361 0.319 0.319 0.318 0.318 convergence code: 0 > anova(lmefit19) Analysis of Variance Table Df Sum Sq Mean Sq F value day 1 1299.9 1299.87 20.0348 I(day^2) 1 2583.5 2583.46 39.8185 Dog 9 4955.0 550.56 8.4857 > lmefit29 <- + lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(day+I(day^2)|Side:Dog), + data=Pixel) ## put this one in > anova(lmefit9, lmefit19) refitting model(s) with ML (instead of REML) Data: Pixel Models: lmefit9: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (1 | lmefit9: Side:Dog) lmefit19: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day | lmefit19: Side:Dog) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) lmefit9 20 830.33 882.83 -395.16 790.33 lmefit19 22 825.85 883.60 -390.92 781.85 8.4821 2 0.01439 * --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > anova(lmefit9, lmefit29) refitting model(s) with ML (instead of REML) Data: Pixel Models: lmefit9: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (1 | lmefit9: Side:Dog) lmefit29: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day + lmefit29: I(day^2) | Side:Dog) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) lmefit9 20 830.33 882.83 -395.16 790.33 lmefit29 25 826.81 892.44 -388.41 776.81 13.517 5 0.01899 * --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > anova(lmefit19, lmefit29) refitting model(s) with ML (instead of REML) Data: Pixel Models: lmefit19: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day | lmefit19: Side:Dog) lmefit29: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day + lmefit29: I(day^2) | Side:Dog) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) lmefit19 22 825.85 883.60 -390.92 781.85 lmefit29 25 826.81 892.44 -388.41 776.81 5.0346 3 0.1693 > ## So far 19 is the best one

44

> lmefit39 <- + lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(day|Side:Dog)+(0+I(day^2)|Side:Dog), data=Pixel) > lmefit49 <- + lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(1|Side:Dog)+(0+day|Side:Dog), + data=Pixel) ## put this one in > anova(lmefit19, lmefit49) refitting model(s) with ML (instead of REML) Data: Pixel Models: lmefit49: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (1 | lmefit49: Side:Dog) + (0 + day | Side:Dog) lmefit19: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day | lmefit19: Side:Dog) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) lmefit49 21 824.45 879.57 -391.22 782.45 lmefit19 22 825.85 883.60 -390.92 781.85 0.5998 1 0.4387 > > ## Finally the best one is model lmefit19 > ## 12+30 = 42 parameters so 102-42=60 d.f for t-test > ## use (1-pt(,60))*2 > lmeFinal <- lmer(pixel~day+I(day^2)+Dog+(day+I(day^2)|Dog)+(day|Side:Dog), + data=Pixel) > lmeFinal1 <- lmer(pixel~day+I(day^2)+(day+I(day^2)|Dog)+(day|Side:Dog), + data=Pixel) ##### worse than lmefitFinal > lmeFinal2 <- lmer(pixel~day+I(day^2)+Dog+(day|Dog)+(1|Side:Dog), + data=Pixel) > anova(lmeFinal, lmeFinal2) refitting model(s) with ML (instead of REML) Data: Pixel Models: lmeFinal2: pixel ~ day + I(day^2) + Dog + (day | Dog) + (1 | Side:Dog) lmeFinal: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day | lmeFinal: Side:Dog) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) lmeFinal2 17 832.85 877.48 -399.43 798.85 lmeFinal 22 825.85 883.60 -390.92 781.85 17.006 5 0.004488 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > lmeFinal3 <- lmer(pixel~day+I(day^2)+Dog+(1|Dog)+(0+day|Dog) + + (0+I(day^2)|Dog) + (1|Side:Dog) + (0+day|Side:Dog), + data=Pixel) > anova(lmeFinal, lmeFinal3) refitting model(s) with ML (instead of REML) Data: Pixel Models: lmeFinal3: pixel ~ day + I(day^2) + Dog + (1 | Dog) + (0 + day | Dog) + lmeFinal3: (0 + I(day^2) | Dog) + (1 | Side:Dog) + (0 + day | Side:Dog)

45

lmeFinal: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day | lmeFinal: Side:Dog) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) lmeFinal3 18 825.17 872.42 -394.58 789.17 lmeFinal 22 825.85 883.60 -390.92 781.85 7.3209 4 0.1199 > lmeFinal4 <- lmer(pixel~day+I(day^2)+Dog+(1|Dog)+(0+day|Dog) + + (1|Side:Dog), + data=Pixel) > anova(lmeFinal, lmeFinal4) refitting model(s) with ML (instead of REML) Data: Pixel Models: lmeFinal4: pixel ~ day + I(day^2) + Dog + (1 | Dog) + (0 + day | Dog) + lmeFinal4: (1 | Side:Dog) lmeFinal: pixel ~ day + I(day^2) + Dog + (day + I(day^2) | Dog) + (day | lmeFinal: Side:Dog) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) lmeFinal4 16 833.94 875.94 -400.97 801.94 lmeFinal 22 825.85 883.60 -390.92 781.85 20.093 6 0.002666 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

05 ‘.’ 0.1 ‘ ’ 1

8. References Andrew Gelman and Jennifer Hill. (2007). Data Analysis Using Regression and

Multilevel/Hierarchical Models. New York: Cambridge University Press.

Andrzej Gatecki and Tomasz Burzykowski. (2013). Linear Mixed-Effects Models Using

R. Springer.

Bates, Jose C. Pinheiro and Douglas M. Bates. (2002). Mixed Effects Models in S and S-

Plus. Springer.

Cai, S. (n.d.). Analyzing Pixel Intensity Data Using Linear Mixed Model.