21st-century statistics: we've never done it that way before!
DESCRIPTION
21st-Century Statistics: We've never done it that way before!. Robin High Statistical Programmer and Consultant Information Services. Relevant Topics. Research with Small Samples (Exact Tests) Survey Research Design Analysis (Frequencies, Means, Regression) Generalized Linear Models - PowerPoint PPT PresentationTRANSCRIPT
21st-Century Statistics:We've never done it that way before!
Robin High
Statistical Programmer and Consultant
Information Services
Relevant Topics
Research with Small Samples (Exact Tests)
Survey ResearchA. DesignB. Analysis (Frequencies, Means, Regression)
Generalized Linear Models
Linear Models
Where to Find More “Information”
www.uoregon.edu/~robinh/statistics.html
GENERALIZED LINEAR MODELS
http://www.uoregon.edu/~robinh/genmod_sas.html
LINEAR MODELS: IS THE LINEAR MODEL PROCEDURE GLM OBSOLETE?
http://www.uoregon.edu/~robinh/mixed_sas.html
1908: Student’s t-test
• William Sealy Gosset (1876-1937) who worked at the Guinness Brewery in Dublin, otherwise known as “Student”.
• Derivation of the t-distribution was published as The Probable Error of a Mean.
1912-1922
Sir Ronald A. Fisher (1890-1962)
Notable Contributions to Statistics (among many)
Maximum Likelihood Estimation
Analysis of Variance
Maximum Likelihood Estimation to Extract “Information”
• 1921 – Fisher utilized information supplied by the data to estimate unknown parameters
• The Variance/Covariance matrix shows how this information has been utilized
• To this day the name "Fisher Information
Matrix" has been attached to it
Maximum Likelihood Estimation: Find the estimate of the parameter that maximizes the likelihood function
Generalized Linear Models• Data analysis methods introduced by Nelder and Wedderburn
(1972)
• Developed a unified theory for a variety of statistical models
• Maximum likelihood estimation works directly with a model chosen for the specific characteristics of the data
• OUTCOMES belong to a member of the exponential family:
Normal Negative Binomial Binomial Inverse Gaussian Poisson Gamma
One "problematic" topic of statistical analysis:
Analysis of data from non-normal distributions as if they were normal
The Choice Today
Select an appropriate model based on the distributional assumptions of the data
http://www.uoregon.edu/~robinh/seven_prblms.txt
Linear Statistical Models
• T-Test• Multiple Linear Regression• Analysis of Variance• Analysis of Covariance
• All these techniques utilize “least squares” to estimate parameters
• Computing different types of “Sums of Squares” is fundamental
ANALYSIS OF VARIANCEAssumptions
1. Independent observations (one observation for each subject randomly assigned to two or more classification groups)
2. Residuals of the model are normally distributed
3. Residuals have equal variances within each classification group (i.e., homoskedasticity)
• The science of statistics allows us to interpret the following statement:
Before you become too entranced with gorgeous gadgets and mesmerizing video displays, let me remind you that information is not knowledge, knowledge is not wisdom, and wisdom is not foresight. Each grows out of the other, and we need them all.” (Arthur C. Clark)
• Bold-face words are now interpreted through simple example
Design an experiment to evaluate two teaching methods
Two groups of randomly assigned subjects
Group 1: assigned the standard teaching method
Group 2: assigned the new teaching method
y: Assessment (which is a continuous response )
Want to test Hypotheses:
Null : HO: mu_1 = mu_2
Alternative: HA: mu_1 <> mu_2
Example
Group y Group y Group y Group y Group y
1 27.30 1 27.32 1 27.65 1 27.43 1 27.80 1 28.10 1 27.93 1 27.59 1 27.44 1 27.91 1 27.40 1 27.32 1 27.73 1 27.42 1 27.61 1 27.70 1 27.62 1 27.79 1 27.65 1 27.64 1 28.00 1 27.34 1 27.73 1 27.83 1 27.51 1 28.10 1 27.63 1 27.24 1 27.81 1 27.67 1 27.40 1 27.42 1 28.16 1 28.07 1 27.69 1 27.10 1 27.53 1 28.01 1 27.73 1 27.68 2 28.40 2 28.11 2 28.26 2 27.96 2 28.63 2 27.80 2 27.91 2 28.30 2 28.10 2 28.03 2 28.10 2 28.43 2 28.39 2 28.53 2 27.98 2 28.30 2 27.84 2 27.88 2 28.44 2 28.17 2 27.90 2 27.63 2 27.99 2 28.07 2 28.17 2 27.60 2 28.31 2 28.03 2 28.32 2 27.89 2 28.50 2 28.33 2 28.33 2 28.13 2 27.85 2 27.90 2 27.90 2 27.41 2 27.85 2 28.43 2 28.40 2 28.19 2 28.20 2 28.15 2 27.84 2 27.70 2 28.36 2 27.92 2 28.02 2 27.65
InformationLooking at the data alone is too overwhelming to interpret.
Knowledge
• Apply statistical methods to interpret data
• Make table of summary statistics
• Make side-by-side box plots of data
• Compute p-value from two-sample t-test
Summary Statistics
------------------------------| | y || |------------------|| | N | Mean | Var ||---------+---+------+-------||group | | | ||1 | 40| 27.65| 0.0676||2 | 50| 28.09| 0.0739|------------------------------
Graphical Display
Wisdom
• Observe a difference in means of .44 with group 2 larger than group 1
• Produces a highly significant p-value (<.0001)
• Is this difference meaningful from your expectations of what a significant improvement should be?
Foresight
• If the difference is meaningful, is it worth the time and expenses required to implement the new program?
Statistical Computations• Examples shown with SAS code
• Similar results for other programs (SPSS, SPLUS, STATA, etc.)
• Point-and-click interface with statistical programs is NOT recommended!
• Syntax method is utilized since “writing program code is a good way to debug your thinking” (Bill Venables, 1997) and as you proceed it clearly documents your analysis process
1980’s: T-test with PROC GLM
PROC GLM DATA=indata;CLASS group;MODEL y = group / solution ss3;ESTIMATE ‘group 1 vs 2’ group 1 -1 ;LSMEANS group / stderr pdiff tdiff;RUN;
Computes Sums of Squares and Mean Squares to compute F-tests
1990’s: T-test with PROC MIXED
PROC MIXED DATA=indata METHOD=type3;CLASS group;MODEL y = group / solution ;ESTIMATE ‘group 1 vs 2’ group 1 -1 ;LSMEANS group / diff ;RUN;
Computes Sums of Squares, Mean Squares, and F-tests just like PROC GLM
• Why do we need yet another statistical procedure for ANOVA?
• Is PROC GLM really obsolete?
Real World Situations• GLM was designed primarily a “fixed”
effects analysis tool and still works fine for these situations
• It’s the “other” situations you should now consider alternatives
• Truly independent observations rarely achieved
And there are many “what ifs”?
• Unequal variances??
One or more group variances are notably different from the others
Prior solution with GLM: Levene’s test to detect and Weighted Least Squares for estimation
MIXED: Unequal Variance Model
PROC MIXED DATA=indat;CLASS group;MODEL y = group / solution;REPEATED / GROUP=group ;LSMEANS group / diff ;RUN;
• The REPEATED statement essentially does weighted least squares
RANDOM Effects??
Experimental data frequently are collected in clusters by design or the nature of the study
Subjects who belong to the same cluster are influenced by common factor
Prior solution: GLM can work with random effects, though it is not very elegant
MIXED: RANDOM Intercepts Model
PROC MIXED DATA=indat;
CLASS group site;
MODEL y = group / solution ;
RANDOM int / SUBJECT=site ;
LSMEANS group / diff;
RUN;
RANDOM site ; * can also enter this statement;
REPEATED Measurements??
• Can achieve greater efficiencies by collecting multiple observations from subjects
• Subjects tend to respond alike across trials
• Along with Random effects, this leads to a Within Subject Covariance Matrix
• Limited number of possibilities under GLM: * Independence (NOT good) * Compound Symmetry (e.g., Equal variances and equal covariances) * Unstructured (Multivariate)
Variety of ways to work with the Within Subject Covariance Matrix
• MIXED can work with it the same as GLM
• Structure and content the within subject covariance matrix can take many forms
• In the 21st Century you can consider many additional possibilities
GLM: REPEATED MEASURES
1980’s data structure Data placed in a multivariate format:
id group time1 time2 time3 1 1 5 6 8 2 1 4 . 7 3 2 6 8 9
PROC GLM DATA=mult_dat;CLASS group ;MODEL time1 time2 time3 = group / nouni ;REPEATED time 3 / PrintE summary;RUN;
MIXED: REPEATED MEASURES2000: Data are placed in univariate format:
id group time y 1 1 1 5 1 1 2 6 1 1 3 8 2 1 1 4 2 1 2 . 2 1 3 7
PROC MIXED DATA=indat;CLASS group id time;MODEL y = group time group*time / solution ;REPEATED / SUBJECT=id TYPE=ar(1) rcorr; *autoregressive;RUN;
Advantages of MIXED Model
• Allows data to be Missing at Random
• Greater variety and much more flexibility for working with within-subject covariance structures (e.g., type=cs or type=ar(1) or type=un among many others)
• Random effects allow one to compute more appropriate standard errors, F-tests, T-tests, and their associated pvalues
Other Advantages
• Options that print the actual within subject covariance and correlation matrices (e.g., rcorr and vcorr)
• Combined with Output Delivery System offers a powerful way to compute means, differences in means with pvalues, and then present them in table or graphs
One Important Feature
• Utilizes Restricted Maximum Likelihood (REML) to estimate parameters for models with complex covariance structures
• Can only find the peak of the likelihood function (next slide) through iterative search methods
• Models with more complex covariance structures don’t produce “Sums of Squares”
Maximum Likelihood Estimation
MIXED as a Study Planning Tool
Assume you will have three groups with these anticipated means
• Group 1 = 10 Group 2 = 15 Group 3 = 20• n=10 subjects per group• Standard Deviation of response = 4
• What is the “power” of this design to detect differences in these 3 means?
ODS OUTPUT TESTS3=tst3;
PROC MIXED DATA=means NOprofile;CLASS group;MODEL mn_y = group ;PARMS (16) / NOiter;RUN;
With an intermediate computation you find:
Power = 0.7614
And Where does this lead to the 21st Century
Many Training Opportunities
•Awareness of Many Possibilities
•Better (more appropriate) statistical tests
•Collaboration essential
GLIMMIX (2008)
Utilizes features of Generalized Linear Models (for a variety of distributions) containing Random Effects (within and between subjects)