Tips for Researchers on Tips for Researchers on Completing the Data Completing the Data

Analysis Section of the IRB Analysis Section of the IRB Application Application

Don Allensworth-Davies, MScDon Allensworth-Davies, MScStatistical Manager, Data Coordinating CenterStatistical Manager, Data Coordinating Center

Boston University School of Public HealthBoston University School of Public HealthIRB Reviewer, Panel BlueIRB Reviewer, Panel Blue

February 15, 2006February 15, 2006

““To write it, it took three To write it, it took three months; to conceive it three months; to conceive it three minutes; to collect the data minutes; to collect the data in it — all my life.”in it — all my life.”

F. Scott Fitzgerald, referring to his novel F. Scott Fitzgerald, referring to his novel This Side of Paradise in a letter to the This Side of Paradise in a letter to the Booksellers’ Convention, April 1920Booksellers’ Convention, April 1920

Data Analysis and Human Data Analysis and Human Subjects ProtectionSubjects Protection

Research means a systematic Research means a systematic investigation, including research investigation, including research development, testing and development, testing and evaluation, evaluation, designed to develop or designed to develop or contribute to generalizable contribute to generalizable knowledge.knowledge.

§46.102(d) Definitions, Federal Regulations Title 45, §46.102(d) Definitions, Federal Regulations Title 45, Public Welfare, Department of Health and Human Public Welfare, Department of Health and Human Services, Part 46, Protection of Human SubjectsServices, Part 46, Protection of Human Subjects

Data Analysis and Human Data Analysis and Human Subjects Protection (cont’d)Subjects Protection (cont’d)

Risks to subjects are minimized: (i) By Risks to subjects are minimized: (i) By using procedures which are consistent using procedures which are consistent with sound research design and which with sound research design and which do not unnecessarily expose subjects do not unnecessarily expose subjects to riskto risk, and (ii) whenever appropriate, , and (ii) whenever appropriate, by using procedures already being by using procedures already being performed on the subjects for performed on the subjects for diagnostic or treatment purposes.diagnostic or treatment purposes.

§46.111 Criteria for IRB Approval of Research, Federal §46.111 Criteria for IRB Approval of Research, Federal Regulations Title 45, Public Welfare, Department of Regulations Title 45, Public Welfare, Department of Health and Human Services, Part 46, Protection of Health and Human Services, Part 46, Protection of Human SubjectsHuman Subjects

IRB: Ethics and Human Research, September-October 2003

Available on the IRB Website

What the IRB Considers in What the IRB Considers in Reviewing an AnalysisReviewing an Analysis

Is the sample size adequate to Is the sample size adequate to answer the research question?answer the research question?– Is the sample size large enough to Is the sample size large enough to

ensure sufficient data to answer the ensure sufficient data to answer the research question?research question?

– Is the sample size justified?Is the sample size justified?

What the IRB Considers in What the IRB Considers in Reviewing an Analysis Reviewing an Analysis

(cont’d)(cont’d) Is the analysis plan clearly described Is the analysis plan clearly described

and adequate to answer the research and adequate to answer the research question?question?– Is it clear which variables will be analyzed?Is it clear which variables will be analyzed?– Is it clear what comparisons will be made?Is it clear what comparisons will be made?– Are the methods being used appropriate Are the methods being used appropriate

for the study design and the information for the study design and the information that was collected?that was collected?

– How will the researcher know whether or How will the researcher know whether or not the study objectives have been met?not the study objectives have been met?

Where Data Analysis Where Data Analysis Appears on the IRB Appears on the IRB

ApplicationApplication

Tip #1: Remember to Include Tip #1: Remember to Include All ‘At Risk’ Subjects and All ‘At Risk’ Subjects and

Withdrawals in Total Sample Withdrawals in Total Sample SizeSize

Sample SizeSample Size == Consent and fully participate in Consent and fully participate in

studystudy+ Expected dropouts, withdrawals, and + Expected dropouts, withdrawals, and

terminationsterminations+ For some studies: Screened and not + For some studies: Screened and not

enrolled enrolled -IF--IF- subjects were placed ‘at risk’ subjects were placed ‘at risk’

during screening (e.g., blood draws, during screening (e.g., blood draws, collection of identifiable information)collection of identifiable information)

Sample Size and the IRB Sample Size and the IRB ApplicationApplication

Sample Size and the IRB Sample Size and the IRB Application: Special Application: Special

ConsiderationsConsiderations International ResearchInternational Research

– If a BUMC researcher will be If a BUMC researcher will be conducting research and having conducting research and having contact with human subjects in contact with human subjects in another country, then these another country, then these subjects may "belong to BUMC" subjects may "belong to BUMC" and therefore should be and therefore should be reported in both the local and reported in both the local and worldwide sample sizesworldwide sample sizes

Sample Size and the IRB Sample Size and the IRB Application: Special Considerations Application: Special Considerations

(cont’d)(cont’d)

Research Involving Proxy Data CollectionResearch Involving Proxy Data Collection– If data are being collected about both the If data are being collected about both the

subject and, through them, about another subject and, through them, about another person then both should be counted in the person then both should be counted in the sample sizesample size

– Examples: Mother-child studies, studies of Examples: Mother-child studies, studies of primary relationship or sexual partners, primary relationship or sexual partners, twins’ studiestwins’ studies

If a study involves interviewing multiple If a study involves interviewing multiple groups (e.g., surveying patient, MDs, nurses, groups (e.g., surveying patient, MDs, nurses, and social workers) ALL SUBJECTS should be and social workers) ALL SUBJECTS should be considered in the sample sizeconsidered in the sample size

Sample Size and the IRB Sample Size and the IRB Progress ReportProgress Report

Should not exceed total approved by

IRB

Sample Size and the IRB Sample Size and the IRB Progress ReportProgress Report

Tip #2: Correctly Justify the Tip #2: Correctly Justify the Sample SizeSample Size

Option #1: Traditional power Option #1: Traditional power calculation describing power that calculation describing power that will be achieved to detect a will be achieved to detect a specific difference or changespecific difference or change

Option #2: FeasibilityOption #2: Feasibility– Based on the researcher’s knowledge Based on the researcher’s knowledge

of the potential subject pool of the potential subject pool – Based on the resources available to Based on the resources available to

the researcher to conduct the studythe researcher to conduct the study

Tip #2: Correctly Justify the Tip #2: Correctly Justify the Sample Size (cont’d)Sample Size (cont’d)

IMPORTANT:IMPORTANT:

Simply stating that the Simply stating that the research is a pilot study is research is a pilot study is NOTNOT adequate justification adequate justification

of sample sizeof sample size

Tip #3: Clearly Describe Tip #3: Clearly Describe Each Analysis That Will Be Each Analysis That Will Be

PerformedPerformedExample:Example:

“ “Study Objective #1, Identify Demographic Differences in Study Objective #1, Identify Demographic Differences in Unsafe Sexual Behavior: To test for differences in proportions Unsafe Sexual Behavior: To test for differences in proportions between different groups classified by sex, age, country of between different groups classified by sex, age, country of birth, education, and income, we will use chi-square or birth, education, and income, we will use chi-square or Fisher's exact tests (if data are sparse). We will present test Fisher's exact tests (if data are sparse). We will present test statistics and associated p-values. Conventional criteria (p < statistics and associated p-values. Conventional criteria (p < 0.05) will be used to identify statistically significant 0.05) will be used to identify statistically significant differences. Significant differences between the study differences. Significant differences between the study groups may indicate potential confounding, and we will groups may indicate potential confounding, and we will adjust using logistic regression models for these potential adjust using logistic regression models for these potential confounders, if they are found ultimately to be associated confounders, if they are found ultimately to be associated with our study outcome of unsafe sexual behavior.”with our study outcome of unsafe sexual behavior.”

Tip #3: Clearly Describe Each Tip #3: Clearly Describe Each Analysis That Will Be Performed Analysis That Will Be Performed

(cont’d)(cont’d) Variables to be analyzed:Variables to be analyzed: sex, age, sex, age,

country of birth, education, country of birth, education, income, unsafe sexual behaviorincome, unsafe sexual behavior

Comparisons to be made:Comparisons to be made: test for test for differences in proportions differences in proportions

Methods to be used:Methods to be used: chi-square or chi-square or Fisher's exact tests (if data are Fisher's exact tests (if data are sparse)sparse)– Appropriate for categorical Appropriate for categorical

bivariate analyses?bivariate analyses? Yes Yes

Tip #3: Clearly Describe Each Tip #3: Clearly Describe Each Analysis That Will Be Performed Analysis That Will Be Performed

(cont’d)(cont’d) How will the researcher know if How will the researcher know if

the study objective has been met the study objective has been met (i.e., if there is a difference in (i.e., if there is a difference in proportions)?proportions)? Conventional criteria Conventional criteria (p < 0.05) will be used to assess (p < 0.05) will be used to assess statistically significant differencesstatistically significant differences

In this example researcher went In this example researcher went one step further and also one step further and also proposed analytic methods to help proposed analytic methods to help control for bias/confoundingcontrol for bias/confounding

Tip #4: Remember That Some Tip #4: Remember That Some Study Designs Require Special Study Designs Require Special

MethodsMethods

Randomized clinical trialsRandomized clinical trials– Once randomized, always analyzedOnce randomized, always analyzed

Longitudinal and case-crossover studiesLongitudinal and case-crossover studies– Methods for repeated measures or pairsMethods for repeated measures or pairs

Matched case-control studiesMatched case-control studies– Methods that take the matching into accountMethods that take the matching into account

Focus groups and open-ended subject Focus groups and open-ended subject interviewsinterviews– Qualitative methodsQualitative methods

Characteristics of a Good Characteristics of a Good Analysis PlanAnalysis Plan

ParsimoniousParsimonious– Only includes the variables needed to Only includes the variables needed to

answer the research questionanswer the research question– Avoids (or adjusts for) multiple comparisonsAvoids (or adjusts for) multiple comparisons

Clearly DescribedClearly Described– Comparison(s) to evaluate each study Comparison(s) to evaluate each study

objectiveobjective– Methods that will be usedMethods that will be used– How the researcher will know if the objective How the researcher will know if the objective

has been reached and their hypothesis has been reached and their hypothesis proved/disprovedproved/disproved

Analytic Resources for Analytic Resources for BUMC ResearchersBUMC Researchers

Data Coordinating Center, BUSPHData Coordinating Center, BUSPH http://www.bu.edu/dbin/sph/research_centers/dcc.phphttp://www.bu.edu/dbin/sph/research_centers/dcc.php

Department of Biostatistics, BUSPHDepartment of Biostatistics, BUSPH http://www.bu.edu/dbin/sph/departments/biostatisticshttp://www.bu.edu/dbin/sph/departments/biostatistics

General Clinical General Clinical Research Center Research Center

InvestigatorsInvestigators If you are currently a GCRC researcher If you are currently a GCRC researcher

––OR– OR–

If you are preparing an application to use If you are preparing an application to use the GCRCthe GCRC

The GCRC can provide assistance with The GCRC can provide assistance with development of the study design and development of the study design and

analysis plananalysis planhttp://dccwww.bumc.bu.edu/gcrcweb/abouttheGChttp://dccwww.bumc.bu.edu/gcrcweb/abouttheGCRC.htmRC.htm

Questions and Discussion