Quantitative Research

Dr Nokuthaba SibandaLecturer in Statistics

School of Mathematics, Statistics & Operations Research

Victoria UniversityWellington

18 August 2009 1

What is quantitative research?

• Quantitative research focuses on gathering numerical data and generalising it across groups of people.

• Sampling bias is important in determining how generalisable the results are – AVOID

• Sampling variability reflects the amount of confidence you can have about how well your sample has captured the characteristics of the population – influenced by sample size

2

Characteristics of a quantitative research project• Researcher has a clearly defined research question

to which objective answers are sought

• All aspects are carefully and precisely designed before data collection

• Data are in the form of numbers and statistics

• Project can be used to generalise concepts more widely, predict future results or investigate causal relationships

3

Considerations in quantitative research

• Study Design

• Data Collection

• Data analysis

• Reporting your results

4

Study Design

18 August 2009 5

Study design• An appropriate study design is essential in

ensuring validity of the results that you eventually report

“To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.”

R. A. Fisher; Presidential Address to the First Indian Statistical Congress, 1938.

6

Study design: Research question and hypothesis

Research question• Question that you are trying to answer in your

research project

• Start with a broad area of research interest, then narrow it down – knowledge of literature is quite important

• Question must be specific and be such that it can reasonably be studied in a research project, bearing in mind time and resource constraints

7

Study design: Research question and hypothesis

Research hypothesis

• A statement that can be proved or disproved

• A research question can be made into a hypothesis by changing it into a statement.

8

Example: Research question and hypothesisStudy area

– What are the effects of sleep deprivation on driver reflexes?

Formulating the research question– Define sleep deprivation – 24 hours without sleep? Sleep for 2

hours night before? Less than 6 hours sleep?– Define driver reflexes – Reaction time to simulated hazard?– Define reaction time – Time to stop from same speed? Time to

foot on break pedal?– Does sleeping 2 hours compared to 8 hours increase the time

it takes to stop when a hazard appears 30 metres away?

• Hypothesis– Sleeping for 2 hours compared to 8 hours increases

the time it takes to stop when a hazard appears 30 metres away.

9

Study design: Type of study

• Experimental– Researcher decides which treatment participants get– Participants are assigned to treatment groups in a

random manner– Random assignment of participants is essential for

the creation of comparative groups– May include a control group

10

Examples of Experimental Studies

Pre and Post test study

11

Randomly assigned experimental group

PRE-TEST

Randomly assigned control group

PRE-TESTPOST-TEST

POST-TESTTREATMENT

Examples of Experimental Studies

Level of Treatment A

Level of Treatment B

Treatment B No treatment

Treatment A Randomly selected Group 1

Randomly selected Group 2

No treatment Randomly selected Group 3

Randomly selected Group 4

(control)

12

Factorial

Study design: Type of study• Observational

– Studies in which relationships between variables are investigated without the researcher manipulating the subjects

– Studies are prone to bias as a result of differences between groups with respect to confounding factors

– Often used where random assignment of participants to study groups is not possible or unethical

– Such studies can only show associations – causality can not be proved

– Require larger sample sizes than experimental studies13

Study design: Number of subjects• Sample size should be determined at the design stage

• May not be under researcher’s control, but you need to have an idea of how many participants you can get and whether the sample would be large enough to detect differences of interest

• Use sample size formulae to determine the ideal number of participants and try to get that number as a minimum

• If ideal sample size is not feasible, consider using more efficient study designs eg repeated measures or cross-over designs

14

Study design: Data collection

• What is your outcome or response variable?– Should be determined from your research question

– You may have more than one

– Is it nominal, ordinal or continuous?

– Example: Final course grade, Number of hours slept

15

Study design: Data collection

• Identify the confounding variables– In observational studies it is important to adjust for

group differences with respect to confounding variables

– Almost always include demographic information: age and gender

– Knowledge of similar published studies and current views is important, therefore read widely

16

Study design: Data collection• Where and how are you going to store your

data?– Software

• Excel is the most commonly used data collection and storage software

• Most statistical software programs are difficult to use for data entry

– Database design• Coding of variables advisable for ease of manipulation• Decide how non-response will be dealt with• Do not mix numerical and character data in one column• Rows represent study participants and columns

represent variables measured

17

Study design: Analysis plan

• Summarise all response and explanatory variables– Numerical and graphical summaries

• Look for any outliers (outside of expected ranges) or missing data (are they really missing?)

• Check distributional assumptions

• Apply appropriate statistical test

• Interpret your results, linking back to your research question

18

Data Collection

18 August 2009 19

Data Collection

• Implementation of study design– Consistency among data collectors– Accuracy in data recording, consider double data entry

“The government is very keen on amassing statistics. They collect them, add them, refer to the nth power, take the cube root, and prepare wonderful diagrams. But you must never forget that every one of those figures comes in the first instance from the [village watchman], who just puts down what he damn pleases.” Stamp, J. S. (1929)

Your study is only as good as the data you have!

20

Data Analysis

18 August 2009 21

Selecting an appropriate test• To select the appropriate statistical test, you

need to answer the following questions– What type of outcome variable do you have?– What is the aim of your analyses?

• Are you comparing groups? If so, how many? • Are measurements matched across groups (eg measurements taken

more than once on the same subject or on pairs of twins)• Do you wish to identify a subset of variables (continuous or

categorical) that influence your outcome variable?

• Make use of any prior knowledge you have, particularly when exploring the distribution of the data. Are the results as you expect them to be?

22

What type of outcome variable do you have?

Continuous – normal/symmetric or

transformed non-normal

Ordinal (eg rank/score) or non-normal continuous

Nominal – two or more possible

outcomes

Describe one group

Compare 1 group to a hypothetical value

Compare 2 unpaired groups

Predict value from one or more measured or

categorical variables

Compare 2 paired groups

Compare ≥3 unmatched groups

Compare ≥3 matched groups Cochrane Q testFriedman testRepeated-measures

ANOVA

McNemar’s testWilcoxon matched pairs testPaired samples t test

Fisher’s or chi-square testMann-Whitney U testIndependent samples t

test

Logistic regressionNon-parametric regression

Linear or non-linear regression

Chi-square testKruskal-Wallis testOne-way ANOVA

Chi-square or binomial test

Wilcoxon signed rank testOne-sample t test

ProportionMedian, IQ rangeMean, SD

What is your aim?

Based on: www.graphpad.com/www/book/Choose.htm

Reporting Your Results

18 August 2009 24

Report descriptive statistics (eg mean, median, standard deviation, range) of the sample. Tell the reader

– Who your subjects are (eg average age, percentages of males and females)

– Whether your comparative groups are similar with respect to confounding values (eg are average ages similar between groups?)

25

Reporting your results

Reporting your results

Report confidence intervals, test statistic, degrees of freedom and p-value•Recall the null hypothesis that you are testing (null meaning no differences between groups).

•Confidence intervals give the range of plausible values of means or mean differences

•The test statistic measures the evidence against your hypothesis (large values are evidence against the null hypothesis and correspond to small p-values).

•The degrees of freedom represents the number of independent observations.

26

Reporting your resultsInterpreting p-values:• The p-value is the probability that your test statistic

is as large or larger than that observed, if the null hypothesis is true.

• If the p-value is very small (less than 0.05), then the test statistic is unusually large compared to what you would expect if the null hypothesis is true.

• Small p-values (less than 0.05) suggest that the null hypothesis is not true (reject null hypothesis) based on the evidence in your data.

27

Reporting your resultsHow to interpret a negative result

• If an effect is not significant (p-value>0.05), you have not proved the null hypothesis is true.– To observe that there is no difference between groups does not

prove that the difference does not exist – “The absence of proof is not proof of absence”

• If the sample size is small, the ability of any statistical test to detect an effect is poor.

• You can quantify the ability of your test to detect an effect with a power analysis.

28

Statistics Resources @ VUW

18 August 2009 29

Software for Statistical Analysis• Excel

– good for data entry, data manipulation and quick summaries

– Limited analytical tools

• SPSS– Available on most computers in the University– Point and click facility for analysis– Script file for recording and storing commands

(similar to macro recording in Excel)– But beware – COPIOUS amounts of output! You need

to know how to interpret the output.

30

Other software

• R – Freeware. Normally has a lot more to offer than other software packages, but harder to learn

• SAS – mainly used in a lot of governmental and private organisations

• SPlus – favoured more by academics

31

Courses and Reading Material• Courses

– STAT193: Statistics for the Natural and Social Sciences– QUAN102: Statistics for Business

• Library– Clark MJ, Randal JA. A First Course in Applied Statistics: with

applications in biology, business and social sciences, Pearson, 2004, cost ≈ $60

– Other introductory statistics texts • Salkind NJ 2004 – Statistics for people who (think they) hate

Statistics• Magnusson WE 2001– Statistics without math

– SPSS manuals • Pallant J 2007 – SPSS survival manual : A step by step guide to data

analysis using SPSS for windows• Field A 2000 – Discovering statistics using SPSS for Windows:

advanced techniques for the beginner32

The University Statistics Consulting Service• Based at the School of Mathematics, Statistics &

Operations Research

• Consultant not available until early next year

• Discuss with your supervisor first!

• Sourcing funds for payment– Get advice from supervisor or school manager– Various ways across different schools– School budget– Grants

33