Download - Approaches to Research
Approaches to Research
Week 2(Dornyei, 2007, p.24-47)
Dr. Hale Işık-Güler 2009-2010
Types of research in SocialTypes of research in SocialSSciencesciences::
• QUALITATIVE RESEARCH• QUANTITATIVE RESEARCH• MIXED METHODS RESEARCH
Research Methods
Research methods are generally categorised as being either quantitative or qualitative.
What matters is that the methods
used fit the intended purposes of the research!
Qualitative and Quantitative Paradigms
The qualitative paradigm concentrates on investigating subjective data, in particular, the perceptions of the people involved. The intention is to illuminate these perceptions and, thus, gain greater insight and knowledge.
The quantitative paradigm concentrates on what can be measured. It involves collecting and analysing objective (often numerical) data that can be organised into statistics.
Qualitative and Quantitative Research
Qualitative Research
Quantitative Research
Also known as
interpretative / responsive
positivist /hypothetico-deductive
Type of reasoning
(usually) inductive (usually) deductive
Link withconcepts
identifies concepts identified concepts and investigates relationships
Action sometimes only describes a situation BUT in action-research openly intervenes
tests relationships between concepts
Outcome illuminates the situation accepts or rejects proposed theory
Approach to validity
truth seen as context bound (socially constructed)
truth seen as objectiveand universal
- Common dichotomies in methodological literature
Main distinctions seen between quantitative and qualitative ‘paradigms’
qualitative
quantitativesubjective objectiveinductive deductive
participant observation survey techniquesanthropology sociology
naturalism anti-naturalism
art sciencehermeneutics positivism
aristotelian galileanteleological causal
finalistic mechanisticunderstanding explanation
phenomenological logical positivism
* bad *gooddescriptive predictiveempiricism rationalism
atheoretical theoretical
The roots of current debates
Early debates: from 17th century to the first half of 20th centuries
Prosperity of positivism and the domination of quantitative methods in social sciences:
from 1930s to 1950s
The debunking of logical positivism after World War II and the pervasiveness of the postpositivist position:
the end of 1950s and beginning of 1960s
The ascendance of interpretivism and constructivism, followed by the ‘paradigm wars’:
from late 1960s onwards
Research MethodologiesA continuum rather than “either/or”
Qualitative Goal: To Understand,
Predict Descriptive accounts Similarities and
Contrasts Applied and
Theoretical Research Questions Field study
Natural conditions
Quantitative Goal: To Predict and
Control Measure and Evaluate Generalize to
population, reproduction
Basic and Theoretical Hypothesis testing Lab study
Controlled, contrived
Approaches to Research
Qualitative and quantitative DATA QuantitativeQuantitative datadata:
Most commonly expressed in NUMBERS
E.g aptitude test score TOEFL score Number of times a
student volunteered in class
QualitativeQualitative data:
Usually involves recorded spoken data that is transcribed to textual form as well as written data
E.g. interview data Diary entries field notes other documents
Language datadata: (may be subsumed under qualitative data)
Language samples in various length elicited from repondents primarily for the purpose of LANGUAGE ANALYSIS
E.g. -A recorded language task -Solicited student essay (for discourse
analysis) - Corpus data
The qualitative-quantitative distinction
Ideological differences Reformists; distaste for mainstream
quantitative paradigm Contrasting categorizing/coding
practices Social world data needs to be reduced
(categories and codes) QUAN: numerical precise coding tables
(values) before study begins: M:1, F: 2 QUAL: short verbal textual labels
Different approaches to individual diversity QUAN: Diversity is a problem that needs to
be fixed (large sample, commonalities-idiosyncrasies (Meaning in the general)
QUAL: -loose individual stories; real meaning lies with individual cases that make up the world (Meaning in the particular)
Statistics versus researcher sensitivity QUAN: Meaning in numbers; greater the
sample the more ‘normal’ the distribution (bell-shaped curve). This idea originally led to the development of statistics.
QUAL-QUAN contrast and the “paradigm war” Categorizing the world
Predetermined numerical category system Emergent, flexile coding
Perceiving individual diversity Using large samples to iron out any individual
idiosyncrasies Focusing on the unique meaning carried by
individual organisms Analysing data
Relying on the formalized system of statistics Relying on the researcher’s individual
sensitivity
Quantitative research is:
"a formal, objective, systematic process in which numerical data are utilised to obtain information about the world"
Quantitative Methods
quantitative research typically seeks to generalize to a large population. It is “hypothetically” more objective and less interpretive.
Some of the most frequently utilized types of quantitative methods in applied lingusitics research include: Surveys Experimental Design Tools
Quantitative Quantitative ResearchResearch
Scientific method:
• Identifying a problem,
• Formulating a hypothesis,
• Testing the hypothesis by collecting and analyzing empirical data usingstandardized procedures.
a hypothesis a hypothesis is a tentative explanation for
certain behaviors, phenomena, or events that have occurred or will occur.
1. The hypothesis states the researcher's expectations concerning the relationship between the variables in the research problem.
2. The hypothesis is a refinement of the research problem. It is the most specific statement of the problem.
3. The hypothesis states what the researcher thinks the outcome of the study will be.
4. The researcher collects data that either supports the hypothesis or does not support it.
5. The hypothesis is formulated following the review of related literature and prior to the execution of the study. The related literature leads the researcher to expect a certain relationship.
6. "A good hypothesis states as clearly and concisely as possible the expected relationship (or difference) between two variables and defines those variables in operational, measurable terms."
7. "A well-stated and defined hypothesis must be (and will be if well-formulated and stated) testable. It should be possible to support or not support the hypothesis by collecting and analyzing data."
hypotheses for an experimental or causal-comparative study:
If X is the independent variable, Y is the dependent variable, and S is the Subject, we can state our research hypothesis as Ss who get X do better on Y than subjects who do not get X (or get some other X).
Research Hypothesis: Girls will achieve higher reading comprehension test scores than boys at the end of the first grade.
The null hypothesis: There will be no difference between the two means on the reading comprehension test scores
hypotheses for correlational research:
If A and B are variables (note that we do not refer to them as independent and dependent variables in correlational research), and C is the subject we can state our research problem as the relationship between A and B for C. (What is the relationship between A and B for C)
Research Hypothesis: The length of time needed to complete an examination will be negatively correlated with the score on the examination for uni. students.
Another Example
The following is a hypothesis for a study.1. "There will be a statistically significant
difference in graduation rates of at-risk high-school seniors who participate in an intensive study program as opposed to at-risk high-school seniors who do not participate in the intensive study program." (LaFountain & Bartos, 2002, p. 57)
IV: Participation in intensive study program. DV: Graduation rates.
Main Characteristics of Quantitative Research
(p.33-34) Using numbers A priori categorization Variables rather than cases Statistics and language of statistics Standardized procedures to assess
objective reality Quest for generalizability and
universal laws
Selecting Statistics for Quantitative Research
Dependent Variable
Dichotomous Continuous
Independent
Variable
Dichotomous
Chi-squareLogistic RegressionPhiCramer's V
t-testANOVARegression Correlation
Continuous Logistic RegressionCorrelation
RegressionCorrelation
William Gossett - nicknamed ‘Student’ was a chemist at the Guinness brewery in Dublin and developed the student t-test in 1908 to ensure that each batch of Guinness was as similar as possible to every other batch! The t-test is used to compare two groups. 3 different types can be performed:
Paired t-test: used when each data point in one group corresponds to a matching data point in the other group.
Unpaired t-test: used whether or not the groups contain matching data points:
o Two-sample assuming equal variances o Two-sample assuming unequal
variances
So, how is a variable defined?
A variable is an object, event, idea, feeling, time period, or any other type of category you are trying to measure. There are two types of variables-independent and dependent.
It is critical to categorize variables as being dependent or independent
Dictates which type of statistical test should be performed– Is one variable likely to be influenced by the other?– Is variation in one variable likely to be
uninfluenced by the other?
In an experiment...
the independent variable is the variable that is varied or manipulated by the researcher, and the dependent variable is the response that is measured.
An independent variable is the presumed cause, whereas the dependent variable is the presumed effect.
What's an independent variable in research? An independent variable is exactly what it sounds
like. It is a variable that stands alone and isn't changed by the other variables you are trying to measure.
For example, someone's age might be an independent variable. Other factors (such as what they eat, how much they go to school, how much television they watch) aren't going to change a person's age. In fact, when you are looking for some kind of relationship between variables you are trying to see if the independent variable causes some kind of change in the other variables, or dependent variables.
What's a dependent variable in research? Just like an independent variable, a dependent
variable is exactly what it sounds like. It is something that depends on other factors.
For example, a test score could be a dependent variable because it could change depending on several factors such as how much you studied, how much sleep you got the night before you took the test, or even how hungry you were when you took it. Usually when you are looking for a relationship between two things you are trying to find out what makes the dependent variable change the way it does.
Many people have trouble remembering which is the independent variable and which is the dependent variable.
An easy way to remember is to insert the names of the two variables you are using in this sentence in they way that makes the most sense. Then you can figure out which is the independent variable and which is the dependent variable:
(Independent variable) causes a change in (Dependent Variable) and it isn't possible that (Dependent Variable) could cause a change in (Independent Variable).
(Independent variable) causes a change in (Dependent Variable) and it isn't possible that (Dependent Variable) could cause a change in (Independent Variable).
For example:
(Time Spent Studying) causes a change in (Test Score) and it isn't possible that (Test Score) could cause a change in (Time Spent Studying).
We see that "Time Spent Studying" must be the independent variable and "Test Score" must be the dependent variable because the sentence doesn't make sense the other way around.
Software Spreadsheets
EXCEL
Statistical Software SPSS (http://www.spss.com/) MINITAB (http://www.minitab.com/) SAS (http://www.sas.com/)
Strengths of Quantitaive Research
Quantitative research “is systematic, rigorous, focused, and tightly controlled, involving precise measurement and producing reliable and replicable data that is generalisable to other contexts” (Dörnei, 2007, p. 34).
Although the preparation period takes quite a long time, “the research process is relatively quick and offers good value for money, particularly because the data analysis can be done using statistical computer software” (Dörnei, 2007, p. 34).
Weaknesses of Quantitaive Research
“Quantitative methods average out responses across the whole observed group of participants and by working with concepts of averages it is impossible to do justice to the subjective variety of an individual life”
(Dörnei, 2007, p. 35)
Similar scores can result from quite different underlying processes
Quantitative methods are not very sensitive in finding out the reasons for particular observations the dynamics underlying the examined
situation or phenomenon
In other words,
For some people, “the general exploratory capacity of quantitative research is rather limited” (Dörnei, 2007, p. 35) .
Thinking it through . . .
Suppose you wondered how students use instructor feedback from the first draft of their papers to write the second draft. How might you design a quantitative investigation to address this question?
…
QuaQuallitative Researchitative Research
Qualitative research came about as a reaction to the
“hard-core” quantitative research.
Qualitative research ‘qualitative research is difficult to define
clearly. It has no theory or paradigm that is distinctly its own. ...Nor does qualitative research have a distinct set of methods or practices that are entirely its own’, (pp.6-7)
‘Qualitative research is many things to many people’ (p. 10) (Denzin and Lincoln as cited by Dörnei, 2007, p. 35)
Some believe there is lack of uniformity A thriving discipline
Qualitative Methods Qualitative methods are used to “understand”
more deeply, reach a ‘THICK’ description. Typically results do not seek to generalize widely.
Some of the most frequently used types of qualitative methods in applied linguistics research are:
Interviews/focus groups text or discourse analysis (think-aloud) protocol analysis observational work, field notes Diary entries Supporting documents Images, videos, etc.
Main characteristics of qualitative data
Emergent research design; grounded theory designs
flexible, no aspect is tightly prefigured, enter the process with an open mind
The nature of qualitative data (i.e. texts) The characteristics of the research setting
Natural setting, no manipulations; prolonged contact ‘Insider’ meaning
Reaching subjective opinions, feelings of individuals; goal is to explore participants views of the situation being studied
Small(er) sample size Interpretive analysis
The research outcome is ultimately the product of the researchers subjective interpretation of data
The researcher is essentially the main ‘measurement device’ in the study!
Strengths of qualitative research (p.39-
40) Exploratory nature (exploring new
uncharted areas) Making sense of complexity Answering “why” questions Broadening our understanding
(‘not’ correct, but possible interpretations) Good starting point for
longitudinal examination of dynamic phenomena
Flexibility when things go wrong Design allows for change/alterations
Rich material for research report tables vs. detailed descriptions
Weaknesses of qualitative research
Sample size and generalizability Researcher role
(influence, bias, observer’s paradox)
Lack of methodological rigour Too complex or too narrow
theories from the indivdual cases studied
Time consuming and labour intensive (reason why sample sizes are small!)
Thinking it through . . .
Suppose you wondered how students use instructor feedback to write the second draft of a paper. How might you design a qualitative investigation to address this question?
...
Mixed methods research
is a combination of qualitative and quantitative methods within a single research project
How can QUAL and QUAN work together (inform each other, give feedback to one another)?
Main characteristics of mixed methods research
the combination of the main characteristics of quantitative and qualitative methods research.
The real breakthrough for the method occurred in 1970 with the introduction of the concept of triangulation
Mixed Mixed MMethods ethods ResearchResearch
Triangulation (in naval navigation):
The process of establishing theunknown position of a spatialpoint, A, by using the measurements from two knownpoints, B and C.
Mixed Mixed MMethods ethods ResearchResearch
Triangulation (in social sciences):
The process of investigating acertain phenomenon by usingdifferent (multiple) methods.
Strengths of mixed methods research
Increasing the strengths while eliminating the weaknesses
Multi-level analysis of complex issues
Improved validity Reaching multiple
audiences (findings more acceptable for a larger audience)
Weaknesses of mixed methods research
The popular belief “The sum may be greater than its parts” may not be always true!
Researchers may lack methodological skills to handle both qualitative and quantitative data.
Examples of mixed methods Examples of mixed methods rresearchesearch
• Qual, then Quan.• Quan, then Qual.• Simultaneous Quan and Qual.
Quantitative Qualitative Objective Subjective
Tests theory Develops theory focus is concise and narrow focus is complex and broad Reduction, control, precision Discovery, description, understanding, shared
Measurable Interpretive Report statistical analysis. Report rich narrative, individual interpretation.
Basic element of analysis is numbers Basic element of analysis is words/ideas. Researcher is separate Researcher is part of process
Context free Context dependent Hypotheses Research questions
Reasoning is logistic & deductive Reasoning is dialectic & inductive Establishes relationships, causation Describes meaning, discovery
Uses instruments Uses communication and observation Designs: descriptive, correlational, quasi-
experimental, experimental Designs: phenomenological, grounded theory, ethnographic, historical, philosophical, case
study. Sample size: determined by statistical power required
Sample size is not a concern; seeks "information rich" sample
The Qualitative / Quantitative continuum
Methodology Comparison
Quantitative
Explanation, prediction Test theories Known variables Large sample Standardized
instruments Deductive
Qualitative
Explanation, description Build theories Unknown variables Small sample Observations, interviews Inductive
MIXED METHODS
Which is the BEST method?
-The best method is the one that BESTBEST suits your aim/ research questions.
-What matters is that the methods used fit the intended purposes of the research!
-NO such thing!!!!