Introduction to Reading Educational Research

What is Educational Research?

A systematic investigation Involves the analysis of information (data)

for the purpose of– answering a question or– Contributing to our knowledge about an

educational theory or practice Educational research relies on methods and

principles that will produce credible and verifiable results

Information is gathered from…

Individuals Groups Documents Existing data bases and Other sources

Steps in research(may vary depending on the nature of the study)

1. Frame the initial question or problem

2. Determine what previous research says about the question or problem

3. Frame a research question, problem or hypothesis

4. Design a plan for collecting data to address the question or problem

5. Analyze the results of gathered data

6. Generate conclusions

How do you determine whether an article or report is research?

1. Does the title suggest that data have been gathered from existing records?

The terms study & investigation denote research Phrases such as an investigation of, the

relationship between & the effect of do also

2. Is there a problem that is investigated? There must be some question or problem that is

the focus of the study An empirical investigation is designed to

answer a specific question or problem

How do you determine whether an article or report is research?3. Are new data collected?

Most research involves collection of new information

Usually there is a section using such terms as instruments, measures, or data collection

4. Is there a methodology or methods section?

5. Is there a findings or results section? This section often includes tables or graphs to

summarize the information

Types of Educational Research

There are different methods by which research is carried out

Each method has distinct purposes, advantages, and disadvantages

Steps in Becoming an Intelligent Consumer of Educational Research1. To be able to recognize that an article or

report is, in fact, research

2. To be able to recognize which method was used

3. To be able to assess the quality of the research according to criteria appropriate for that method

Analyzing a study of parental attitudes toward AIDS EducationYou would be most interested in questions

like these: How were the parents chosen for the study? How many participated? What method was used to collect data from

the parents? How appropriate were the questions asked? What inferences were drawn from the

results? How valid were these inferences?

Analyzing a study reporting results of an experimental curriculum What precisely was different about the

“experimental” curriculum? How was it implemented? What effects was it supposed to have How did the researchers choose to measure

these effects? Did the researcher use a control group? How was it selected? Did the researchers find the effects they

were looking for? Were they significant?

Quantitative? Qualitative? Analytical?

Quantitative research involves the use of numerical indices to summarize, describe and explore relationships among traits—reliance on control, statistics, measurements, and experiments

Qualitative research—the emphasis is on conducting studies in natural settings using mostly verbal descriptions, resulting in stories and case studies, not statistical reports

Analytical research involves the identification and interpretation of data contained in documents, pictures, and artifacts.

Quantitative Research Experimental Study:

– The researcher has control over one or more factors (variables)

– Purpose: to determine whether one factor causes a change in another

– Types: true experimental; quasi-experimental; single-subject

Nonexperimental Study– No control over what may influence subjects’

responses– Researcher interested in what occurs naturally– Describes phemenona/uncovers relationships

Experimental Research Categories True Experimental

– Use of comparison groups– Random assignments to groups

Quasi-experimental– “Manipulates treatments” but doesn’t use randomly

assigned treatment groups– Used when randomization of groups is impossible– What they gain in feasibility, they give up in exploratory

power Single-subject

– Only one person (or a few) in the study– Used when it is difficult or impossible to identify large

groups of subjects– Most common method of data collection is careful

observation of targeted behavior

Nonexperimental Research

Descriptive – Describe a phenomenon with statistics such as

frequencies, percentages, averages, etc.– No comparisons/no relationships– i.e., 20% of teachers indicated that high-stakes tests have a significant effect

on what they teach

Comparative– Investigates relationship between 2 or more

variables in 2 or more groups– Unlike ex post facto studies, comparative research

make no claim of causality

Nonexperimental Research Correlational

– Investigate relationships among variables by calculating correlation coefficients rather than looking at differences

– Indication of the direction and strength of the relationship sought: positive or negative?

– i.e., How well do SAT scores predict college success? Ex post facto (“after the fact”)

– Examines a phenomenon that has already occurred and attempts to infer cause-and-effect relationships

– i.e., Do smokers have higher rates of lung cancer than nonsmokers?

– Teasing out whether in fact any differences are due to the “treatment” variable is very difficult

Qualitative Research Designs

Ethnography– Involves direct engagement with the

participants and environments – In-depth description and interpretation of

behavior within a culture or social group– Conclusions are based on a synthesis of the

data

Case study– In-depth analysis of one or more “bounded

systems”

Qualitative Research Designs Phenomenology

– Examines in depth the experiences and perceptions of selected participants

– Purpose: to understand the essence, the meaning of participants’ perspectives

Grounded Theory– To discover or generate a theory that pertains to a

specific entity or phenomenon Critical Studies

– Designed to provide alternative perspectives on contemporary societal problems

– Focus on exposing social manipulation and oppression to advocate changes in social systems

Analytical Research Historical analysis

– Provides a description and interpretation of past events or persons. Uses both qualitative and quantitative data

Legal Analysis– Focuses on laws and court decisions to

understand how legal principles apply to educational practice

Concept Analysis– Conducted in order to understand the meaning

and usage of educational concepts (i.e., ability grouping)

– Very seldom used in educational research

Mixed Models

There is a growing trend in educational research to use mixed models in a study

Combination of two of the three major types of research (i.e., a quantitative study combined with qualitative methods)

Research Articles: General Sections1. Introduction

2. Review of the Literature

3. Research Problem, Question, or Hypothesis

4. Methodology: subjects, participants, or sources of information

5. Methodology: Instruments

6. Discussion & Conclusions

Section 1: Introduction

Purpose: to set the context of the study and to indicate the research problem

One to several paragraphs long Provides background to the study; indicates

why it is significant to conduct research Use of quotes or references to other studies General statement of the problem

Section 2: Review of Literature

Quantitative– One to several paragraphs in which previous

studies of the same or similar problem are summarized & analyzed

– Purpose: to show how the current study is related to other studies

– A good review is selective– Goes beyond a simple reporting of findings—a

critical analysis of previous studies

Section 2: Review of Literature

Quantitative—Questions to Ask:1. Does the review contain relevant, up-to-date

studies?2. Is there an emphasis on primary sources? 3. Is there a critical review or a summary of

findings?4. Is the review well organized?5. Does the review clearly relate previous

studies to this research problem?6. Does the review help establish the importance

of research?

Section 2: Review of Literature Qualitative

– Use a preliminary literature review to present conceptual frameworks, broad areas of study, and scholarly concepts

– This provides a foundation for phrasing foreshadowed questions

– A continuing search of the lit is integrated with the methodology and tentative findings and interpretations

– By the end of the study, there is a complete review that contains a critique and analysis of works

– Quantitative Studies—a clear lit review preceded the methodology

– Qualitative Studies—The lit review is interspersed throughout the document

Step 3: Research Problem, Question or Hypothesis State a specific research problem statement, question,

and/or research hypothesis Can come at the end of the introduction; more

commonly follows the lit review It denotes the type of research method being used

(i.e.,experimental or non experimental), the independent and dependent variables, and the subjects

Qualitative: more general statement that provides a broad framework for beginning the study and is reformulated as data are collected. Emphasis on what, where, and why, rather than on relationship among variables

Step 3: Research Problem, Question or Hypothesis Questions to ask:

1. Is the problem or hypothesis clear and concise?

2. Is there a hypothesis, is it consistent with the lit review?

3. If quantitative, does the problem or hypothesis communicate the variables, type of research, and population?

4. If qualitative, is the initial problem reformulated?

Step 4: Methodology:Subjects, Participants, or Sources of Information

Participants will be described in this section Some qualitative and analytical studies use

documents, artifacts, pictures, and other sources of data that are not directly from people—these will be described here

Step 4: Methodology:Subjects, Participants, or Sources of Information

Questions to ask:1. Is the population described adequately?

2. Is the sample of subjects and/or other sources of information clearly described?

3. Is the method of selecting the sample clear?

4. Could the method of selection affect results?

5. Are subjects likely to be motivated to give biased responses?

Step 5: Methodology: Instruments Instruments or measures are devices that are used

to gather information from subjects Instruments can include tests, oral or written

surveys, ratings, observation, and various archival and “unobtrusive” measures

Quantitative: credibility depends on validity and reliability

Qualitative: technical feature of instrument validity and reliability aren’t used, but the more general ideas of appropriateness of the inferences (validity) and error in collecting information (reliability) are still important

Step 5: Methodology: Instruments Questions to ask:

1. Is evidence for validity and reliability clearly presented and adequate?

2. Is there a clear description of the instrument and how it was used

3. Is there a clear description of the procedures for collecting the information?

4. Is it likely that the subjects would fake their responses?

5. Are interviewers and observers trained?

6. If appropriate, what are the norms used to report percentile rank and other results?

Step 6: Methodology: Procedures This subsection will indicate how subjects or

participants were selected, how instruments were developed and administered, and how experimental treatments, if any, were conducted

A qualitative study will describe procedures the researcher used to gain entry into the field, the nature and length of observations and interviews, and how participants were approached

In this section, look for characteristics of the study design that might lead to erroneous interpretations of the data

Step 6: Methodology: Procedures

Questions to ask:1. Are there any clear weaknesses in the design

of the study?

2. Are the procedures for collecting information described fully?

3. Is it likely that the researcher is biased?

Step 7: Results

Presents a summary of the data analysis Quantitative: researchers often use

inferential statistics to make probability statements about what is likely to be true

Qualitative: coding, triangulation

Step 7: Results

Questions to ask:1. Are the findings presented clearly?2. Is there appropriate use of tables, charts, and

figures?3. Is the # of subjects taken into consideration?4. Is there sufficient descriptive information to

interpret the results?5. Are the results presented in relation to the

research question?6. If qualitative, are the results accompanied by

illustrative quotes and specific instances?

Step 8: Discussion & Conclusions

Discussion of meaning of results; this interpretation is helpful to understand why certain results were obtained

Conclusions—summary statements of the findings and how the reseacher interprets them

Quantitative: generalizability of the findings to the larger population

Qualitative: transferability and comparability of the findings

Step 8: Discussion & Conclusions Questions to ask:

1. Is the discussion based on the research problem and results?

2. Is there an adequate interpretation of the findings?3. Is the interpretation separate from the results?4. Are the results discussed in relation to previous studies?5. Are limitations due to methodology included in the

discussion?6. Are the conclusions clearly stated and based on the results

and discussion?7. Are the conclusions reasonable? Do they go beyond the

interpretation of the findings?8. What is the generalizability of the findings? What factors

would affect generalizability?