class 6 ( chapter 8+9) - faculty.ecnu.edu.cn

ECNU Educational Research Methodology

Class 6 ( Chapter 8+9)Qualitative research designHistorical research

2016/5/19 1 Fang Chen 陈芳华东师大英语系


CHAPTER 8Qualitative Research Design

2016/5/19 Fang Chen 陈芳华东师大英语系2


1. definition2. common assumptions3. common components4. Perspectives for designing the qualitative

study5. reliability and Validity

Qualitative Research: Processes



McMillan and Schumacher (1993) “Primarily an inductive process of organizing data

into categories and identifying patterns (relationships) among categories.”

1. Definition


ECNU Educational Research Methodology2. Common Assumptions /

Epistemology of Qualitative Research

By epistemology of qualitative research we mean the underlying assumptions and basic ideas of how research is done.

5 basic descriptors as the underlying epistemology of qualitative research.

① Holistic② Natural Settings③ Reality exists as the subjects see it.④ Favor Post Hoc conclusions⑤ Phenomena are loosely modeled2016/5/19 6 Fang Chen 陈芳华东师大英语系


⑤ Phenomena are loosely modeled

Qualitative research does not reduce phenomena to a basic set of components and therefore predictions (i.e., models) are not mechanically defined.

Prediction is flexible (more a set of concepts)



In summary



3. Common Components of Research

These are general components that are mostly common to qualitative research but they are quite flexible and depend heavily on natural settings.a. Working Design (or Emergent Design)b. Working Hypothesisc. Data Collectiond. Analysis and Interpretation



a. Working Design

A preliminary design for getting a study under way…

Example: An ethnographic study of the school environment

of regular and learning disabled students to determine factors related to potential dropout.1. Which school to use? --- purposeful sampling2. Whom to interview or observe? --- teacher, students,

counselors3. Length of time?



b. Working Hypothesis

In general, qualitative research is to use inductive inquiry. Data collection is usually done without any

preconceived theories. However, there are foreshadowing statements and sometimes even hypotheses that will direct the research.

Example: What is the role of the faculty in attempting to

reduce the dropout? (research question) Interaction of school administration and students.

(foreshadowed problem)



c. Data Collection

Data are collected in natural settings; access to the setting is important:

Participant-observer Vs Observer

Interactive Vs Non-interactive



Interactive or non-interactive?

Typical methods of data collection include:

Interview Observation Collection and review of related documents



How to collect data for our example on counseling and drop-out rate? Interview students and faculty. Observe the interaction taking place between

students and faculty. Review school records relative to factors such as

grading patterns. If available, interview recent dropouts.

Data Collection Example



skip

Two activities mentioned in the textbook: specimen records: a special case of observation

• A narrative description of one person in a natural noncontrived situation as seen by skilled observers over a substantial time period.

• Case study

oral history http://srmo.sagepub.com/view/the-sage-encyclopedia-of-social-science-research-methods/n670.xml

• Interviews taken through the use of a tape recorder???• Emphasize open questions; more efficient???


http://srmo.sagepub.com/view/the-sage-encyclopedia-of-social-science-research-methods/n670.xml�

http://srmo.sagepub.com/view/the-sage-encyclopedia-of-social-science-research-methods/n670.xml�


Skip: Specimen records: some details



Typically, the data can be quite large in qualitative research. A data record of 1000 pages or more is not unusual.

So it is also important to record all conditions and thoughts that are occurring during the collection as well.



d. Analysis and Interpretation

Analysis is done during collection of the data so that working hypotheses can be checked and possibly modified.

Because of the large amounts of data it must be organized. This process is called Coding. Coding is a process of organizing data and

obtaining data reduction.



Analysis and Interpretation

In coding, the data is organized using categories. The various types of data are evaluated and

placed in the appropriate categories These categories are usually decided after the

data is collected (in some cases it may be before) They are selected by the researcher and are

specific to the study. Purpose for the research largely influences coding.



Coding Example

The dropout study. Teacher’s perceptions of potential dropout

characteristics Potential dropouts’ perceptions of school Patterns of potential dropouts’ academic

performance (behavior) patterns



Categories

Need not be mutually exclusive. Categories can be whatever as long as they

a) Accurately capture information.b) Are useful in describing phenomena.



General codes

Depending on the data collected we might use: Setting or context codes.

• E.g. Vocational school, comprehensive school Subjects’ perceptions of people or things.

• E.g. Dropouts have different views about their teachers. Process codes

• E.g. different ways in which students go about dropping out of school

Based on experiences



Screenshot① Perspectives of children about interpersonal

conflicts.② Innovative capacity of secondary schools: team

involvement which has four categories.

More examples



4. Perspectives for Qualitative Research

In qualitative research the method of data collection and hypotheses are much more flexible than in quantitative research. In fact, there are two different approaches where the data collection is used to improve future data collection.a) The Funnel Approach

b) Modified Analytic Induction



Questions

How do I choose between a funnel or modified analytic approach? Your case.

When to stop? The point of diminishing returns for data collection,

and additional data collection is not worth the time and effort.



5. Reliability and Validity

Remember the traditional quantitative concepts of reliability and validity? They can cause problems in qualitative research: Reliability:

• External-Can replicate the study• Internal-Based on the internal constancy of experiment.

Validity:• External – Generalizability of conclusion• Internal – Can make a reasonable conclusion



Qualitative Views of Reliability

Reliability: External: So specific to natural settings, but can

make a persuasive argument so that the reader can evaluate if it can be replicated.

Internal: Typically, there are observations from multiple observers so we should compare the agreement rates. Or at least look at the agreement of conclusions.

• Interrator reliability• Concordance : the ratio of agreements to agreements and

disagreements



Qualitative Views of Validity

Validity External: Not usually concerned with

generalizability (tend to be case studies). Can be left up to the reader.

Internal : still apply; Can the data provide a conclusion?



1. Interpretative validity Usefulness: the extent the report informs and

stimulates further research Contextual completeness: fullness and richness

of the description Research positioning: researchers’ direct and

indirect effects on the research sites, participants, etc.

Reporting style: description perceived as authentic or not

2. Trustworthiness

Arguing for internal validity



1. Interpretative validity2. Trustworthiness

1. Triangulation2. Member checking: subjects’ perceptions are

verified in a systematic manner3. Chain of evidence

• outlier• pattern matching• representative check of an interview or artifact• long-term involvement• coding check

Arguing for internal validity continued



1. Comparability the ability of other researchers to extend

knowledge based on the “richness and depth” of the description

2. Translatability the extent to which other researchers understand

the results given theory and procedures underlying the study

External validity：NOT the job of the qualitative researcher, but …



Break : 10 minutes



Use of Technology

Because of the large amounts of data, technology has made things a lot better

The Ethnograph NUD*IST (Non-numeric Unstructured Data-

Indexing, Searching, and Theorizing NVIVO



Types of Designs

While there are many designs specific to the study some general types are Case Study Ethnographic Phenomenological Historical Focus group



CHAPTER 9:Historical Research



Historical Research

DEFINITION: A systematic process of searching for the facts and then using the information to describe, analyze, and interpret the past.

Goal: take a holistic approach to some setting occurring in the past, and understand and interpret it.



Values of Historical Research

1. Shows what is right and what is not working2. Helps provide insight to current events3. Provide perspective for decision making4. Can help predict future trends

• “Those who are unfamiliar with the mistakes of history are doomed to repeat them”

….



The career path of a single school executive from 1914-1922.

P225 See conclusions.

Example



1. The past: within the immediate preceding year or go back

several centuries.2. May have a variety of foci: The history of the development of the teachers’

college in the US would focus on a movement or a concept.

Or we could focus on a specific institution.3. Deals with events occurring in natural rather

than contrived settings for the most part.

Basics



Interpretation is Key

Interpretation is important because the data collection is based on getting information from the past

Mainly, we are interpreting sources. Our interpretation reflects our values and interests. E.g. an interview, but from the historical data.



Knowledge of Sources is Key

Also, because all data is from previous sources it is important that we be very sensitive to the relevance of these sources (data) as not to bias their results.

Need to know where to look for information.



Sources of Information

Because the research involves the past they must use various sources that describe the period of study.

Written documents (books, papers, diaries, letters, etc…)

Physical remains or objects Folk songs, stories, personal interviews



Sources of Information

In collecting the data, it is important to differentiate between a primary and secondary source Primary – First hand account of event being studied

1) E.g. to stud desegregation problem– A court transcript of a desegregation hearing

2) John Dewey’s writing is the primary source to study his views

Secondary – Accounts that are at least one step away from event

1) A news paper editorial concerning the desegregation problem

2) An interpretation by Dewey’s student is secondary source evidence



Methodology

Edson (1986): “There is no single, definable method of historical inquiry.” (p.29)

However, there are 4 steps that have to be done in one way or the other. Figure 9.1.

① Identify the research problem② Collect and Evaluate Materials③ Synthesis of Materials④ Analysis, Interpret, and Formulate conclusions



① Identify the research problem

This is the same as in all other designs. However, they are more flexible so the research problem and hypotheses (if they exist) can be as implicit or explicit as needed.

Also, in identifying the research problem and hypotheses you should be aware of the key terms (beware of vague hypotheses) Are there words that had or have different

meanings across time.



Characteristics of hypotheses in historical research

Hypotheses are not formulated in a statistical sense.

Instead, they are conjectures about characteristics, causes, effects, issues, etc…

Typically, these hypotheses rest on the assumption of fact.

https://en.wikipedia.org/wiki/The_Protestant_Ethic_and_the_Spirit_of_Capitalism


https://en.wikipedia.org/wiki/The_Protestant_Ethic_and_the_Spirit_of_Capitalism�

https://en.wikipedia.org/wiki/The_Protestant_Ethic_and_the_Spirit_of_Capitalism�


Example P228

a) Lewin and Hui 1989: Rethinking revolution: Reflections on China’s 1985 educational reforms.

b) Culter III, 1989: Cathedral of culture: The school house in American educational thought and practice since 1820.

c) Nelson-Rowe 1991: Corporation schooling and the labor market at General Electric.

d) Galenson 1998: Ethnic differences in neighborhood effects on the school attendance of boys in early Chicago.



History of Education Quarterly by the History of Education Society.

Educational Studies by the American Educational Studies Association.

Comparative Educational Review by the University of Chicago Press

Harvard Education Review by the Harvard University Press



② Data Collection and Evaluation

Interpretation is key but because we are collecting sources that describe a time in the past we need to interpret these carefully.

We must evaluate each source to determine how useful it will be External Criticism Internal Criticism

These two are not mutually exclusive



Collection and Evaluation

External criticism: Establishes validity of DOCUMENT Here we are concerned with the reputation of the

article, mistakes, who was the author

Internal criticism: Establishes validity of CONTENT Did the author or document have a reason to lean

in a specific direction (i.e., leave out or add stuff)?



Collection and Evaluation

In evaluating external and internal criticism typically, more that one document is used. Evaluate in chronological order Use multiple sources

• Are reports consistent？

Without these, a source can’t be used



③ Synthesis of Materials

In combining the material, we need to determine the correct set of information.

If sources contradict one another, we need to decide which one is correct.

Pull together information for continuity Maybe chronologically

Reformulate hypotheses



④ Analysis, Interpret, Formulate

Final step is the decision making of the research problem

Hypotheses are supported or rejected

Recognize the possibility of alternative interpretations



Analysis, Interpret, Formulate

In making conclusions be sure not to:

Confuse associations with causality Fail to separate how people SHOULD act with how

people DO act Fail to distinguish between intent and

consequences



Quantitative methods in Historical Research

As was mentioned earlier, Historical Research is typically associated with qualitative research

However, there is the possibility that it will also use some quantitative analysis Usually these are summary statistics Use data from papers, files, census…



Reporting Historical Research

Intro – Introduce the topic, ideas and research questions

Methods – Provides the method and form of the data

Results – Summarize the data that has been collected

Conclusion – Make general conclusions related to research problem



The general structure of a plan or proposal

Reading



Reading:

Next week: Ethnographic Research

Assignment


class 6 ( chapter 8+9) - faculty.ecnu.edu.cn

Documents