copyright ©2014 by pearson education, inc. all rights reserved. research methods, design, and...

Research Methods, Design, and Analysis, Twelfth EditionChristensen • Johnson • Turner

CHAPTER

Twelfth Edition

Research Methods, Design, and Analysis

Introduction to Scientific Research

Why Learn About the Scientific Research Process?

• To learn the research process• Provides a foundation for other courses• To become a critical consumer of information• To develop critical and analytic thinking• Learn to critically read a research article• Necessary for most graduate programs

Definitions

• Research: the process of systematically and carefully investigating a subject in order to learn or discover new information about the world and its components!

Examples of Research Goals

• Identifying and classifying new problems• Determining risk factors or important issues/

behavior..etc • Developing and testing new interventions for

educational challenge • Synthesizing existing knowledge so that it can be

applied by others

Examples of Researcher Benefits

• Acquiring new skills• Satisfying personal curiosity• Fulfilling degree or work requirements• Becoming a published author

The Research Process

The steps of any population or any educational research project are:

Preparing to Publish

The likelihood of publication depends on:– The appropriateness of the research topic for a

wide audience– How well designed the study is and whether it uses

valid methods– How compelling and well written manuscript is

If the goal is to publish the findings of a study, then the researcher must prepare for publication at every step of the process

Step 1

Identifying a Study Question

Selecting a General Topic

https://www.questia.com/library/education/current-issues-in-

education

https://gatorball.wordpress.com/2009/02/04/top-10-research-topics-for-education/

Identifying a study topic is often the most challenging part of a research project.

Each of the possible study topics has its own set of virtues and shortcomings.

Brainstorming & Topic Mapping

• Begin by creating a long list of possible study topics:– Jot down areas of personal interest– Ask friends / colleagues for ideas– Skim abstracts, journals, and books for inspiration

• What topics emerge as a repeating theme?• What might be enjoyable to explore?

FIGURE 2- 1 Brainstorming Questions

To know How to Start a Research? You need to know

why you are doing the research? And what research can do?

• Research teaches methods of discovery? And allows you to discover!

•Its ask you to discover what you know on a topic and what others can teach you. Beyond reading, it often expects you to venture into the field for interviews, observation, and experimentation.

• The process tests your curiosity as you probe a complex subject. You may not arrive at any final answers or solutions, but you will come to understand the different views on a subject.

Why Do Research ?

• Research teaches investigative skills:

A research project requires you to investigate a subject, gain a grasp of its essentials, and disclose your findings. The exercise teaches important methods for gaining knowledge on a complex topic.

• Your success will depend on your negotiating the various sources of information, from reference books in the library to computer databases and from special archival collections to the most recent articles in printed periodicals.

Why Do Research ?

• Research teaches critical thinking:

As you go through the evidence on your subject, you will learn to discriminate between useful information and unfounded or ill-conceived comments.

• Some sources, such as the internet will provide timely, reliable material but may also give you with worthless and undocumented opinions.

Why Do Research ?

• Research teaches logic:

• For example: like a judge in the courtroom, you must make perceptive judgment about the issues surrounding a specific topic.

• Your decisions, in effect, will be based on the wisdom gained from research of the subject. Your paper and your readers will rely on your logical response to your reading observation, interviews, and testing.

Why Do Research ?

• Research teaches the basic ingredients of argument:

• In most cases, a research paper requires you to make a claim and support it with reasons and evidence. For example, if you argue that “urban sprawl has invited wiled animals into our school backyards” you will learn to anticipate challenges to you theory and to defend your assertion with evidence.

Learning Format Variations

Scholarly writing in each discipline follows certain conventions- that is, special forms are required for citing the sources and for designing the pages. These rules make uniform the numerous articles written internationally by million scholars.

https://www.youtube.com/watch?v=VKWKswH29kM

https://www.youtube.com/watch?v=fm4DI53nB6U

Writing skills

The society of language and literature scholars, the modern language Association, has a set of guidelines generally known as MLA style.

Similarly, the American psychological association has its own APA style. Other groups of scholars prefer a footnote system, while still others use a numbering system. These variations are not meant to confuse; they have evolved within disciplines as preferred style.

Understanding a research Assignment

Beyond selecting an effective subject, you need a reason for writing the paper. Literature instructors might expect you to make judgments about it.

Education instructors might ask you to examine the merits of testing a program.

History instructors might want you to explore an event-perhaps the causes and consequences of the 2003 U.S. war on Iraq.

• Understanding the Terminology:

• Assignment in education, psychology, political science, and other social science disciplines will usually require analysis, definition, comparison, or a search for precedents leading to proposal.

• Evaluation:

• To evaluate, you first need to establish clear criteria of judgment and then explain how the subject meets these criteria. For example, student evaluations of faculty members are based on a set of expressed criteria an interest in a student progress, a thorough knowledge of the subject, and so forth.

Continue

Similarly, you may be asked to judge the merits of poem, and art show, or a new computer software.

Your first step should be to create your criteria. What's makes a good movie? How important is a poem’s form and structure? You cant expect the sources to provide the final answers; you need to experiences the work and make your final judgment on it.

• Interpretation:

• To interpret, you must usually answer, “What does it mean? You may be asked to explain the symbolism in a piece of literature, examine a point of law, or make sense of test results. Questions often point toward interpretation:

What does this passage mean?

What are the implications of these results?

What does this data tell us?

Can you explain your reading of the problem to others?

• Definition

Sometimes you will need to provide an extended definition to show that your subject fits into a selected and well-defined category.

Example: Slapping a child on the face is child abuse.

• Proposal

This type of argument says to the reader, “We should do something”. It often has practical applications.

Example: We should cancel all drug testing of athletes because it presumes guilt and demeans the innocent

In addition, a proposal demands special considerations.

First, writers should convince readers that a problem exists and is serious enough to merit action.

Second, the writer must explain the consequences to convince the reader that the proposal has validity.

Third, the writer will need to address any opposing positions, competing proposals, and alternative solutions.

• Questions so far

• https://www.youtube.com/watch?v=q52JYZ8PaEU

Establishing a schedule

• Topic approved by the instructor• Reading and creating a working bibliography• Organizing • Creating notes • Drafting the paper• Formatting the paper• Writing a list of your references • Revision and proofreading• Submitting the manuscript

Methods of Acquiring Knowledge

• Intuition – knowing without reasoning– used in forming some hypotheses – problem – no mechanism for separating accurate

from inaccurate knowledge

• Authority – facts stated from a respected source– can be used in the design phase of a study– can be used when interpreting the data– problem – authority can be wrong

Methods of Acquiring Knowledge

• Rationalism – knowledge from reasoning – used to derive hypotheses– used to identify outcomes that would indicate the

truth or falsity of the hypotheses– potential problem – relying solely on rationalism can

lead two people to reach different conclusions– Insufficient by itself

Methods of Acquiring Knowledge (cont'd)

• Empiricism – knowledge from experience– observation used to collect data in science– facts that concur with experience are accepted and those

that do not are rejected– potential problem is researcher bias– must be conducted under controlled conditions– systematic strategies must be used to reduce researcher

bias and maximize objectivity

Science

• Designed to systematically produce reliable and valid knowledge about the natural world

• From the Latin verb scire which means “to know”

• The English term was coined in the 19th century by William Whewell (1794-1866)

• Different scientific methods have been popular historically

Scientific Methods

• Induction – specific to general reasoning – Aristotle (384-322 BCE)– still used today when generalizing from specific experiments

to general hypotheses or theory

– Researchers relying on a sample to represent a population

Scientific Methods

• Deduction – general to specific reasoning– involved in forming hypotheses from theory

Scientific Methods• Hypothesis testing

– formulating a hypothesis to explain some phenomenon that has been observed and then comparing the hypothesis with the facts

– prominent from mid-19th century to about 1960, but still used extensively today

– associated with logical positivists philosophical position started by scholars at University of Vienna believed that statements meaningful only when verifiable by observation

Scientific Methods

• Naturalism – science should be studied and evaluated empirically – we should continually evaluate our theories based

on empirical adequacy

1.Thomas Kuhn (1922-1996) paradigm – framework or thought or belief by which you

interpret reality science governed by types of activities

– normal science-shared paradigm– revolutionary science – replace one paradigm with another

Scientific Methods

2. Paul Feyerabend (1924-1994)– Feyerabend’s Theory of Science

argued there is no such thing as a method of science, but science has many methods

advocated that science does not give knowledge superior to other forms of knowledge

his position – the unchanging principle of scientific method is that “anything goes”

scientific knowledge is not better than other forms of knowledge

What Is Science?

• Multiple methods and practices used to develop secure scientific knowledge

• Scientists must– be skeptical, creative, and systematic – identify problems – question current solutions that are not working – creatively and systematically come up with new

solutions – subject these new solutions to empirical testing

What Is Science?

• To be successful, science must– conduct research ethically – critically self-examine its practices to determine

what is working and what is not – engage in ongoing learning and improvement

Basic Assumptions Underlying Scientific Research

• Uniformity or regularity in nature– determinism – the belief that mental process are

fully caused by prior natural factors– probabilistic causes – a weaker form of determinism

that indicates regularities that usually, but not always, occur

• Reality in nature – the assumption that the things we see, hear, feel,

smell, and taste are real

Basic Assumptions Underlying Scientific Research

• Discoverability– the assumption that it’s possible to discover the

regularities that exist in nature– task may not be simple, e.g., cure for cancer or

AIDS– Two components

discovering the pieces of the puzzle putting them together

Characteristics of Scientific Research

• Control– holding constant or eliminating the influence of

extraneous variables– allows for unambiguous claims about cause and

effect– Placebo effect

improvement due to participants’ expectations for improvement, rather than the actual treatment

• Operationism– representing constructs by a specific set of

operations– operational definition

defining a concept by the operations used to represent or measure it

Example of Operationalization

• Replication– reproduction of results in a new study– reasons for failure to replicate

effect doesn’t exist replication study is not an exact replication effect may depend on context

– meta-analysis a quantitative technique for describing the relationship

between variables across multiple studies

Role of Theory in Science

• To summarize and integrate existing data• To guide new research• Continuous interaction between theory and

empirical observation– logic or context of discovery

the inductive part of science

– logic or context of justification the deductive part of science

Figure 1.2Illustration of the relationship between theory and research.

Role of Scientist in Psychological Research

• Curiosity– goal is the pursuit of knowledge and the uncovering of

regularities in nature

• Patience– gaining knowledge from nature can be a slow, tedious

process

• Objectivity– the scientist’s personal wishes and attitudes should not

affect his or her observations

• Change– devising new methods and techniques for investigating

nature

Objectives of Psychological Research

• Description– portraying the phenomenon accurately– focusing on characteristics and degree to which

they exist– e.g., Piaget’s theory of child development arose

from detailed observations of his own children

• Explanation– identifying the cause(s) of the phenomenon

Objectives of Psychological Research

• Prediction– anticipating the outcome and the occurrence of an event– identifying risk factors of a phenomenon can help you to

predict when it might happen– e.g., academic success

• Control– manipulation of the conditions that determine a phenomenon– different meanings of the word control

controlling antecedents to produce a desired outcome eliminating influence of extraneous variables

Pseudoscience

• An approach that claims to be scientific, but in fact violates many role of science

• Attempted association with science made in an attempt to gain legitimacy

• Examples– commercials that claim “scientifically proven”

Strategies Used in Pseudoscience

• Creating new (ad hoc) hypotheses in order to explain away negative findings– characterized by statements that can’t be falsified

or ad hoc hypotheses to explain problems with the claim

• Exclusive use of confirmation and reinterpretation of negative findings as supporting the claim– science tries to prove hypotheses wrong

• Absence of self-correction through continual and rigorous testing of the claim– does not try to verify or refute claims

• Reversed burden of proof – asks critics to prove that their claims are wrong

• Overreliance on testimonials

• Use of ambiguous or confusing language to make a claim sound as if it has survived scientific method– language that confuses vs. clarifies– uses scientific terms to sound respectable

• Absence of any connection to other disciplines that study issues related to the claim

Questions

Reviewing the Literature

“Reviewing the literature” = background reading

Start with informal sources that provide basic information about the education related

issue of interest, then move on to more formal reports.

Abstract Databases

Abstract: paragraph-length summary of an article, chapter, or book

Abstract databases: searchable collections of thousands of abstracts

– Some are subscription services available through university libraries

– Some are open access and available for free to everyone

Full-Text Articles

The only way to truly understand a study is to read the full text of the article.

How to acquire full text articles:•Library websites (e-journals) and physical collections•Journal websites / Internet searches•Request an “interlibrary loan” from a university library•Email the author and request an electronic copy

Steps for Reading Articles

1. Re-read the abstract

2. Look carefully at the tables and figures for important results

3. Read the entire text of the article– Take notes about which exposures, diseases, and

populations the study examined – and how they might relate to a new research project

4. Review of the reference lists for related papers

What Makes Research Original?

Originality does not require the discovery of a newly emergent disease in a previously unrecognized people group on a remote island.

Originality requires one substantive difference from previous work: • a new behavior of interest• a new policy of interest• a new source population• a new time period under study• a new perspective on a field of exploration

Example: A literature review finds several studies that have shown that older adults (the population) who take 30-minute walks several times a week (the exposure) score higher on memory tests (the disease or outcome) than adults who do not routinely walk for exercise.

A proposed new study could examine…

• Is playing table tennis equally effective at improving academic achievement at school?

• The real challenge in reviewing the literature and selecting a study question is not finding a previously unexplored topic.

• The main challenge is to limit the research project to one solid idea out of the many possibilities.

• The aim of a research project is usually to find and address gaps in the literature (missing pieces of information that a new study could fill) and to build on previous work.

Focusing the Research Question

FIGURE 4- 1 Primary, Secondary, and Tertiary Research

FIGURE 4- 2 Key Considerations

Study Goals & Specific Objectives

The literature review and consideration of a study approach should lead to the selection of one very specific study topic that can be stated in terms of a single overarching study goal or study question.

A study goal often includes the specific exposure, disease, and population that will be the focus of the study

FIGURE 4-3 Examples of Study Goals

Study Goals & Specific Objectives

After finalizing the overarching study goal, the researcher should identify three or more specific objectives (also called specific aims or specific hypotheses) that stem from the main study goal.

•Each of these specific objectives should take the form of a measurable question or a “to” statement.•Each should represent a logical step toward answering the main study question.

Example

• Study goal: “to assess the impact of air quality on school performance in kindergarten students in Liwa area.”

• Specific objective #1:

1. To measure the prevalence of high blood contaminant levels in a random sample of kindergarten students in Liwa.

Example

• Study goal: “to assess the impact of air pollution on school performance in kindergarten students in Liwa.”

2. To determine whether children in that sample with high blood metal levels have lower scores on academic tests than children with lower metal levels.

Example

• Study goal: “to assess the impact of lead poisoning on school performance in kindergarten students in southeast Michigan.”

3. To estimate the total impact of air contamination levels on kindergarten performance in Liwa by applying the rates in the sample population to the total population of the region.

Example

• Study goal: “to assess the impact of air pollution on school performance in kindergarten students in Liwa area.”

• Note that all three of these specific objectives relate to the overall study goal and provide a clear pathway for achieving the main goal.

FIGURE 4- 4 Questions Essential to the Success of the Project

Questions

Understanding & avoiding plagiarism

Outline

What is plagiarism? Using sources to enhance your credibility. Using sources to place your work in its proper context. Honoring property rights Avoiding plagiarism. Sharing credit & honoring it in collaborative projects. Honoring & crediting source in online class room. Seeking permission to published material on your web site.

WHAT IS PLAGIARISM?

Plagiarism: is offering the words or ideas of another person as your own.

Also, its taking another's work & copying other’s ideas without citation or references.

You must put others' words in quotation marks & must cite your source(s) by give citations when using others' ideas.

Using sources to enhance your credibility

Citing a source in your paper even if its short sentence, signals that you have researched the topic, explored the literature about it and have the talent to share it.

Citation will enable readers to identify the sources used and enhance your image as a researcher.

Placing your work in its proper context

Your sources will reflect all kinds of special interest, so you need to position them within your paper as reliable source.

It must be screen internet sites and examined printed articles for:Special interests that might color reportLack of credentialsAn unsponsored Web siteOpinionated speculation.Trade magazines that promote special interestsExtremely liberal

Honoring property rights

Honoring property rights mean to have the copyright for your work in order to be able to publish .

Copyrights begins at the time of creative work is recorded in written document.

Avoiding plagiarism• Try to express your own ideas & words by using summary,

paraphrase or direct quotation.• Rethink & reconsider ideas gathered by make meaningful

connection & give the writer full credit when refer their ideas.

• Providing academic citation you must:– WHO you’ve read.– HOW you used it in your paper.– WHERE others can find it.

Avoiding plagiarism

Major violations, which can bring failure in the course : The user of another students work. The purchase of a scanned research paper. Copyright whole passages into a paper without

documentation. Copyright a key, well-worded phrase into a paper

without documentation. Putting specific ideas of others into your own words.

Avoiding plagiarism

• Errors caused by carelessness:

– The writer fails to enclose quotation marks, yet he/she provides an in-text citation with name and page number.

– The writer’s paraphrase never becomes paraphrase.

Avoiding plagiarism

The academic writer needs two things beyond reliability:A clear trial others researches to follow if they

also want to consult the sourceInformation for other researches who might

needs to replicate .

Common knowledge exceptions1. Local knowledge.

You and your reader share local or regional knowledge on a subject.

Example: Early native Indians on the plains called themselves Illiniwek ( which meant strong men), and French pronounced the name Illinois (angle 44)

2. Shared experience. coursework and lectures will provide the class with the

same information.

Common knowledge exceptions

1. Common facts. Factual information that might found in fact book, or

dictionary need not be cited.

Correctly Borrowing from a source: Mean borrow idea from original reference marital

without copy and past them

Checklist Common knowledge exceptions

• Don’t document the source if an intelligent person would & should know the information

• Don’t document terminology & information from the classroom

• Don’t document the source if you know the information without reading it in article or books

• Don’t document information that has become general knowledge

Required instances for citing a source

An original idea derived from a source whether quoted or paraphrased.

Your summery of original ideas by a source Factual information that is not common knowledge within

the context of the course. Any exact wording copied from a source.

Sharing credit in collaborative projects

• Collaborative writing as group where each members support and understand his/her role in order to achieve an agreed-upon goal.

Honoring & crediting source in online class room.

Honoring: mean copyright, having a permission to publish your work.

Crediting source in online mean need to cite each document, image, URL, electronic articles using from the web.

Seeking permission to published material on your web site.

Principles to cite online classroom:Credit sources in your online communications

just as you would in printed research paperDownload to your file only graphic image &

text from siteSeek permission if you download substantive

blocks of material.

Seeking permission to published material on your web site.

publishing in web site need to follow these guideline:Seek permission for copyrighted material you

publish . If you make an attempt to get permission you

must provide it for not profit.Document all the sources that you featureBe prepared for people to visit your web site &

borrow from it.

Assembling a Support Team

Research projects benefit from the input of technical and cultural experts.

A team of collaborators should be assembled early in the research process.

FIGURE 5- 1 Support Team Members

Coauthors

Although some papers in education and humanities have solo authors, most papers have about four coauthors, and some have dozens of coauthors.

Core Collaborators

Sample terms used to describe key roles:• Lead researcher = first author = the researcher

who will do the majority of the work• Senior researcher = last author = an

experienced researcher who guides the work of a newer investigator

Core Collaborators

Other potential coauthors:• An expert on the research topic or the study

population• An expert on the study design or other methods

being used for the research• A statistician• Other key contributors who are significantly

involved in the design and conduct of the study and in the editing and polishing of the manuscript

Additional Technical Support

Consultants might contribute to the project on a very limited basis and may not meet the criteria for being coauthors, but they should be acknowledged and thanked in any resulting manuscript:• Laboratory technicians• Statistical consultants• Librarians• Technical editors

Authorship Criteria

Each coauthor must have met all three of the following conditions:

1. Substantial contributions to conception and design and/or acquisition of data and/or analysis and interpretation of data

2. Drafting the article and/or revising it critically for important intellectual content

3. Final approval of the version to be published

Authorship Criteria

• “Acquisition of funding, collection of data, or general supervision of the research group alone does not constitute authorship.”

Authorship Example #1

• A person who conducts interviews for the project but does not contribute further would not be eligible for authorship.

• An interviewer who also writes a paragraph for the discussion section would meet authorship criteria.

• A data entry assistant who makes no additional contributions to the project would not be considered an author.

• A data manager who runs statistical tests and creates a table for the manuscript would meet authorship criteria.

• A technical editor who cleans up the grammar and spelling in a manuscript does not earn authorship.

• An editor who raises important questions about the interpretation of the results and the meaning of the work may be eligible for authorship.

Authorship Criteria

• “All persons designated as authors should qualify for authorship, and all those who qualify should be listed.”

• No gift authorships• No ghost authorships

Authorship Order

A typical justification for authorship order:• The first author is the person who was the most

involved in writing the manuscript.• The remaining authors are listed in order of

contribution, defined in terms of time dedicated to the project as well as intellectual contribution. – List authors with equal contributions in

alphabetical order• The senior (supervising) author is listed last,

unless s/he prefers to be listed in order of contribution.

Decisions about Authorship

• Publications are an important metric of success in the sciences and academia, and authorship is often the only reward for the time put into a project. As a result, authorship decisions can be very stressful.

• It is helpful to decide before a contributor does any work on the project whether that person will be a coauthor and what role s/he will play.

• At the end of the project, there should be no surprises about who is being included or excluded as an author.

Decisions about Authorship

• If someone making a minor contribution will not be a coauthor, make sure that s/he is not asked to write any part of the paper or to provide critical feedback.

• If someone will be a coauthor, make sure that s/he has the opportunity to make an important intellectual contribution to the writing of the paper.

• Any disputes over authorship criteria or the order of authors are usually best referred to the senior author on the paper.

Step 2

Selecting a Study Approach

Overview

The second step in the research process is to select a general study approach.

This section provides an overview of 8 common study designs.

1. Reviews / meta-analyses

2. Correlational (ecological) studies

3. Case series

4. Cross-sectional surveys

5. Case control studies

6. Cohort studies

7. Experimental studies

8. Qualitative studies

Overview of Study Approaches

FIGURE 6- 1 Summary of Study Approaches

FIGURE 6-2 Primary, Secondary,

and Tertiary

Study Approache

Study Duration?

Primary Studies:

Study Duration?

Secondary & Tertiary Studies:

The study duration is dependent on how existing data & articles will be acquired.

Questions

Reviews

Overview

A review article or meta-analysis carefully gathers all prior publications on a specific topic and summarizes them to provide a big-picture analysis.

Steps:

1. An extensive search of the literature

2. Extraction of key information from relevant articles

3. Clear and concise presentation of this information

Benefits

• A literature review is an effective way to become an expert in the literature on a well-defined topic

• A literature review is a helpful step in preparing for future primary or secondary analyses

• Review articles are often cited more often than reports of individual field studies

Limitations

• Not all journals publish review articles (especially reviews that the editors do not solicit)

• Reviews are sometimes perceived to be a less rigorous form of research than projects that collect new data and/or involve statistical analysis

Selecting a Topic

• The most important decision is to select a topic that is narrow enough that all the relevant publications can be acquired.

• The topic may need to be modified after a preliminary search, depending on the number of articles available.– 8 = too few expand the scope– 352 = too many narrow the scope

Library Access

• The full text of every relevant article must be identified and obtained.

• Check with a university librarian about the library’s policies and the fees that may be charged for the use of interlibrary loan services.

• Maintain a meticulous system for tracking articles that have already been acquired, those that have been requested but not yet received, and those that need to be requested.

Narrative Reviews

• Narrative reviews tell a “story” about a well-defined topic using evidence from the literature to support the “plot”

• Narrative reviews must be carefully organized by theme, methodology, chronology, or some other guiding principle

• The absence of a systematic search strategy must be justified by the researcher

Systematic Reviews

• Systematic reviews are designed to minimize the bias that might occur when review article authors handpick the articles they want to highlight

• After the identification of the study question, the most important decision in a systematic review is the selection of keywords and inclusion criteria

• The goal is to craft a search strategy that identifies all the articles ever published on the narrow, well-defined area covered by the review

Systematic Reviews

• Once the articles are identified from one or more abstract databases, each article is screened to see whether it is eligible for inclusion.

• Relevant information is extracted from all eligible articles and presented in table form.

• Then the trends and key observations are summarized.

Meta-Analysis

• The goal of a meta-analysis is to combine the results of several high-quality articles that used similar methods to collect and analyze data into one summary statistic.

• Meta-analysis usually begins with a comprehensive systematic review of the literature to identify every single possibly relevant article.

Meta-Analysis

• The inclusion criteria for a meta-analysis are usually more restrictive than they are for systematic reviews.

• These restrictions are important because a summary statistic is only meaningful when every study included in the meta-analysis has very similar definitions for exposures and outcomes, similar study designs and methods, and similar populations.

• Trying to combine dissimilar studies could hide real and meaningful differences among populations.

Meta-Analysis

The steps of a meta-analysis are to:• Conduct a systematic review• Assess the quality and comparability of each eligible

study• Extract statistical results from each study that meets

all inclusion criteria • Combine these statistical results into one summary

statistic

FIGURE 7-1 Key Characteristics of Reviews and Meta-Analyses

Key Characteristics of Reviews

and Meta-

Analyses

Correlational (Ecological) Studies

A correlational (ecological, aggregate) study uses population-level data

to examine the relationship between exposure rates and disease rates.

http://www.youtube.com/watch?v=nm9X_eO812Y

Overview

• Population-level data are used to look for associations between two or more group characteristics

Data Sources

One (or more) data source(s) that contains comparable information about the population characteristics of interest must be identified.

Information about all the variables of interest must be available for a suitable number of populations, which can be grouped by place or time.

Examples of Populations

• All member nations of the United Nations• All 50 states from the United States• The largest 20 metropolitan areas in the United

Kingdom• All the counties in the state of Michigan• A random sample of census tracts in New York City• Historic data for the past several decades from one or

more place-based populations

Exposures and Outcomes

• At least one characteristic of the populations being examined is designated as an exposure– Exposures are often environmental measures likely to be

fairly consistent across an entire population• At least one characteristic is designated as an outcome

Aggregate Data

• Population characteristics are in the form of aggregate (grouped) data, such as:– the proportion of each population with a particular

characteristic– the average value of the variable in the population

Examples of Exposures

• The percentage of adults age 30 and older who have not completed at least 12 years of education

• The mean income in the population• The median age• The number of rainy days over a given year in the

population• The average ultraviolet radiation index during midday in the

hottest month of the year

Cautions

• Correlational studies are valid only if the data points are comparable.

• If multiple sources of data are used or if the data were collected over a lengthy period of time, then the definition of exposure or disease may differ from one population to another and may not be comparable.

• In some populations, exposures and diseases may be routinely undercounted or routinely over-diagnosed compared to other populations.

Data Management

• Data should be entered into a spreadsheet• Each population (A, B, C, etc.) is in its own row• Each exposure and each outcome is in its own column

Analysis: Correlation• On a scatterplot used to illustrate correlation, each point

represents one population in the study. • The exposure is plotted on the x-axis, and the outcome or

disease is plotted on the y-axis.

Analysis: Correlation

• When all the points fall neatly in a line, then the correlation is strong.

• When the points are not exactly linear but a line for trend can be drawn, then the correlation is mild or moderate.

• When the points appear to be randomly placed and no obvious line can be drawn through them, then the correlation is weak or nonexistent.

Analysis: Correlation• If higher levels of exposure are linked to higher rates of

disease, then the slope is positive. • If higher levels of exposure are linked to lower rates of

disease, then the slope is negative.

• For continuous variables and other variables with responses that can be plotted on a number line, a Pearson correlation coefficient (r) should be used to calculate the correlation.

• For variables that assign a rank to responses or that have ordered categories, use the Spearman rank-order correlation (designated by the letter r or the Greek letter r (rho) in most statistical programs).

• r = –1: all points lie perfectly on a line with a negative slope• r = 1: all points lie perfectly on a line with a positive slope• r = 0: no association between the exposure and outcome• r2 shows how strong a correlation is without indicating the

direction of the association

Analysis

• Use linear regression models when the goal is to:– compare more than two variables– understand the relationship between two variables while

controlling or adjusting for the effects of other variables

Age Adjustment

When the populations being compared have very different age structures, age adjustment may be necessary to make a fair comparison among populations.

Avoiding the Ecological Fallacy

Correlational studies compare groups rather than individuals.

No individual-level data are included in the analysis, only population-level data.

The incorrect attribution of population-level associations to individuals is called the ecological fallacy.

Even though a population with a higher rate of exposure to something has a higher rate of disease than populations with lower exposure rates, individuals in that population who have a high level of exposure do not necessarily have the disease.

The experience of an individual in a population may vary significantly from the population average.

It would be incorrect to assume that any one individual from a country with a high average body mass index (BMI) will be obese or that an individual from a country with a low average BMI will not be obese.

However, it is appropriate to identify trends in populations and to use those observations to generate hypotheses for individual-level studies that will test for relationships between the characteristics of interest in individuals.

FIGURE 8- 1 Key Characteristics of Correlational (Ecological) Studies

copyright ©2014 by pearson education, inc. all rights reserved. research methods, design, and...

edition research methods

research topic

pearson education

education slide

scientific research

educational research

process slide

research article necessary

Documents

christensen 71

shakespeareonfilm twelfth night · 2015-10-27 ·...

creative problem solving with lego •tamara christensen...

diamantina christensen

christensen fÄrdigt

christensen partimento

christensen 140

c christensen

winterizing your vehicle - christensen automotive€¦ ·...

twelfth night

dana christensen

one hundred twelfth congress elijah e. … · michael r....

engineering mechanics: statics, twelfth edition russell c...

rudebusch lopez christensen

shrr- y...standard 84 buchwald otto b • ••.^4^ ^^3...

steven a. christensen, #5190 zane l. christensen, #14614

biographies & abstracts · clayton christensen: the...

research methods, design, and analysis · research methods,...

12 christensen

think big – start small hal christensen...