methodology. research methodology research methods are specific procedures used to gather and...

Methodology

Research Methodology Research methods are specific procedures

used to gather and analyze research data of a particular paradigm (e.g. experimental versus descriptive; quantitative versus qualitative).

They are concerned with the protection of human participants, research design principles, sampling, data collection, instrumentation, and statistical analysis.

Research participants are essential to the conduct of research and to the progress and discoveries researchers make in a variety of fields.

Research Ethics The relationship between researchers and

participants is critical and should be based on accurate information, trust, and respect.

Throughout history, harmful research has been conducted on unwilling human participants.

Over the years, public outrage has led to the development of a number of advisory commissions and codes of research ethics.

As recently as 2000, the Office of Human Research Protections (OHRP) was established within the US Department of Health and Human Services to provide leadership for all 17 Federal agencies that carry out research involving humans.

Guiding Principles The three fundamental ethical principles

that guide the ethical conduct of research involving human participants areo respect for persons (autonomy)o beneficenceo justice

You will learn more about human subjects protection when you complete your tutorial.

Respect for PersonsTwo ethical standards are contained

within this principle:o Individuals should be treated as autonomous

agents;o Persons with diminished autonomy may need

additional protections. An autonomous individual is one capable

of deliberation about personal goals and acts, as well as opinions and choices.

Respect for Persons

Prospective research participants must be given the information needed to determine whether or not to participate in a research study.

There should be no pressure to participate and ample time to decide.

Individuals must enter into research voluntarily and with adequate information.

This is called informed consent.

Respect for Persons When an individual or a class of participants

is considered incapable of informed decision making (e.g., children, people with severe developmental disorders or dementias), respect for persons requires giving him/her the opportunity to choose, to the extent they are able, to participate or not participate in the research activity.

In some instances, respect for persons may require seeking permission of other parties, such as a parent or legal guardian.

BeneficenceAnother ethical standard has to do with

efforts to secure the well-being of human participants and to protect them from harm.

The principle of beneficence obligates the researcher to maximize possible benefits and minimize possible harm.

The researcher must decide when it is justifiable to seek certain benefits despite inherent harms or risks.

No individual should be intentionally injured.

Justice The ethical considerations of risks versus

benefits leads to the question of justice. This principles requires that participants

be treated fairly and without bias. The concept of justice may be questioned

when deciding who will be given an opportunity to participate, who will be excluded, and the reasons for exclusion.

When selecting participants for research, researchers are responsible for ensuring that selection is equitable.

Justice Research should only involve persons from

groups that are directly related to the problem being studied and that are likely to benefit from the research.

To be fair, the researcher must ask: Are some classes of persons being selected simply because of their availability, their compromised position, or their vulnerability.

Vulnerable research participants are persons who are relatively or absolutely incapable of protecting their own interests.

Vulnerable populations include:

Justiceo Childreno Individuals with questionable capacity to

consento Prisonerso Fetuses and pregnant womeno The terminally illo Students/employeeso Comatose patients

The researcher should be cognizant of the special problems of research involving vulnerable populations.

Justice It is important to justify the proposed

involvement of these populations in the research, and include additional safeguards for their safety and welfare.

Researchers must also be careful not to overprotect vulnerable populations and exclude them from participating in research in which they wish to participate, particularly where the research involves therapies for conditions in which there are not available treatments.

Informed Consent Once the researcher has a carefully defined

research question, a valid design, and protocol for a research project, it is time to plan for the informed consent of those invited to participate.

Informed consent is a legal, regulatory, and ethical concept based on respect for the individual and the individual’s autonomy and right to define his/her own goals and make choices designed to achieve those goals.

This right applies to all types of medical intervention and clinical research.

Informed Consent In research, however, informed consent is

more than just obtaining a signature of the potential research participant.

It is a process that involves:o conveying accurate information about the study

and its purpose;o disclosing known risks, benefits, alternatives,

and procedures;o answering questions; ando enabling the potential participant to make an

informed decision about whether to participate.

Informed Consent

Finally, participants must be informed that even after they have made a voluntary agreement to participate in the study, they may withdraw such agreement at any time without penalty.

If a person is unable to provide his/her own consent, a legally authorized representative can, in some cases, give permission for participation in research.

A legally authorized representative is a legal guardian; a parent (for children only); and in some cases, a validly designated durable power of attorney for health care.

Informed Consent Because children have not attained an age at

which they can consent to research or treatment, the parent or guardian may provide “permission” for the child to participate in the study.

In most cases, the child must indicate willingness to participate by assenting to the study.

Edinboro University has a Human Subjects Review Board (HSRB) which reviews research applications of planned research studies by faculty, staff, and students.

The policy, the application form, the informed consent guidelines, and the informed consent outline form are available at http://www.edinboro.edu/departments/provost/human_subjects_review_board.dot

Research Design Principles There are as many research designs as

there are hypotheses to be tested. Two major classes of research design that

have broad applicability in communicative disorders research include group designs and single-subject designs.

For both experimental and descriptive research, group designs can be classified as between-subjects, within-subjects, or mixed.

Groups Designs In group designs, one or more groups of

subjects are exposed to one or more levels of the independent variable, and the average performance of the group of subjects on the dependent variable is examined to determine the relationship between the independent and the dependent variables.

In between-subject designs, different groups of subjects are compared to each other.

Group Designs

In within-subject designs the same group of subjects is compared in different situations.

Mixed designs include both types of comparisons in the same study.

In experimental between-subject designs, different groups of subjects are exposed to different treatments or levels of the independent variable.

Between-Subjects Designs The independent variable or experimental

treatment is applied to one group of subjects (the experimental group) but not applied to another group of subjects (the control group).

The difference between the performance of the two groups is taken as an index of the effect of the independent variable on the dependent variable.

There are four types of between-subject designs: random selection designs, random assignment designs, matched group, and natural group designs.

Random Selection Designs With random selection designs, two groups are

randomly selected from the same population. Variables such as age, gender, and education

are controlled by random selection of subjects. For example, two groups of 100 children are

randomly selected from the total population of children with language disorders.

The independent variable, or experimental treatment, is vocabulary training.

The two levels of the treatment are vocabulary training and no vocabulary training.

Random Selection Designs The experimental group is given extensive

training in which new words are introduced in the context of stories.

The other group, the control group, is given no training.

At the completion of the training, both groups take a vocabulary test that includes new words.

The score on the vocabulary test, or the measure of the effect of treatment, is the dependent variable.

Subject variables such as age, gender, and education of the children are controlled by allowing them to vary randomly.

Random Selection Designs Situation variables such as testing room,

time of testing, and the person giving the vocabulary test are held constant.

The difference between the two groups in vocabulary test scores is evaluated by statistical analysis to estimate the effect of training in defining words on the vocabularies of children with language disorders.

Random Assignment Designs Random assignment designs overcome the

difficulty of access to an entire population. When only a restricted population of

subjects is available, they can be randomly assigned to one group or another.

Like the random selection procedure, subject variables are controlled by allowing them to vary randomly.

Except for the population from which subjects are selected, there is no difference between random selection and random assignment designs.

Matched Group Designs The subject variability that may be a

problem in random assignment designs can be overcome by matched group designs.

In a simple matched group design, one or more variables that may affect the dependent variable can be held constant between groups by matching the groups on those variables.

The two groups can be matched on the dependent variable prior to treatment.

The two groups can be matched also on the independent variable prior to training.

Matched Group Designs

Matched Group Designs When matching to either a dependent or

independent variable, it is important that the groups have the same mean and distribution of the scores, or ages, around the mean.

The matched group design is very appropriate for communication disorders research, because the relatively small populations available to most researchers make it very difficult to select comparable groups by random assignment.

What is sacrificed in the simple matched group design, as compared with the random selection design, is the ability to generalize findings to the entire population.

Naturalistic Group Designs In the previous designs, the two groups

were selected from the same population and the independent variable was different treatment of the two groups.

In this design, the groups are selected from two different populations.

The independent variable is a difference between groups “created by nature” that exists prior to the selection of the groups.

The effect of this independent variable is studied.

Naturalistic Group Designs

For example, a randomly selected group of boys can be compared with a randomly selected group for girls in their performance on a vocabulary test to determine whether girls have better vocabularies than boys.

The independent variable is gender, and dependent variable is once again vocabulary score, and all other variables are uncontrolled.

Natural Group & Matched Group Design: Combined

In studies in which communication disorder groups are compared with normal control groups, the natural group design must be used because the two groups come from different populations.

This is also true of comparisons between communication disorder groups that differ in type or severity of disorder, or in other variables such as age and gender.

Since the combined matched group and natural group design is so important in our discipline, several examples will be given.


Let’s say it is important to determine whether children with phonological disorders also have higher level language disorders.

Higher level language skills of a natural group of children with phonological disorders and a natural group of children without phonological disorder would be compared.

The independent variable would be the presence of phonological disorders.


The dependent variable would be a measure of higher level language skill (.e.g., a vocabulary test or a grammar test).

The control group of children without phonological disorders would be selected to match the phonological disorder group in relevant variables such as age, gender, education, and cultural background.

Natural Group & Matched Group Design: Design

With all relevant variables controlled, a difference between groups on the measure of higher level language skill would provide evidence that children with phonological disorders do have higher level language disorders.

However, because the difference could be attributable to variables other than phonology per se (i.e., organic or environmental factors), one could not conclude that there is a cause and effect relationship between phonological disorders and higher level language disorders.

Natural Group & Matched Group Design: Design

All that can be concluded is that children with phonological disorders tend to have higher level language disorders, for whatever reason.

Natural/matched group designs can also be used to compare two groups of communication disorders where the natural groupings are defined by differences in experience.

A study might be designed to assess the effects of different methods of training children with congenital sensorineural hearing impairment.


A group trained by auditory oral methods and a group trained by total communication methods would be compared in educational achievement.

The independent variable would be the method of language training, and the dependent variable a measure of educational achievement especially adapted for children who are hearing impaired.

Variables controlled by matching might include age, gender, years of education, cultural background, amount of hearing impairment, and time of beginning language training.


However, it is often difficult to match natural groups on all relevant variables; and

A difference between natural groups does not prove that the independent variable has caused the difference.

Descriptive Between-Subjects Designs

Between-subject designs are also common in descriptive research.

In descriptive between-subject designs, different groups of subjects are compared with each other with regard to their performance on some criterion variable.

Examples of between-subject descriptive research include comparative research, cross-sectional developmental research, and surveys that compare the responses of different groups.


Comparative research involves the description of dependent variable differences between groups of subjects who differ with respect to some classification variable (e.g., children with palatal clefts vs. children without palatal clefts).

Cross-sectional development research uses a between-subject design because separate groups of subjects who differ with respect to age are compared.


Between-subject descriptive research designs may be bivalent, in which cases the classification variable is broken down into two mutually exclusive categories (e.g., laryngectomees vs. speakers with normal larynges).

Between-subject descriptive designs can also be multivalent, in which case the classification variable is divided into categories that are ordered along some continuum (e.g., mild vs. moderate vs. severe hearing loss).


Between subject descriptive designs can also include comparisons of subjects who are simultaneously categorized with respect to more than one classification variable (e.g., male vs. female; mild vs. moderate vs. severe mental retardation).

The first step in between-subject descriptive research is to define criteria for selecting subjects from each category of the classification variable.


Classifications must be constructed that are mutually exclusive, that is, subjects should fall into only one category with regard to each classification variable.

For example, in a comparison of patients with cochlear hearing loss and patients with conductive hearing loss, all subjects must fit the definition of only one of the two groups.

Patients who were found to have both a cochlear and a conductive component to their losses would have to form a third comparison group, that is, patients with mixed hearing losses.


The second step to between-subject descriptive research is the attempt to equate subjects on extraneous variables.

Because subjects cannot be assigned randomly to the various classification, equivalence of groups on all extraneous variables is quite difficult to achieve.

The best alternative is to try to minimize group differences on extraneous variables known to correlate with the dependent variable.

Within-Subjects (Repeated Measures) Designs

In repeated measures design, the levels of the independent variable are varied within a single group of subjects.

They are used in communication disorders research when there are not enough subjects available for two independent groups, when it is difficult to match relevant variables in two independent groups, or when it is more efficient to carry out the experimental procedures with one group.

The basic rules of the within-subjects design are to assess the dependent variable twice in a single group of subjects.

Within-Subjects (Repeated Measures)

Designs The difference between two

assessments demonstrates the effect of the independent variable.

Subject variables such as age, gender, and education do not have to be controlled because the same group is used of both values of the independent variable.

Limitations of repeated measurement designs include order effects.


Designs Order effects may take the form of a practice

effect that improves performance or a fatigue or boredom effect that impairs performance.

When repeated measurement designs are used to assess the effects of training, there is a lack of control for the possibility that improvement might have occurred without training.

A control procedure for order effects is to counterbalance the order of presentation.


Designs However, counterbalancing cannot be

used to control for order effects in training studies because the pretraining must always be the first measure.

To control for the possibility of improvement without training, the independent group and repeated measure designs can be combined, with a trained and untrained group tested before and after training.

Between-Group and Within-Subjects (Mixed) Designs

In many research studies, more than one independent variable is considered.

The effects of two or more independent variables on a dependent variable may be examined.

In other cases, one independent variable is studied with a between-subjects comparison, and the other independent variable is studied with a within-subject comparison.

This mixed design incorporates both the between-group and the within-subjects tactics.

Single-Subject Designs Single-subject designs focus on the individual

behavior of subjects rather than considering the average performance of a group of subjects.

Single case research designs are appropriate for communication disorders research because they are intended to demonstrate that interventions cause changes in behavior.

Single-subject designs can examine the behavior of more than one person, but the data of each person will be evaluated individually rather than as part of a group average.

A number of different designs are available.

ABA, ABAB Designs The most basic single subject designs are

withdrawal and reversal designs. After several baseline (A) measures, a

treatment is given until the target behavior (B) changes.

Then the treatment is taken away (withdrawal design) or nontarget behaviors are reinforced (reversal design) until the target behavior returns to baseline, and the treatment (B) is usually given once more.

ABA, ABAB Designs This procedure is designed to prove that the

treatment caused the change in behavior. When there is baseline, treatment, and either

withdrawal or reversal , it is an ABA design. When the treatment is is given again after the

withdrawal or reversal, it is an ABAB design, which provides more complete proof of the effectiveness of treatment.

These designs should only be used in communication disorders research when treatment effects may be temporary or can be reversed, and may be ethically inadvisable.

Multiple Baseline Designs Another way to demonstrate the effect

of a treatment is the multiple baseline design.

Treatment effects are first demonstrated for one dependent variable, and then for two or more additional dependent variables.

The additional variables can be target behaviors, conditions, or subjects.

Multiple Baseline Designs: Behavior

In the multiple baseline design across behaviors, three or more target behaviors (e.g., the articulation of three different phonemes) are selected.

Baseline measures (A) are taken for all three behaviors.

Then treatment of the first target behavior (B) is begun, while baseline measures are continued for the other two behaviors.

When the treatment effect for B has reached the desired level, treatment of the second target behavior (C) is begun, while baseline measures are continued for the other behavior.

Multiple Baseline Designs Finally, when target behavior C reaches

the desired level, training of a third target behavior (D) can be begun, and continued until it reaches the desired level.

If the baseline behaviors change only when the appropriate treatment is introduced, there is evidence of a causal relationship.

Multiple Baseline Designs: Conditions

In the multiple baseline design across conditions, a single target behavior is trained in three or more different training conditions.

For example, the treatment of nonfluencies might be carried out in a research laboratory (B), at home (C), and in a public place (D).

The sequence of changes from baseline measures (A) to the three conditions follows exactly the same sequence as those in the multiple baseline design across behaviors.

Multiple Baseline Designs: Subjects

In the multiple baseline design across subjects, a baseline (A) is established for three or more subjects (B, C, and D), and then the treatment of a target behavior is introduced at different times for the subjects, following the sequence described for the multiple baseline design across behaviors.

The multiple baseline design across subjects is frequently used in communication disorders research.

Multiple Treatment Designs The effects of two or more different

treatments can be compared with several single subject designs.

In some cases the treatments are given one after the other, and in other cases the treatments are trained at the same time.

The simple ABAB design can be extended to ABABC, ABABAC, ABABCD, and so forth, where new treatments are introduced after training with the first treatment.

Multiple Treatment Designs Such designs are called multi-treatment

designs. The usual multi-treatment design involves a

preplanned comparison of methods. Baselines can be taken between each

treatment (ABACAD), and a theoretically optimal sequence of different treatments can be presented, such as training in imitating speech sounds followed by training in naming the speech sounds in words represented by pictures.

Multiple Treatment Designs The alternating treatment design, also

called the multiple schedules design, presents the treatments (usually only two) in each session in counterbalanced order, or in alternating sessions.

It is not necessary to take baseline measures because the treatment effects are compared, but it is advisable to take the baseline measures to demonstrate the magnitude of the effects.

Multiple Treatment Designs The difficulty with multiple treatment

designs in communication disorder research is assessing the possible carryover effects from one treatment to another in the treatment of nonreversible behaviors.

This difficulty may be overcome by using different target behaviors for each treatment, e.g., treatment B for one misarticulated phoneme and treatment C for a second misarticulated phoneme.

Generalization Designs In multiple baseline designs across behaviors

and conditions, generalization from one target behavior to another or from one condition to another is undesirable, because the baselines for the untrained behaviors or conditions will change.

In practical training studies, however, it is hoped that training effects will not be confined to the exact targets used in training, but will generalize to nontrained behaviors (e.g., a phoneme correctly articulated in a set of training words will be articulated correctly in non-trained words).

Generalization Designs There are several ways of assessing

generalization to nontrained behaviors. One simple method is to probe

nontrained behaviors during baseline, at intervals during training, and after training.

If training is done in a laboratory setting, it is important to assess generalization, that is, carryover, to a normal conversational setting.

Single-Subject Designs Single subject designs have some of the

advantages of group designs and some of the advantages of observational and case study designs.

They provide direct quantitative measures of the behaviors studied, averaged neither across subjects in groups nor across studies.

Experimental conditions are rigorously controlled to obtain information about causal relationships between independent and dependent variables.

Single-Subject Designs Large groups of subjects are not needed. Information is obtained for individuals

rather than groups. The information may have direct practical

applications. Single subject designs have their own

inherent limitations. Stable baselines may be difficult to

establish. If the treatment is not immediately

effective in changing the baseline behavior, it may be difficult to demonstrate causal relationships.

Single-Subject Designs Reversal designs cannot be used if

treatment effects do not or should not reverse.

Multiple baseline designs cannot be used if treatment effects generalize to nontreated behaviors.

Multiple treatment designs may yield ambiguous results if treatment effects carry over and if differences between treatment effects are not clear-cut.

Single-Subject Designs The very rigid specification of target

behaviors, treatments, and control procedures may make the treatment too artificial for direct application to clinical intervention.

The treatment may change the target behaviors only in the experimental condition and not in natural communication situations.

Finally, there is the problem of generalizing the results.

There is no way of predicting that all subjects of the same type will show the same treatment effects.