Glossary of Terms

Achievement Test – measures knowledge and skills individuals have acquired (example: multiple choice test) Alternative Hypothesis – (H1) the alternative to the null hypothesis which is also called the directional research hypothesis Applied Research –in which researchers wish to apply findings directly to such practical decisions as whether to modify a program evaluation Aptitude Test – designed to predict some specific type of achievement (example: SATs) Auditor – the peer who provides the review; an outside expert Average – descriptive statistic; how the typical individual would have scored on an assessment Axial Coding – second step in grounded theory approach to data analysis; at this stage the transcripts of the interviews and any other data sources, are reexamined with the purpose of identifying relationships between the categories and themes identified during open coding; some important types of relationships that should be noted at this time are (1) temporal (X usually precedes Y in time), (2) causal (X caused participants to do Y), (3) associational (X and Y usually or always occur at about the same time but are not believed to be causally connected, (4) valence (participants have stronger emotional reactions to X than to Y), (5) spatial (X and Y occur in the same place or X and Y occur in different places) Baseline – initial observations constitute the baseline which determines the typical variation in the behavior of participants over time before any intervention Basic Research – researchers are attempting to understand underlying theories that explain behavior without necessarily looking for direct immediate applications – with experimental research Bell Curve – see normal curve Bias – exists whenever some members of a population have a greater chance of being selected for inclusion in a sample than other members of a population; freedom from bias is the most important characteristic of a sample Bimodal Distribution – has two high points – the points do not have to be equal in height; such a curve is most likely to emerge when human intervention or a rare event has changed the composition of a population; for example: if a civil war cost the lives of many young adults, the distribution of age would be bimodal with a dip in the middle; it is much less common in research Blind – when administering a placebo, researchers say they are using a blind procedure when they do not disclose to the participants whether they are receiving an active or inactive substance Case Study – usually only involves one individual or family; the emphasis is on obtaining through knowledge of an individual, sometimes over a long period of time; researchers do not limit questions as they would on a survey

Causal- Comparative Research – research in which researchers look to the past for the cause(s) of a current condition; it is used primarily when researchers are interested in causality, but it is not possible to conduct and experiment Causal-Comparative Study – (ex post facto study) the essential characteristics of this type of non-experimental study are (1) researchers observe and describe some current conditions (such as lung cancer) and (2) researchers look to the past to try to identify the possible cause(s) of the condition; they do not give treatments – only describe observations; have more potential pitfalls than experimental method, it is often the best researchers can do when attempting to explore causality Causal Relationship – just because a correlation between two variables is observed, it does not necessarily indicate that there is a causal relationship between the variables Cause-and-Effect Relationships – in which the independent variable is the possible cause and the dependent variable demonstrates the possible effect; it is the purpose of an experiment; a controlled experiment is needed in which different treatments are administered to the participants to determine cause and effect Census – a count or study of all members in a population; everyone is included so there is no base error Central Limit Theorem – says that the sampling distribution of means is normal in shape (i.e. forms a normal curve) Checking the Accuracy of a Transcription – helps to ensure the quality of the data as transcribers sometimes make clerical errors Chi-Square – (X2) used to test differences among frequencies; in one-tired chi-square participants are classified on only one way and in two-way chi-square participants are classified in two ways Cluster Sampling – researchers draw groups (or clusters) of participants instead of drawing individuals Code into Domains – the first step in the CQR method; this refers to segmenting the data into groups into groups according to the topics they cover Coefficient of Determination – (r2) useful when interpreting a Pearson r; simply square r to find the coefficient of determination; when converted to a percentage indicates how much variance on one variable is accounted for by the variance of the other Conceptual Definitions – the definition is perfectly adequate if a researcher merely wants to communicate the general topic of his or her research to other individuals Concurrent Validity Coefficient – a validity coefficient that is obtained by administering the test and collecting the criterion data at about the same time Confound – a source of confusion regarding the explanation for a given difference

Consensual Qualitative Research (CQR) – strives to have a team of researchers arrive at a consensus on the meaning of the data collected Constant Comparison – a key element throughout the analysis of data using the grounded theory approach; technically a term that refers to constantly comparing each new element of the data with all previous elements that have been coded in order to establish and refine categories Construct Validity – the type of validity that relies on subjective judgments and empirical data (i.e. data based on observations) Content Validity – to determine content validity of a measure, researchers make judgments on the appropriateness of its contents (it is essential validity for achievement tests) Control Group – a group of subjects closely resembling the treatment group in many demographic variables but not receiving the active medication or factor understudy and there by serving as a comparison group when treatment results are evaluated Core Category – final stages of grounded theory approach to analysis, qualitative researchers develop which is the main overarching category under which the other categories and subcategories belong; attempt to describe the process that leads to the relationship identified in the previous state of analysis Correlation – refers to the extent to which two variables are related across a group of participants Correlational Coefficient – range from 0.00 (no correlation between variables) to 1.00 (a perfect correlation); used to help determine validity Correlational Statistics – special subgroup of descriptive statistics; the purpose is to describe the relationship between two or more variables for one group of participants Correlational Research – researchers are interested in the degree of relationships among two or more quantitative variables i.e. GPA and college admission Criterion – in the empirical approach to validity, researchers make planned comparisons to see if a measure yields a score that relate to a criterion (criteria); set of ratings is called the criterion – it is the standard by which the test is being judged Criterion Related Validity – general term for concurrent validity coefficient and predicative validity coefficient Criterion-Referenced Tests (CRTs) – tests designed to measure the extent to which individual examinees have met performance standards (i.e. specific criteria) Critical Assessment – assess the quality of each research article that will be cited; assess the measurements and discuss the limitations of their studies Cronbach’s Alpha – an alternate to the split-half method for estimating internal consistency whose symbol is a; after the test has been administered, mathematical procedures are used to obtain the equivalent of the average of all possible split-half reliability coefficients

Cross-Analysis – third step in CQR; in this step the core ideas are grouped into categories based on similarities; this results in a higher level of generalization (i.e. the results are becoming less specific and more general) Curve – when there are many participants, the shape of a polygon becomes smoother and is referred to as a curve Data – the observations researchers make through their research; data is typically in the form of numbers Data Triangulation – one technique in qualitative research that is used to establish trustworthiness and dependability of their data is to use multiple sources for observing data on the research topic; typically two or more participants (such as employees and employers) are used to collect data on a research topic Debrief – debrief participants after their participation in the study to promote ethical values; it consists of reviewing the purpose(s) of the study and the procedure(s) used, and offering to share the results with the participants when it becomes available; includes reassurances that data will remain confidential Deduces – test hypothesis derived from an existing theory; deduces hypotheses that are consistent with the theory; failure to confirm a hypothesis calls theory (or part of it) into question, causing theorists to consider reformulating the theory Deductive Approach – the researcher is deducing from the literature possible explanations (i.e. hypothesis) to be tested – quantitative; approaching the research task with preconceived notions based on published theory and research Demand Characteristics – can also be a source of confounding; a demand characteristic is a cue that lets participants know the expected outcome of an experiment; this can be a problem if participants attempt to please the experimenter by reporting or acting in a way they believe is expected Demographics – background characteristics such as socioeconomic status and ethnicity used to set up experiments and studies Demographic Information – information that should be collected in research to help consumers of research “see” the participants; allows for the participants to be described in a research report Descriptive Statistics – summarizes data Dependent Variable – the response to an independent variable or treatment; the variable that is the constant; control group; the response or outcome variable; does not receive the treatment or physical manipulation Develop Core Ideas Within Domains – second step in the CQR process; this is done by writing short summaries (i.e. abstracts) that reduce the original ideas of participants to fewer words Deviations – subtracting the mean from each score produces the deviations from the mean; the deviations from the mean sum to zero

Diminishing Returns – increasing sample size produces diminishing returns; adding more individuals to a sample produces very little increases in the precision of results Direct Relationship – (positive relationship) – those who score high on one variable tend to score high on the other, and those who score low on one variable tend to score low on the other; one particular groups average is higher than the other groups Double Blind Experiment – in a double blind experiment neither the participants nor the individual dispensing the drug know which is the active drug and which is the placebo; this is done to prevent the possibility that the individual dispensing the drug will subtly communicate to the participants their status as either control or experimental participants Effect Size – effect size refers to the magnitude (i.e. size) of a difference when it is expressed on a standardized scale; the statistic d is one of the most popular statistics for describing the effect size of the difference between two means Empiricism – refers to using direct observation to obtain knowledge Empirical Approach – is based on observation of individuals or objects of interest Empirical Research – when researchers systematically use empirical approach to acquire knowledge; the planning of the who, whom, how, when, and under what circumstances to observe is made through the conception of research questions; what do you want to observe to answer the questions; it is planned in advance Equivalent Time-Samples Design – quasiexperiment; has only one group (or possibly only one participant); treatment conditions are alternated preferably on a random basis Ethnography – when the focus is on cultural issues, the research may be referred to as ethnography Everyday Observations – an application of the empirical approach; using observations to acquire knowledge; an example of empirical approach – is not planned in advance Examinees – sometimes used in reference to participants who have taken an examination (such as an achievement test for research on the validity of a test) Experiment – a study in which treatments are given to see how participants respond to them or if it changes their behavior; purpose is to determine cause-and-effect relationships i.e. try different detergents(the treatment) to see if clothes are cleaner (the response) than the old brand, a waiter may be more friendly (treatment) to see if it increases tips (response) Experimental Group – a group of subjects that are exposed to the variable of a control experiment; the group of participants in a clinical study who receives the actual drug or treatment being studied; see independent variable Experimental Research – in experimental research, treatments are given for a research purpose, such as treating some students with a hand on manipulative and others with a workbook approach in order to

determine which treatment causes the greatest achievement; researchers give treatment(s) in order to observe their effects External Validity – external validity of an experiment is concerned with ‘to whom and under what circumstances can the results be generalized?’ Face Validity – in this approach to validity, judgments are made on whether a measure appears to be valid on the face of it; in other words on superficial inspection, the measure appears to measure what it purports to measure Field Research – researchers who use observational methods often refer to their research as this which has historical roots in the field of anthropology Formative Evaluation – program evaluators collect information during the course of a program that assists in the process of modifying the program while it is being implemented; collecting this information is called formative evaluation; it has two prongs, information is collected on the progress of implementing a program and involves collecting information on the progress toward the ultimate goal Frequency – the number of participants or cases; the symbol is f; the symbol of number of participants is N Frequency Distribution – the shape of a distribution of a set of scores can be seen by examining a frequency distribution – a table that shows how many participants have each score; examination of the table indicates that most of the participants are near the middle of the distribution and that the participants are spread out on both sides of the middle with the frequencies tapering off Frequency Polygon – a figure (i.e. drawing) that shows how many participants have each score; shows clearer the distribution General Domain – CQR internal stability; domains that apply to all the participants Generalize – is it accurate to assume that the treatment administered to the experimental group will work as well for the population as it did in the sample Grounded Theory – researchers try to develop theories that explain events they have observed; researchers who practice qualitative research complete this; theory that is grounded in observation; often thought of as evolutionary – usually develop during the process of making observations, and it is regularly revised (i.e. it evolves) as new observations warrant; the term theory in grounded theory can be a bit misleading because it does not refer to an existing theory of human behavior – it refers to the inductive method of analysis that can lead to theories of behavior Hawthorne Effect – important source of confounding which can be thought of as the attention affect; to control for the Hawthorne Effect some researchers use three groups: (1) an experimental group, (2) a control group that receives attention, and (3) a control group that receives no special attention (named for factory experiment in which productivity increased when the lights were dimmed and when they higher because of the attention received)

Historical Research – information is examined in order to understand the past; it is an attempt to understand the dynamics of human history; researchers are able to develop theories that may explain historical events and patterns which then lead to a hypothesis which are evaluated in terms of additional historical data that are collected History – threat to internal validity; other environmental influences on the participants between the pretest and the posttest Hypothesis – when a researcher predicts the answer to a research question prior to conducting research; a prediction of the outcome of research Independent Variable – administered so that researchers can observe the possible changes in dependent variables; experimental group; the stimulus or input variable – given physical manipulation; more than one can exist Induces Theory – provide the observations and conclusions on which researchers can induce theory; develop theories to explain why events have happened they observed Inductive Approach – a qualitative approach – emphasize induction from the preliminary data that he or she collected Inferential Statistics – enable researchers to estimate the amount of error to allow for when analyzing the results from unbiased samples; tools that tell us how much confidence we can have when generalizing from a sample in a population; when we generalize from a sample to a population, we are inferring that the sample is representative of a population Inferential Tests – statistical techniques that can be used to test the truth of the null hypothesis Informed Consent – promotes ethical values; researchers inform the participants of (1) the general purpose of the research, (2) what will be done to them during the research, (3) what the potential benefit(s) to them might be, (4) what the potential for harm to them might be, and (5) the fact that they may withdraw at any time without penalty Input Variable – see Independent Variable Instrument – synonym for measure Intact Groups – previously existing groups (this is diagrammed by putting a dashed line between the symbols for the groups which indicates the groups were intact) Intelligence Test – designed to predict achievement in general, not any one specific type; the most popular intelligence tests (1) are culturally loaded and (2) measure knowledge and skills that can be acquired with instruction Internal Stability – examined in CQR by determining the extent to which each category was general, typical, or variant

Interobserver Agreement – sometimes called intercoder agreement; in an analysis of data the members of a research team should initially work independently (without consulting each other) and then compare the results of their analyses; to the extent that they agree, the results are dependable; QUALITATIVE Interobserver Reliability – in an analysis of data the members of a research team should initially work independently (without consulting each other) and then compare the results of their analyses; to the extent that they agree, the results are dependable; QUANTITATIVE Interview Protocol – consisting of written directions for conducting the interview as well as a standard set of predetermined questions to be asked of all participants Instrumentation – the heading for the section of a research report in which the measures are described; threat to internal validity – possible changes in the measurement procedure from the time it was used as a pretest to the time it was used as a posttest Internal Consistency – checks the consistency of the tests scores within the test itself; the split-half method belongs to a class of reliability estimates known as the internal consistency Internal Validity – it is concerned with this question ‘is the treatment in this particular case, responsible for the observed change(s)?’ threats to internal validity are potential explanations for the observed changes, that is they are possible explanations for the observed changes; they are possible explanations that become confounded with the treatment as an explanation for any observed changes; threats are controlled by using true experimental designs Interobserver Reliability – the researcher observes unobtrusively to classify those who pass or count the number of seconds from the time the customer enters a store to when they are greeted by a salesperson Interval – measure how much participants differ from each other, has an equal interval, Does NOT have an absolute zero i.e. if we are measuring intelligence we do not know the point which measures zero and therefore cannot have a zero point Interval Estimate – when a researcher reports the limits of a confidence interval it is called an interval estimate of the mean Inverse Relationship – those who score high on one variable tend to score low on the other John Henry Effect – this effect is a confound; this effect refers to the possibility that the control group might become aware of its ‘inferior’ status and respond by trying to outperform the experimental group; the researchers should try to conceal the control groups knowledge of its status whenever possible Likert-Type Scales – scales that have choices from “strongly agree” to “strongly disagree”; each statement should present a clear statement on a single topic Literature Review – when writing a research report for publication in a journal, it is typical to integrate the literature into the introduction – in theses and dissertations it is often presented in a separate

chapter immediately following the chapter that contains the introduction; it allows the researcher to cite relevant research Longitudinal Research – when researchers repeatedly measure traits of the participants over a period of time in order to trace developmental trends i.e. following a newborn through its first year to observe visual activity Major Headings – helps to organize the topic by topic description of relevant research Margin of Error – inferential statistic; reported as a warning to readers of research that random sampling may have produced errors, which should be considered when interpreting results; i.e. it says that Obama’s approval rating is 52% with a margin of error of + or – 2.3%. This means that the pollster is confident the true percentage for the population was within 2.3% of 52; the high probability that the results are off by at least a slim amount Mathematical Deduction – when we deduce a proof in mathematics based on certain assumptions and definitions; empirical approach Maturation – threat to internal validity; perhaps participants matured (e.g. became older, wiser, or smarter) during the period between the pretest and posttest and the increase is due to maturation and not to the treatment Mean – most frequently used average; to compute – sum the scores and divide by the number of scores; most common symbol are m1 or M; Some also use X; it is defined as the balance point in a distribution of scores – specifically it is the point around which all the deviations sum to zero; synonym is measure of central tendency Measure of Central Tendency – see average or mean Measures – term that serves as the heading for the section of the report where the measurement devises are described; generic term for ant type of measurement device (e.g. test, questionnaire, interview scale, etc) Member Checking – the dependability of the results can also be enhanced by member checking; this term is based on the idea that the participants are “members” of the research team Meta-analysis – a set of statistical methods for combining the results of previous studies; the prefix meta means occurring later and/or being later and more highly organized; meta-analysis is conducted on previous results and mathematically synthesizes them Methods Triangulation – one example qualitative research establish dependability and trustworthiness would be through two methods that were employed to collect the data from one group of participants; only one type of participant (such as parents) is used to provide data, but two or more methods are used to collect data Modified Replication – a replication with some major modification(s) such as examining a new population or using an improved measurement technique

Mortality – if some of the participants drop out of the experiment at midcourse, mortality is said to have occurred; internal validity threat – differential loss of participants from the groups to be compared Minor Subheadings – helps to organize the topic by topic description of relevant research Multiple Correlation Coefficient – uppercase italicized R is the symbol; it has the same basic characteristics as r; to determine the degree of relationship between a combination of the two predictors Multiple Treatment Interference – threat to external validity; threat occurs when a group of participants is given more than one treatment Multistage Random Sampling – for large scale studies; a researcher does the following (an example) (1) draw a sample of counties at random from all counties in a state, (2) draw a sample of voting precincts at random from all precincts in the counties previously selected, and (3) draw individual voters at random from all precincts that were sampled Needs Assessment – a non-experimental research in which researchers attempt to determine the practical needs of those who will be served by the program Negative Relationship – see inverse relationship Negative Skew – when the long tail points to the left, skewed to the left, the distribution is a negative skew; example: a large population was tested on the nursing skills in which they have been thoroughly trained – a large number of graduates with a high score on the PRAXIS and a long tail to the left showing the small number of nurses who did not perform well on the test Nominal – the lowest level of measurement, also called categorical; it is helpful to thinking of the level as the naming level because names (i.e. words) are used instead of numbers; i.e. naming the gender, where they reside, religious affiliation of the participants Nonequivalent Control Group Design – widely used quasiexperiment design that has two intact groups (not assigned at random as indicated by the dashed line) Nondirectional Hypothesis – the researcher might hypothesize that there is a difference between two groups but there is insufficient information to hypothesize which group is higher – they are hypothesizing that there is a difference (that the populations of the two groups are not equal) but he or she is not willing to speculate on the direction of the difference Non-Experimental Study – aka descriptive study; defined as a study in which observations are made to determine the status of what exists at a given point in time without the administering of treatments; researchers do not give treatments, rather they observe participants in order to describe them as they naturally exist without experimental treatments Nonparticipant Observation – the qualitative researcher observes individuals as an outsider, for instance, sitting at the back of a classroom to observe student/teacher interactions; a particular concern with this type of observation is that the participants’ behavior may change from its normal course because they know they are being observed

Normal Curve – the most important shape which is also called the bell curve; it is important because it is a shape often found in nature and is used as the basis for a number of inferential statistics Norm Referenced Tests – tests designed to facilitate a comparison of an individual’s performance with that of a norm group One-Group Pretest-Posttest Design- changes from pretest to posttest may be attributable to history, maturation, instrumentation, testing, and if the participants were selected on the basis of extreme scores, statistical regression One-Shot Case Study – one group is given a treatment followed by a test Open Coding – the first step in grounded theory; in this stage the transcripts of the interviews are examined for distinct, separate segments (such as the ideas or experiences of the participants), which are identified by type and “coded” with individual names; preliminary notes on any overarching themes noticed in the data should also be made at this point Operational Definition – redefining a variable in terms of physical steps is called operationalizing a variable; can be operational without being adequate; needs to be fully multidimensional; never completely operational definition can exist Ordinal – measurements put participants in rank order from high to low but does not indicate how much higher or lower one participant is in relation to another; i.e. three shampoos are ranked according to customers’ preferences, participants are ranked according to height- the tallest given one, the next tallest a 2, and so on Parallel Forms Reliability – some published tests come in two parallel (or equivalent) forms that are designed to be interchangeable; they have different items that cover the same content; the two tests are administered a couple weeks apart and then the results are correlated Parameters – summarized results (such as averages); yielded by a population Participant – consists of all members of a group in which a researcher has an interest; used when individuals have consented; implies that the individuals being studied have freely consented to participate in research; predominant term used in research Participant Observation – the researcher becomes (or already is) a member of the group being researcher and, thus, makes his or her observations as an insider Pearson r – full name is Pearson product moment correlation coefficient; the symbol is a lowercased italicized r; it describes the relationship between two variables; the basic properties of the Pearson r; (1) it can range only from -1.00 to 1.00, (2) -1.00 indicates a perfect inverse relationship which is the strongest possible inverse relationship, (3) 1.00 indicates a perfect direct relationship which is the strongest possible direct relationship, (4) 0.00 indicates the complete absence of a relationship, (5) the closer a value is to 0.00 the weaker the relationship, and (6) the closer the value is to -1.00 or 1.00 the stronger the relationship; it is not a proportion

Peer Review – the use of an outside expert can also help to ensure the quality of the research; a researcher’s peer (such as another experienced qualitative researcher) can examine the process used to collect data and the resulting data and the conclusions, and he or she can then provide feedback to the researcher Percentage – symbol is %; indicates the number per hundred who have certain characteristics; to calculate use division – divide the smaller number by the total and then multiply by 100; helpful when comparing two or more groups of different sizes Phenomenological Approach – examining perceptions is known as a phenomenological approach to acquire knowledge Physically Manipulate – to physically administer treatments Pilot Studies – studies designed to obtain preliminary information on how new treatments and measures work Placebo - researchers use a control group that is given a placebo to control for the placebo effect; in drug studies for instance a placebo could be a pull that contains inert ingredients like a sugar pill which provides a treatment for the control group Placebo Effect - the placebo effect refers to the tendency of individuals to improve or at least feel they are improving simply because they know they are being treated; potential confounding effect Point Estimate – when a researcher reports a single value as an estimate of a population mean based on a sample, it is called a point estimate of the mean Population – including the entire group in an experiment or survey, the who portion of the empirical research i.e. it could be all mentally ill patients in a hospital ward; all public school teachers in Boston Population Parameters – a quantity or statistical measure that, for a given population, is fixed and that

is used as the value of a variable in some general distribution or frequency function to make it

descriptive of that population

Positive Relationship - see direct relationship Positive skew – skewed to the right, the curve drops off dramatically to the right forming a long tail pointing to the right (on a number line – positive numbers, the higher numbers, are on the left); example: if the plot distribution of income for a large population in all likelihood you will find a positive skew, it indicates that there are a large number of people with relatively low incomes this the curve is high on the left, the long tail is created by a small number of people having large incomes Posttest-Only Randomized Control Group Design – conducting an experiment without a pretest Precision – the extent to which the same results would be obtained if another random sample were drawn from the same population; to increase precision then you increase sample size; technical term for discussing the magnitude of sampling errors; increase in sample size increases precision

Predictive Validity – because the purpose of an employment test is to predict the success on the job, the most appropriate test of validity is that of predictive validity which poses the question: to what extent does the test predict the outcome it is supposed to predict Preexperimental Designs – designs that are of limited value for investigating cause and effect relationships because of their poor internal validity (one group pretest-posttest design, one shot case study, and static group comparison design) Pretest-Posttest Randomized Control Group Design – classic design for exploring cause and effect relationships; by assigning participants at random to groups, researchers are assured that there are no biases in the assignment; the two groups are equal in all ways except the random differences Pretest Sensitization (Reactive Effect of Testing) – changes observed in the experimental group may be the result of a combination of the pretest and the treatment Program Evaluation – conducted through evaluation research; a report on the implementation and effectiveness of the programs for which an organization is providing funding Proportion – a proportion is part of one (1); i.e. .246 is the proportion and 24 is the percent Purposive Sampling – when researcher use this method, they purposively select individuals who they believe will be good sources of information Purposive Criterion Sampling – a sampling technique when there are a number of criteria to be applied in the selection of a sample Qualitative Research – results are presented as discussions of trends and/or themes based on words, not statistics; deemphasizes the previous research; prefer measures that yield words that are not easily reduced to numbers; select small number of participants; purposeful sample; make adjustments while conducting the research; limit conclusions to only the individuals who were directly studied; researchers gather data that must be analyzed through the use of informed judgment to identify major and minor themes expressed by participants; most published research is qualitative through semi-structured interviews in which there is a core list of questions from which the interviewers may deviate as needed to obtain in-depth information Quantitative Research – presented as quantities and numbers (statistics); use literature as the basis for planning research; prefer measures that yield data reduced to numbers; select large samples for participants; random sample; does not make adjustments while conducting the research; generalize the results; a distinctive feature of quantitative research is that researchers gather data in such a way that the data are easy to quantify, allowing for statistical analysis Quasi-Experimental Designs – the intermediate value for exploring cause and effect relationships; it is in between the true experiment and preexperimental design (nonequivalent control group design and equivalent time samples design are examples) Random Cluster Sampling – all members of a population must belong to a cluster (i.e. an existing group) and existing groups of participants are drawn

Random Sampling – used to eliminate bas; a method of selecting a sample (random sample) from a statistical population in such a way that every possible sample that could be selected has a predetermined probability of being selected Range – descriptive statistic; the highest to lowest scores on an assessment which would indicate how much the scores vary Ratio – measure how much participants differ from each other, has an absolute zero point on its scale; the zero point on a tape measure for height or scale for weight; only scale which is appropriate to compute ratios Reactive Effects of Experimental Arrangements – threat to external validity; reminds researchers that if the experimental setting is different from the natural setting in which the population usually operates, the effects that are observed in the experimental settings may not generalize to the natural setting Reactive Effect of Testing (Pretest Sensitization) - changes observed in the experimental group may be the result of a combination of the pretest and the treatment; threat to external validity refers to the possibility that the pretest might influence how the participants respond to the experimental treatment Researcher Triangulation – reduces the possibility that the results of qualitative research represent only the idiosyncratic views of one individual researcher; an important technique to assure the quality of qualitative research is to form a research with each member of the team participating in the collection and analysis of data Reinforcement Theory – anything that increases the frequency of a response from an animal or individual; a unified set of principles that helps explain why certain behaviors increase in their frequency Reliability – a test is said to be reliable if it yields consistent results; reliability is consistency Reliability Coefficient – when researchers use correlation coefficients to describe reliability; when studying interobserver reliability, the researchers usually obtain the two measurements at the same time Reliance on Authority – relying on a dictator’s pronouncements as a source of knowledge; empirical approach Replicate - recreate a study that has already been published Replication – instead of striving for complete operational definitions, researchers try to produce definitions that are adequate to permit replication in all important respects by another researcher; a replication is an attempt to confirm the results of a study by conducting it again Research Hypothesis – a researcher’s ‘expectation’ – the personal hypothesis Research Questions – at the heart of all empirical research; developed on a theory or hunch Respondents – most frequently used when individuals respond to a survey such as a political poll

Reverse Scoring – if five points are awarded for strongly agreeing to a positive statement toward school, a researcher would ward five points for strongly disagreeing with a negative statement toward school; it is a good idea to provide some positive and some negative statements and score them according to whether the associated statements are positive or negative Sample – the subset of a population; used when a population is to large in its entirety; important to know how to draw a sample from a population Sample Size – important to drawing a sample but freedom from bias is first Samples of Convenience – (Accidental Samples) – biased; in a study where there are increased odds that some members of a population would be selected while reducing the odds that other members would be selected Sampling Distribution of Means – the hypothetical researcher would then have a very large number of

means which would create a sampling distribution of means

Sampling Errors – random errors, caused by random sampling, produced sampling errors; in statistics,

sampling error or estimation error is the error caused by observing a sample instead of the whole

population; an unbiased sample may still contain these errors

An estimate of a quantity of interest, such as an average or percentage, will generally be subject to sample-to-sample variation. These variations in the possible sample values of a statistic can theoretically be expressed as sampling errors, although in practice the exact sampling error is typically unknown. Sampling error also refers more broadly to this phenomenon of random sampling variation.

The likely size of the sampling error can generally be controlled by taking a large enough random sample from the population, although the cost of doing this may be prohibitive; see sample size and statistical power for more detail. If the observations are collected from a random sample, statistical theory provides probabilistic estimates of the likely size of the sampling error for a particular statistic or estimator. These are often expressed in terms of its standard error.

Saturation – as data is collected the qualitative researcher conducts a preliminary, informal analysis, noting the major and minor themes that are emerging; at the point at which several additional participants fail to respond with new information that leads to the identification of additional themes, the researcher might conclude that the data collection process has become saturated; it may determine the final sample size

Scales of Measurement – (Scales of measurement) help researchers to determine what type of statistical analysis is appropriate for a given set of data; NOIR – nominal, ordinal, interval, ratio Scattergram – also known as a Scatter Diagram; is a statistical figure (i.e. drawing) that illustrates the correlation between two variables; when dots form a pattern that goes from lower left to the upper right, the relationship is direct; when the dots form a pattern that goes from the upper left to lower right, the relationship is inverse; the dots perfectly follow a single straight line the relationship is perfect; the greater the amount of scatter around the line, the weaker the relationship

Selection – threat to internal validity; notice that when researchers do not assign participants to the two groups at random, there is a very strong possibility that the two groups are not initially the same in all important respects; it can interact with all the other threats to internal validity Selection Bias – if a sample is biased a researcher’s ability to generalize to a population is greatly limited – no generalizations should be made in this case; it is a threat to external validity Selection-History Interaction – the selection of participants for the two groups was not at random, they may be systematically subjected to different life experiences Selection-Maturation Interaction – perhaps the two groups, on average, were at somewhat different developmental stages at the time of the pretest which would have led to different rates of maturation in the two groups which could affect self-concept Self-Disclosure – refers to considering the research problem in relation to the interviewer’s background and attitudes before conducting the interviews; can help an interviewer achieve unbiased attitude Self-Selection Bias – (Volunteerism) – those who do have the chance to be included in the study but refuse to participate; presumed to create an additional source of bias because those who decide not to participate have no chance to be included; many who volunteer tend to be highly educated And from a higher SES than counterparts Semistructured Interviews – are by far the most widely used type of measure for collecting data for qualitative research; typically they are face-to-face and often recorded Simple Random Sampling – each member of a population is given an equal chance of being selected; example: put the names of all members of a population on slips of paper, thoroughly mix the slips and have a blindfolded assistant selected the number of desired for the sample; identifies an unbiased sample Skewed Distribution – a distribution that has some extreme scores at one end but not the other is called a skewed distribution; the mean is almost always inappropriate for describing a highly skewed distribution Snowball Sampling – can be used when attempting to locate participants who are hard to find; a researcher initially needs to find one; technique is based on trust Solomon Randomized Four Group Design – a combination of the pretest-posttest randomized control group design and posttest only randomized control group design; it has four rows of symbols and four groups- the first two rows are the same as the first design and the bottom two are the same as the second design; in effect it is two experiments conducted at the same time Split-Half Reliability – a researcher administers a test but scores the items in the test as though they consisted of two separate tests, typically it is done with an odd-even split; all the even items are scored and then the odd numbers are scores and then two scores result is a split-half reliability coefficient

Stability Check – in CQR there may be an external stability check which can be done by examining data in addition to the interview transcripts (perhaps eyewitness accounts or physical evidence) if it is available Standard Error of the Mean (SMM) – the standard deviation of the sampling distribution is known as the standard error of the mean, which is a useful statistic; type of standard deviation Static-Group Comparison Design – this has two groups, but participants are not assigned to the groups at random Statistical Regression – threat to internal validity; occurs only if participants are selected on the basis of their extreme scores Statistics – yielded by a sample; the summarized results Stimulus Variable – see Independent Variable Stratified Random Sampling - usually superior to simple random sampling; draw participants at random separately from each stratum (the population is first divided into strata that are believed to be relevant to the variable(s) being studied); the same percentage is drawn from each stratum; the purpose of stratifying is to obtain a single sample that is representative of the population; does not eliminate all sampling errors Strict Replication – researchers try to mimic the original study in all important respects because the purpose of a strict replication is to see if the same types of results as those of the original study are obtained Subjects – traditional term for the individuals being studied; used as an appropriate term for those who have not consented to be studied (observing teenagers in a mall to study traffic flow to stores) Summative Evaluation – when evaluators collect information about participants’ attainment of the ultimate goals at the end of a program; contains information about the final or long-term benefits of the program for its ultimate clients; often involves a comparison with a control group Survey – (or poll) one of the most common types of nonexperimental studies; in which participants are interviewed, questioned, or otherwise observed so that their attitudes, beliefs, and behaviors as they exist without experimental interventions are determined; the purpose of the survey is to describe the attitudes, beliefs, and behaviors of a population, study the sample, then make inferences to the population from the sample data Synthesis – providing a whole picture of what is known and what is not known as well as an attempt to show how diverse pieces of information fit together and make sense Systematic Sampling – in this type of sampling, every nth individual is selected; the number n can be any number; potential problem is if the order has been arranged in a particular way Table of Random Numbers – it may be used to draw a random sample; it is more efficient than writing all the names on paper and pulling the sample form hat; in this table there is no sequence to the

numbers; each number appears the same number of times; everyone is assigned a number name; flip to any page in a book of random numbers and put your finger on the page without looking to determine the starting point Team of Researchers with a Diverse Background – diversity in their research team helps to provide a comprehensive view of the data obtained in interviews on age and gender discrimination in public employment Testing – threat to internal validity; defined as the effects of the pretest on the performance exhibited on the posttest Test-Retest Reliability – researchers measure at two different points in time The How – to construct new measuring instruments or select instruments that have been developed by others; how to draw an adequate sample; i.e. researchers might review existing multiple choice tests and select those most valid for answering their research question The Whom (or What) – to observe The When – when the observations will be made; timing and circumstances of the observations may affect the results Theory – a unified explanation for discrete observations that might otherwise be viewed as unrelated or contradictory Threats to External Validity – extent to which the experiment is subject to what researchers call the threats to external validity; examples include selection bias, reactive effects of experimental arrangements, and reactive effect of testing Threats to Internal Validity – depending on the designs of the experiment there may be explanations for the changes other than the treatment; examples are history, maturation, instrumentation, testing, statistical regression, and selection Topic-by-Topic – description of relevant research; provide major and minor subheadings to guide readers through a long literature review True Experiment - when the participants are divided at random (such as through the drawing of names from a hat to determine who will be in the experimental group and who will be in the control group); participants are assigned at random to experimental and control conditions, because random assignment has no bias (or favorites) both the experimental and control groups are equally likely to experience the same environmental events (have the same history), mature at the same rates, drop out at the same rates, and so on. True Experimental Design – are characterized by random assignment to treatments t-Test – developed by William Gosset to test the difference between two sample means to determine statistical significance; when a t test yields a low probability that a null hypothesis is correct, researchers usually reject the null hypothesis; larger the sample the more likely the null hypothesis will be rejected;

the larger the observed difference between the two means the more likely the null hypothesis will be rejected; the smaller the variance the more likely the null hypothesis will be rejected Type 1 Error – the error of rejecting the null hypothesis when it is correct; a synonym for rejecting the null hypothesis is declaring the results statistically significant Type 2 Error – the error of failing to reject the null hypothesis when it is false Typical Domain - CQR internal stability; domains that apply to half or more of the participants Unbiased Sample – a researcher selected names using simple random sampling and obtained the cooperation of everyone leads to this sample – all are included; preferred by researchers for inferential statistics and the amount of sampling error obtained from unbiased samples tend to be small when large samples are used Under Particular Circumstances – observations made under particular circumstances; i.e. will all observations be made at the mall or the park Variability – needs to be considered in a sample size; if there is little variability (i.e. the population is very homogeneous) researchers can obtain accurate results from a small sample – if the sample is very heterogeneous a researcher needs a large sample Validity – accuracy (reliability – consistency); researchers say that a measure is valid to the extent that it measures what it is designed to measure and accurately perform the function(s) it is purported to perform Validity Coefficient – it is a correlation coefficient used to express validity; it is used to look at a test’s predictive validity; only the basics will be considered – 0.00 to 1.00 Variant Domain - CQR internal stability; domain that applies to less than half but more than two of the participants