questionary design

RESEARCH DESIGN 2

Dr. Md. Zakir HossainAssistant Professor

Cross-Sectional DesignCross-sectional design is often called a survey

design(structured observation, content analysis, official

statistics and diaries)

More than one case At a single point in time Quantitative or quantifiable data Patterns of Association

2

Longitudinal designs

Panel Survey Cohort survey (e.g. The National Child

Development Study)

3

Qualitative Research Designs The historic origin for qualitative research comes from

anthropology, sociology, the humanities, and evaluation.

Narrative research is a design of inquiry from the humanities in which the researcher studies the lives of individuals and asks one or more individuals to provide stories about their lives (Riessman, 2008). This information is then often retold or restored by the researcher into a narrative chronology. Often, in the end, the narrative combines views from the participant’s life with those of the researcher’s life in a collaborative narrative (Clandinin & Connelly, 2000).

Qualitative Designs Phenomenological research is a design of inquiry coming

from philosophy and psychology in which the researcher describes the lived experiences of individuals about a phenomenon as described by participants. This description culminates in the essence of the experiences for several individuals who have all experienced the phenomenon.

Grounded theory is a design of inquiry from sociology in which the researcher derives a general, abstract theory of a process, action, or interaction grounded in the views of participants. This process involves using multiple stages of data collection and the refinement and interrelationship of categories of information (Charmaz, 2006; Corbin & Strauss, 2007).

Qualitative designs Ethnography is a design of inquiry coming from

anthropology and sociology in which the researcher studies the shared patterns of behaviours, language, and actions of an intact cultural group in a natural setting over a prolonged period of time. Data collection often involves observations and interviews.

Case studies are a design of inquiry found in many fields, especially evaluation, in which the researcher develops an in-depth analysis of a case, often a program, event, activity, process, or one or more individuals. Cases are bounded by time and activity, and researchers collect detailed information using a variety of data collection procedures over a sustained period of time (Stake, 1995; Yin, 2009, 2012).

Mixed methods design Mixed methods involves combining or integration of

qualitative and quantitative research and data in a research study. Qualitative data tends to be open-ended without predetermined responses while quantitative data usually includes closed-ended responses such as found on questionnaires or psychological instruments.

Triangulating data sources—a means for seeking convergence across qualitative and quantitative methods—was born (Jick, 1979). By the early 1990s, mixed methods turned toward the systematic convergence of quantitative and qualitative databases, and the idea of integration in different types of research designs emerged.

Mixed methods design Convergent parallel mixed methods is a form of mixed

methods design in which the researcher converges or merges quantitative and qualitative data in order to provide a comprehensive analysis of the research problem. In this design, the investigator typically collects both forms of data at roughly the same time and then integrates the information in the interpretation of the overall results

Explanatory sequential mixed methods is one in which the researcher first conducts quantitative research, analyzes the results and then builds on the results to explain them in more detail with qualitative research. It is considered explanatory because the initial quantitative data results are explained further with the qualitative data.

Mixed methods design

Exploratory sequential mixed methods is the reverse sequence from the explanatory sequential design. In the exploratory sequential approach the researcher first begins with a qualitative research phase and explores the views of participants. The data are then analyzed, and the information used to build into a second, quantitative phase. The qualitative phase may be used to build an instrument that best fits the sample under study, to identify appropriate instruments to use in the follow-up quantitative phase, or to specify variables that need to go into a follow-up quantitative study.

Mixed methods design

These basic models can then be used in more advanced mixed methods strategies. Transformative mixed methods is a design that uses a theoretical lens drawn from social justice or power as an overarching perspective within a design that contains both quantitative and qualitative data. The data in this form of study could be converged or it could be ordered sequentially with one building on the other. An embedded mixed methods design involves as well either the convergent or sequential use of data, but the core idea is that either quantitative or qualitative data is embedded within a larger design (e.g., an experiment) and the data sources play a supporting role in the overall design. A multiphase mixed methods design is common in the fields of evaluation and program interventions.

Summary

Questionnaire Design

Questionnaire

Definition – A set of questions designed to generate the information necessary to meet research objectives

Characteristics Elicits information from respondents Results can be tabulated Standardized across respondents Understandable to respondents

A good questionnaire must: provide the necessary information be considerate of respondents meet coding and data processing requirements

The Questionnaire Design Process

Determine SurveyObjectives and

Constraints

Determine DataCollection Methods

Evaluate theQuestionnaire and

Layout

EstablishQuestionnaire Flow

and Layout

Decide on QuestionWording

Determine QuestionResponse Format

Obtain Approvalfrom all Relevant

Parties

Pretest and Revise

Prepare Final Copy

Implementation

Step 1: Determine Survey Objectives

Translates research objectives into information requirements

Survey objectives should be spelled out as clearly and precisely as possible

Rule 1 - Let the research objectives dictate what questions to ask

Rule 2 - Avoid unnecessary questions

Step 2: Determine the Data Collection Method

Survey data can be gathered by variety of ways Personal Telephone Electronic Mail Self administered

Each method has a different impact on survey design

Step 3: Determine the Question Response Format

DICHOTOMOUS

MULTIPLE CHOICE

RATING SCALES

OPEN-ENDED

YES/NO

ONE/MANY RESPONSES

ODD/EVEN NUMBERS

FREE RESPONSE

DICHOTOMOUS QUESTIONS Good for clear answers; Yes/no questions are often better

rephrased as ‘to what extent’ or ‘how much’ types of question.

MULTIPLE CHOICE Need for a pilot to gather exhaustive

categories of response; Do not allow for range of response; If more than one response permitted

then each choice is a separate variable.

LIKERT SCALES Useful for measuring degrees of intensity of feeling; No assumption of equal intervals; No assumptions of matched intensity of feeling; No way of knowing if respondents are telling the truth; No way of knowing if there should be other categories or

items; Halo effect; Allows for different scaling and mid-points, e.g.:

(a) strongly disagree – neither agree nor disagree – strong agree;(b) not at all – a very great deal;

Central tendency; Ordinal data.

SEMANTIC DIFFERENTIAL SCALES A word and its semantic opposite, e.g.:

Approachable . . . unapproachable Generous . . . Mean Friendly . . . hostile

Same concerns as for Likert scales.

Ranking scale

Scaled-Response Questions Question format that permits measurement of the “intensity” of a

respondent’s answers Example of a scaled-response question:

Now that you have tried the new sugar-free Slurpee, would you say that you would… (Check one) definitely buy it probably buy it might or might not buy it probably not buy it definitely not buy it

Advantages Easier to answer Reduce measurement error by overcoming

respondent articulateness and possible interviewer bias

Easy to code and analyze More likely to respond for some personal data (e.g.

income, age)

Disadvantages Information is lost Answers may not be precise Not able to develop alternatives

Close-ended questionnaire

Open-ended questions are useful:

to interpret closed-ended responses when there are too many alternatives to list when alternatives vary between respondents

or are not clear when intensity of feeling is important for some sensitive issues (e.g. illness)

Open-ended questions

Advantages More information Wide range of responses Maybe more accurate description

Limitations Respondent articulateness can lead to error/variation Time consuming for respondent Coding and analysis is difficult

Open-ended questions

Rule 3: Use simple words e.g., not “marital status” but whether “married,”

“single” or “divorced” Rule 4: Use unambiguous words (words that are

easily and clearly understood) e.g. not “where do you usually buy school

supplies?”but “from which retailer(s) did you buy school supplies this year?”

“where” and “usually” are often ambiguous

Step 4: Question Wording

Rule 5: Avoid “double-barreled” questions (two questions in one) e.g., How satisfied are you with pay and

conditions in your job? Rule 6: Avoid leading questions (questions

that imply an answer) e.g. “Do you agree that the government

should cut taxes so that we can lead better lives?”

Question Wording

Rule 7: Avoid “loaded” questions (framing questions so that they are leading or emotionally charged)

e.g., “Do you think chemical companies are doing everything possible to control pollution?”

Rule 8: Avoid implicit assumptions or alternatives –

State them clearly e.g., “Do you favor a law mandating methanol-free

gasoline” [if it means an increase in gas prices]

Question Wording

Rule 9: Avoid burdensome questions (questions that tax memory or skills) e.g., “How many different brands of breakfast

cereal have you bought in the last twelve months?”

Rule 10: Clarify references (how the should respondent answer) e.g., “State your annual income” [personal income

or household income?]

Rule 11: Alternatives should be mutually exclusive, collectively exhaustive, and have reasonable range

e.g., What is your annual household income?__ $0 - $10,000__ $10,000 - $15,000__ $15,000 - $20,000__ $40,000 - $60,000

What is wrong with this question and how would you correct it?

Rule 12: Use appropriate phrasese.g., Are you: (1) amazingly happy, (2) middling

happy or (3) kind of unhappy (check one)

Are you: (1) very happy, (2) happy or (3) unhappy (check one)

Rule 13: Avoid order biase.g., should you start with (1) as “very happy” or

(1) as “unhappy?”

Question wording- Scaled Responses

Questions about borrowing money, personal hygiene, sexual activity, criminal history, etc. must be phrased carefully to minimize measurement error

Suggestions/Techniques - Not Rules Ask non-threateningly and mix with other questions

e.g., How many times each day do you brush your teeth?

Frame question to prompt more honest responsese.g., Many people find it difficult to brush their teeth

twice a day. How often do you brush your teeth?

Sensitive/Embracing Questions

Step 5: Questionnaire Flow and Layout Rule 14: Opening questions should be simple and

interesting – begin with open-ended question if warm-up is necessary

Rule 15: Use “funnel approach” – ask most general questions first, then increasingly specific questions

Rule 16: Questions should flow smoothly and logically

Step 5: Questionnaire Flow and Layout

Rule 17: Ask “screener” questions up front to assess respondent qualifications

Rule 18: Ask sensitive questions near the end

Rule 19: Ask for demographic information at the end

Rule 20: The questionnaire’s appearance should be attractive and professional

Step 6: Pretest and Revise

Rule 21: Always pretest your questionnairePretesting can be done by giving the questionnaire to a few friends, “experts”, and potential respondents (may be 5-10 persons total), asking them to fill out, and obtaining their feedback