What is Research Design

Research design refers to the basic strategy of theresearch by defining an action plan that proceeds fromthe primary research questions up to the conclusionsand the justification behind it (Oppenheim, 1992).

Yin (2003), “a research design is an action plan forgetting from here to there, where here is defined as theinitial set of questions to be answered, and there issome set of conclusion” (p, 20).

The Analogy of RD

How do we build a house?

Do we buy the materials first, select fittings or setting artificial dates without first establishing the type of the house we want to build?

Different types of house will require a very different building strategies, materials, permits and timeliness.

Only when the type of the house (design) is specified, only then we know what materials, fittings etc that are appropriate and suitable.

Research Methodology

Research methodology is a strategy or plan of action that links methods to outcomes. It directs our choice and use of methods (eg, experimental research, survey research, ethnography, etc).

Research methods – techniques and procedures do we propose to use (e.g. questionnaire, interview, focus group, etc).

Not all methods can be applied to all research questions, so the choice of method is limited by the area of research that you wish to explore.

Research Design

Research design involves determining how your chosen method will be applied to answer your research question.

The design of your study can be thought of as a blueprint detailing what will be done and how this will be accomplished.

Key aspects of research design include: research methodology; participant/sample collection and assignment (if different conditions are being explored); and data collection procedures and instruments.

Exploratory Research

Designed to generate basic knowledge, clarify relevant issues uncover variables associated with a problem, uncover information needs, and/or define alternatives for addressing research objectives.

Not intended to test specific research hypotheses.

A very flexible, open-ended process.

Typical exploratory design – focus group, in-depth interview, case studies, videotaped/photograph

Descriptive Research (who, what, where, how)

Designed to provide further insight into the research problem by describing the variables of interest. For example:Who is likely to be most satisfied?How much investment is required?Which brands is most preferred?

DR usually structured and specifically designed to measure characteristics described in RQs.

Can be used for profiling, defining, segmentation, estimating, predicting, and examining associative relationships.

Explanatory - Hypothesis Testing (Analytical and Predictive)

Studies that engage in hypotheses testing usually explain the nature of certain relationships, or establish the differences among groups or the independence of two or more factors in a situation.

Hypotheses testing is undertaken to explain the variance in the dependent variable or to predict organizational outcomes.

Designed to provide information on potential cause-and-effect relationships.

Qualitative The inquirer makes knowledge claims based primarily on

constructivist perspectives, i.e. the multiple meanings of individual experiences, meanings

socially and historically constructed, with a intent of developing a theory or pattern) or participatory perspectives (i.e, political issue-oriented, collaborative or change oriented).

Other strategies – narratives, phenomenologies, ethnographies, grounded theory studies, or case studies.

The researcher collect open-ended, emerging data with the primary intent of developing themes from the data.

Quantitative

Investigator primarily uses post-positivist claims for developing knowledge i.e, cause and effect thinking, reduction to specific variables

and hypotheses and questions, use of measurement and observation, and test of theories

Employ strategies of inquiry such as experiments and surveys

Collect data on predetermined instruments that yield statistical data

Mixed-Method

The researcher tends to base knowledge claims on pragmatic grounds (eg, consequence-oriented, problem-centred and pluralistic)

It employs strategies of inquiry that involve collecting data either simultaneously or sequentially to best understand research problems.

Data collection involve numeric information (on instruments) as well as text information (on interview).

The final database represents both qualitative and quantitative information.

Type of Research (Salkind, 2011)

Time Horizon

Cross-sectional design Collects data at one time

Longitudinal Designs Takes place over time with two or more data collectionsTrend design Each survey collects data on the same items or variables

with a new independent sample of the same target population

Panel studies Each survey collects data at different times from the

same respondents

Prior To Research Design

Variable: what is measured or varied. An attribute or characteristic of a person (or object) that can change from person to person.

① Independent Variable: a variable that is manipulated, measured or selected by the researcher in order to observe its relation to the subject's "response” on another variable.

② Dependent Variable: the variable that is observed and measured in response to an independent variable.

Instrumentation

Introduction: One of the most important components of a research design is the research instruments because they gather or collect data or information.

These research instruments or tools are ways of gathering data. Without them, data would be impossible to put in hand.

QUESTIONAIRE

QUESTIONAIRE The most common instrument or tool of research for obtaining the data beyond the physical reach of the observer which, for ex. May be sent to human beings who are thousands of miles away or just around the corner.

Two Forms of Questionnaire :1. Closed form / Closed-ended 2. Open form / Open-ended

AdvantagesAdvantages Facilitates data gathering

Is easy to test data for reliability and validity

Is less time-consuming than interview and observation

Preserves the anonymity and confidentiality of the respondents’ reactions and answers

Disadvantages Printing and mailing are costly Response rate maybe low Respondents may provide only socially acceptable answers There is less chance to clarify ambiguous answer Respondents must be literate and with no physical handicaps Rate of retrieval can be low because retrieval itself is difficult

Questions to Ask Before Doing Survey Research

Do you have a clear hypothesis?

Do your questions focus on that hypothesis?

Will participants answers provide accurate answers to your questions?*

To whom will your results apply?*

Planning a Survey Deciding on a research question

Choosing the format of your questions*

Choosing the format of your interview--if you use an interview*

Editing your questions*

Sequencing your questions*

Refining your survey instrument*

Choosing a sampling strategy*

Questionnaire DesignSpecify the Information Needed

Design the Question to Overcome the Respondent’s Inability and Unwillingness to Answer

Determine the Content of Individual Questions

Decide the Question Structure

Determine the Question Wording

Arrange the Questions in Proper Order

Reproduce the Questionnaire

Specify the Type of Interviewing Method

Identify the Form and Layout

Eliminate Bugs by Pre-testing

Questionnaire Design

Specify Information needed It is the first step for questionnaire design as it helps in

clearly understanding needs

Type of Interviewing method Mail Questionnaire Telephone Questionnaire Personal Questionnaire Electronic Questionnaire

(very similar to mail questionnaire)

Questionnaire Design (Cont’d)

Individual Question Content Is the question necessary? Are several questions needed instead of one?

E.g.:-

“Do you think Coca-Cola is a tasty and refreshing soft drink??”

“Do you think Coca-Cola is a tasty soft drink?”

“Do you think Coca-Cola is a refreshing soft drink?”

Questionnaire Design (Cont’d)

Overcoming inability to answer Is the Respondent informed?

e.g.:- A husband may not be aware of monthly expenses for groceries and department store if wife make purchase or vice versa Can the Respondent Remember? Can the Respondent articulate?

e.g.:- If asked to describe the atmosphere of the department store.

Questionnaire Design (Cont’d)

Overcome Unwillingness to Answer Effort required of the Respondents Context Legitimate Purpose Sensitive information Increase the willingness of Respondents

Choosing Question Structure Unstructured Question Structured Question

Choosing Question Wording

define the issue

use ordinary words

use unambiguous words

avoid leading questions

avoid implicit alternatives

avoid implicit assumptions

avoid generalizations and estimates

use positive and negative statements

Define the issueThe W's Defining the Question

Who The RespondentIt is not clear whether this question relates to the individual respondent or the respondent's total household.

What The Brand of ShampooIt is unclear how the respondent is to answer this question if more than one brand is used.

When UnclearThe time frame is not specified in this question. The respondent could interpret it as meaning the shampoo used this morning, this week, or over the past year.

Where At home, at the gym, on the road?

Define the issue

A question should clearly define the issue being addressed

Which brand of shampoo do you use??

Which brand or brands of shampoo have you personally used at home during the last month? In case of more than one brand, please list all the brands that apply?

Use ordinary Words

ordinary words should be used in questionnaire vocabulary level should match the respondents vocabulary

Do you think the distribution of soft drink is adequate??

Do you think soft drinks are readily available when you want them to buy?

Use Unambiguous WordsWords in questionnaire should have a single meaning known

to respondents

In a typical month how often do you shop in department store??

Never….. Occassional ….. Sometimes…..

Often….. Regularly…..

In a typical month how often do you shop in department store?

Less than once….. 1 or 2 times…. 3 or 4 times….

More than 4 times…..

Avoid leading questions

A leading question clues, the respondent to answer in a certain way

Do you think patriotic Americans should buy imported automobiles when that would put American labor out of work??

Do you think Americans should buy imported automobile?

Avoid implicit alternatives

Alternative not explicitly expressed is an implicit alternative

Do you like to fly when travelling short distances??

Do you like to fly when travelling short distances or would you rather drive?

Avoid implicit assumptions

Questions should not be worded so that the answer is dependent upon implicit assumption

Are you in favor of a balanced budget??

Are you in favor of a balanced budget if it would result in an increase in the personal income tax?

Avoid Generalizations and Estimates

Questions should be specific not general

Respondent does not make generalization or compute science

What is the annual per capita expenditure on groceries in your household??

What is the monthly expenditure on groceries in your household and How many members are their in your household?

Determining the order of questions

Opening question

Can be crucial in gaining the confidence, should be interesting, simple, and non threatening

Type of Information

As a general guideline, basic information should be obtained first, followed by classification, and, finally, identification information

Difficult Questions

Difficult questions which are sensitive, embarrassing, complex, or dull, should be placed late in the sequence

Funnel Approach

Questions asked in sequence can influence the response to subsequent question

So questions asked from general to specific

What considerations are important to you in selecting a department store??

In selecting a department store how important is convenience or location?

Form and Layout

format, spacing and positioning questions have significant effect on results

should be numbered serially

2000 population of census reveals questions at the top received more importance than the bottom

Instructions in red made little difference that they made the questionnaire appear more complicated

Reproduction of Questionnaire

Paper quality can reflect on project

Vertical response columns

Tendency to crowd should be avoided

Instructions to be placed close to the question

Motto easy to read and answer

Should be large and clear

Pilot the research instrument to:

• identify any problems such as the wording of questions, the length of the interview, and whether the research instrument has been compiled in a logical fashion

• test the interviewers' / focus group moderators’

• ability to administer the research instrument and indicate whether further training is required.

Example of closed-ended questions

1. What is your gender1. Male 2. Female

2. What is your ethnic identification?1. African American2. American Indian3. Asian/Pacific Islander4. European American5. Mexican American/Latino6. Multicultural or multi-racial7. Other ____________________

More examples3. What is your age?

1. 18 or under2. 19-293. 30-494. 50 – 645. Over 65

4. How satisfied are you with Social Work 292?1. Very satisfied2. Satisfied3. Neutral4. Unsatisfied5. Very Unsatisfied

INTERVIEW

INTERVIEW It is in a sense of an oral questionnaire. Instead of writing the response, the interviewee gives the needed information orally and face-to-face. With a skillful interviewer, the interview is often superior to other data-gathering device.

The purposes of interview are : to verify information gathered from written sources to clarify points of information to update information and to collect data

Avoid to do: Too structured - does not allow interviewees to put

forward their own point of view and experiences Too unstructured - may result in too many data that are

irrelevant to the study Complex and technical languageMultiple questionso Ex:'What do you feel about the risks of sharing syringes

now as opposed to five years ago?‘ Leading questionso Ex:'Why is there so much plagiarism nowadays?'o Ex:Should be: 'Is there any plagiarism issue in this

faculty?' or 'Tell me about plagiarism around here.'

Case StudyMany well-known case study researchers such as Robert E. Stake (1995), Helen Simons (1980), and Robert K. Yin (1984) have written about case study research and suggested techniques for organizing and conducting the research successfully.

Steps that should be used in case study: 1) Determine and define the research questions 2) Select the cases and determine data gathering and analysis

techniques 3) Prepare to collect the data 4) Collect data in the field 5) Evaluate and analyze the data 6) Prepare the report

Step 1. Determine and Define the Research Questions

The researcher establishes the focus of the study by forming questions about the situation or problem to be studied and determining a purpose for the study.

The research object in a case study is often a program, an entity, a person, or a group of people.

The researcher investigates the object of the case study in depth using a variety of data gathering methods to produce evidence that leads to understanding of the case and answers the research questions.

Case study research generally answers one or more questions which begin with "how" or "why."

To assist in targeting and formulating the questions, researchers conduct a literature review.

This literature review establishes what research has been previously conducted and leads to refined, insightful questions about the problem.

Step 2. Select the Cases and Determine Data

Gathering and Analysis Techniques Determines what approaches to use in selecting single or multiple real-life cases

to examine in depth and which instruments and data gathering approaches to use.

When using multiple cases, each case is treated as a single case.

carefully select cases and carefully examine the choices available from among many research tools available in order to increase the validity of the study.

Careful discrimination (distinction) at the point of selection its helps establish boundaries around the case.

A key strength of the case study method involves using multiple sources and techniques in the data gathering process.

Determines in advance what evidence to gather and what analysis techniques to use with the data to answer the research questions.

Data gathered is normally largely qualitative, but it may also be quantitative. Tools to collect data can include surveys, interviews, documentation review,

observation, and even the collection of physical artifacts.

Entire the design phase, researchers must ensure that the study is well constructed to ensure construct validity, internal validity, external validity, and reliability.

The researcher attempts to establish a chain of evidence forward and backward.

Step 3. Prepare to Collect the Data Because case study research generates a large amount of data from multiple

sources, systematic organization of the data is important to prevent the researcher from becoming overwhelmed by the amount of data and to prevent the researcher from losing sight of the original research purpose and questions.

Researchers prepare databases to assist with categorizing, sorting, storing, and retrieving data for analysis.

Investigators need to be good listeners who can hear exactly the words being used by those interviewed.

Qualifications for investigators also include being able to ask good questions and interpret answers.

Good investigators review documents looking for facts.

Investigators need to be flexible in real- life situations and not feel threatened by unexpected change, missed appointments, or lack of office space.

Investigators need to understand the purpose of the study and grasp the issues.

Investigators must also be aware that they are going into the world of real human beings who may be threatened or unsure of what the case study will bring.

4. Collect Data in the Field The researcher must collect and store multiple sources of evidence

comprehensively and systematically.

Researchers carefully observe the object of the case study and identify causal factors associated with the observed phenomenon.

Renegotiation of arrangements with the objects of the study or addition of questions to interviews may be necessary as the study progresses.

Case study research is flexible, but when changes are made, they are documented systematically.

Maintaining the relationship between the issue and the evidence is mandatory.

The researcher may enter some data into a database and physically store other data, but the researcher documents, classifies, and cross-references all evidence so that it can be efficiently recalled for sorting and examination over the course of the study.

Step 5. Evaluate and Analyze the Data The researcher examines raw data using many interpretations in

order to find linkages between the research object and the outcomes with reference to the original research questions.

Throughout the evaluation and analysis process, the researcher remains open to new opportunities and insights.

Its use of multiple data collection methods and analysis techniques, provides researchers with opportunities to triangulate data in order to strengthen the research findings and conclusions.

Researchers categorize, tabulate, and recombine data to address purpose of the study, and conduct cross-checks of facts and differences in accounts. Focused, short, repeat interviews may be necessary to gather additional data to verify key observations or check a fact.

Researchers use the quantitative data that has been collected to confirm and support the qualitative data which is most useful for understanding the rationale or theory underlying relationships.

Step 6. Prepare the report case studies report the data in a way that transforms a

complex issue into one that can be understood, allowing the reader to question and examine the study and reach an understanding independent of the researcher.

Case studies present data in very publicly accessible ways and may lead the reader to apply the experience in his or her own real-life situation.

Some researchers report the case study as a story.

The researcher uses representative audience groups to review and comment on the draft document. Based on the comments, the researcher rewrites and makes revisions.

Analyze the data using: 1) SWOT analysis (Strengths, Weaknesses, Opportunities and

Threats) 2) Environmental scan 3) Critical success factors;

Scaling involves creating a continuum upon which measured objects are located.

Attitude Scale

1 = Extremely Unfavorable

2

3

.

.

.

100 = Extremely Favorable

MeasurementActual assignment of a Numbers

Scaling The process of placing the respondents on continuum

Primary Scales of Measurement

7 38

Nominal Numbers Assigned

Ordinal Rank Orderof Winners

Interval PerformanceRating on a0 to 10 Scale

Ratio Time to Finishin Seconds

Thirdplace

Secondplace

Firstplace

Finish

Finish

8.2 9.1 9.6

15.2 14.1 13.4

Primary Scales of Measurement

Nominal measures offer names or labels for certain characteristics.

serve as Tags for identifying objects.

Do not reflect the amount of the characteristic

Counting is the only permissible operation

Limited number of statistics based on frequency counts

o Ex: different types of blood: A, AB, O, B

o 1=A, 2=O, 3=B, 4=AB

o Male = 0 and Female = 1, ‘sex’ is measured on a nominal scale

Nominal Scale

Primary Scales of Measurement

Record information about the rank order of scores.

Ordinal measurements describe order

but not relative size or degree of difference between the items measured.

In this scale type, the numbers assigned to objects or events represent the rank order (1st,2nd ,3rd ,etc) of the entities assessed.

ordinal variables do not tell us anything about the absolute magnitude of the difference between 1st and 2nd or between 2nd and 3rd. That is, we know 1st was before 2nd, and 2nd was before 3rd, but we do not know how close 3rd was to 2nd or how close 2nd was to 1st. The limits of ordinal data are most apparent when one looks at the distance between the third and the fourth place finishers.

o Ex: A horse race, which says only which horses arrived first, second, or third but include no information about race times.

Ordinal Scale

Primary Scales of Measurement

Interval scales tell us about the order of data points, and the size of the intervals in between data points.

Quantitative attributes are all measurable on interval scales

The location of the zero point is not fixed. And negative values can be used.

o An example of an Interval scale is the Farenheit scale of temperature. In the Farenheit temperature scale, the distance between 20 degrees and 40 degrees is the same as the distance between 75 degrees and 95 degrees.

Interval Scale

Primary Scales of Measurement

A ratio scale is an interval scale with a true zero point.

Possesses all the properties of the nominal, ordinal, and interval scales

Meaningful to compute ratios of scale values

All statistical techniques can be applied

Most measurement in the physical sciences and engineering is done on ratio scales such as Mass, length, time, angle, energy.

The Klvine temperature scale has a non-arbitrary zero point of absolute Zero, which is denoted 0K and is equal to -273.15 degrees Celsius.

Ratio Scale

Each type of question corresponds to a level of

measurement. These levels of measurement will

determine what type of data analysis we use: All qualitative research is nominal.

All categorical data in which there is no difference in value among the categories is nominal.

All categorical data in which there is an implied ranking (for example, high-medium-low) is ordinal.

Any question that asks respondents to rank order responses is also ordinal.

Any question in which the response is a number or can be interpreted as a number with equal values among the data points is ratio.

ClassificationNoncomparative

Rating Scales

ContinuousRating Scales

ItemizedRating Scales

SemanticDifferential Stapel Likert

Continuous Rating ScaleRespondents rate the objects by placing a mark at the appropriate position

on a line that runs from one extreme of the criterion variable to the other.

The form of the continuous scale may vary considerably.

How would you rate Sears as a department store?

Itemized Rating Scales The respondents are provided with a scale that has a

number or brief description associated with each category.

The categories are ordered in terms of scale position, and the respondents are required to select the specified category that best describes the object being rated.

The commonly used itemized rating scales are the Likert, semantic differential, and Stapel scales.

Are you satisfied with our service?

Using the 5-point scale provided, please indicate how satisfy you are with our service

Itemized Rating Scales:

(1) Likert Scale/ Summated-Ratings Scale

Itemized Rating Scales:

(2) Semantic Differential ScaleThe semantic differential is a seven-point rating scale with end points

associated with bipolar labels that have semantic meaning.

Marks & Spencer is:Powerful --:--:--:--:-X-:--:--: WeakUnreliable --:--:--:--:--:-X-:--: ReliableModern --:--:--:--:--:--:-X-: Old-fashioned

The negative adjective or phrase sometimes appears at the left side of the scale and sometimes at the right.

This controls the tendency of some respondents, particularly those with very positive or very negative attitudes, to mark the right- or left-hand sides without reading the labels.

Individual items on a semantic differential scale may be scored on either a -3 to +3 or a 1 to 7 scale.

Itemized Rating Scales: (3) Stapel Scale

The Stapel scale is a unipolar rating scale with ten categories

numbered from -5 to +5, without a neutral point (zero). This

scale is usually presented vertically. +5 +5 +5

+4 +4 +4

+3 +3 +3

+2 +2X +2

+1 +1 +1

HIGH QUALIT POOR SERVICE WIDE VARIETY

-1 -1 -1

-2 -2 -2

-3 -3 -3

-4X -4 -4

-5 -5 -5

Basic Noncomparative Scales

Scale Basic Characteristics Examples Advantages Disadvantages

Continuous Rating Scale

Place a mark on a continuous line

Reaction to TV commercials

Easy to construct

Scoring can be cumbersome unless computerized

ItemizedRating Scales

LikertScale

Degree of agreement on a 1 (strongly disagree) to 5 (strongly agree) scale

Measurement of attitudes

Easy to construct, administer, and understand

More time consuming

Semantic Differential

Seven-point scale with bipolar labels

Brand, product, and company images

Versatile Difficult to construct bipolar adjectives

StapelScale

Unipolar ten-point scale, -5 to +5, without a neutralpoint (zero)

Measurement of attitudes and images

Easy to construct and administer over telephone

Confusing and difficult to apply

Surfing the Internet is____ Extremely Good____ Very Good____ Good____ Bad____ Very Bad____ Extremely Bad

Surfing the Internet is____ Extremely Good____ Very Good____ Good____ Somewhat Good____ Bad____ Very Bad

Balanced Scale Unbalanced Scale

Balanced and Unbalanced Scales

Rating Scale ConfigurationsA variety of scale configurations may be employed to measureSome examples include: the comfort of Nike shoes

Nike shoes are:

1) Place an “X” on one of the blank spaces…Very Very

Uncomfortable Comfortable

2) Circle the number…Very 1 2 3 4 5 6 7 Very

Uncomfortable Comfortable

3) Place an “X” on one of the blank spaces…Very UncomfortableUncomfortableNeither Uncomfortable nor ComfortableComfortableVery Comfortable

Thermometer ScaleInstructions: Please indicate how much you like McDonald’s hamburgers by coloring in the thermometer. Start at the bottom and color up to the temperature level that best indicates how strong your preference is.

Form:

Smiling Face ScaleInstructions: Please point to the face that shows how much you like the Barbie Doll. If you do not like the Barbie Doll at all, you would point to Face 1. If you liked it very much, you would point to Face 5.

Form:

1 2 3 4 5

Like very much

Dislike very much

100 75 50 25 0

Some Unique Rating Scale Configurations

---

Criteria for Good Measurement (1)

Reliability – Reliability is the degree to which measurements are devoid of error and therefore in the position to yield consistent results, also over repeated attempts over time (ordinal measures always yield the same order, interval measurements always yield the same order and same distance between the measured items)

Validity – Validity is the ability of a scale or measuring instrument to measure what it is intended to measure (e.g. is absenteeism from work a valid measure of job satisfaction or are there other influences like a flu epidemic which is keeping employees from work)

Criteria for Good Measurement (2)

Sensitivity – Sensitivity is the ability of a measurement instrument to accurately measure variability in stimuli or responses (e.g. on a scale, the choices very strongly agree, strongly agree, agree, don’t agree offer more choices than a scale with just two choices - agree and don’t agree – and is thus more sensitive)

Scale EvaluationScale

Evaluation

Reliability Validity

Test-RetestInternal

ConsistencyAlternative

Forms Construct

Criterion

Content

Convergent Validity

Discriminant Validity Nomological

Validity