Copyright © 2020 Statswork. All rights reserved 1

RESEARCH DESIGN DECISIONS AND BE COMPETENT IN THE PROCESS

OF RELIABLE DATA COLLECTION AND ANALYSIS

Dr. Nancy Agens, Head,

Technical Operations, Statswork

Brief

Research Design may be described as the

researcher’s scheme of outlining the flow

of his project. It is based on research

design, that the researcher goes about

gathering data to answer his research

question. If the idea is to complete a

building, then it has to be decided whether

it is going to be an apartment, stand-alone

house or a shopping complex, who are its

occupants? and what are the materials

needed? The plan of the project, namely

the planning for the materials and the

logistics involved follows this. Similarly, in

research as well, the researcher chooses his

data collection process based on his

Research design decision. It enables the

researcher to prioritize his work, create

better questionnaires and arrive at

conclusions with greater clarity.

It would be worthwhile to take a look at an

example:

Table 1

Evaluation Matrix: Matching Data Collection To Key Evaluation Questions

Examples of

key evaluation

questions

(KEQs)

Programmed

participant

survey

Key

informant

interviews

Project

records

Observation of

programme

implementation

KEQ 1 What

was the quality

of

implementation?

✔ ✔ ✔

KEQ 2 To what

extent were the

programme

objectives met?

✔ ✔ ✔

KEQ 3 What other

impacts did the

programme have?

✔ ✔

KEQ 4 How could the

programme be

improved?

✔

✔

Source: Peersman,(2014)

Copyright © 2020 Statswork. All rights reserved 2

In the above diagram, table1 shows

the type of questions and the data collection

methods that were used for the same. For

instance, Key informant interviews and

Project records were used for collecting

information on the quality of the

implementation. Quantitative research

design may be sub-divided into

experimental, Quasi-experimental, Survey

and Correlational, while, Qualitative

research may be divided into Ethnography,

Case study, Historical and Narrative.

Broadly, RD can be classified into

Exploratory and Conclusive. Exploratory

research is a research conducted for

a problem that has not been studied more

clearly, intended to establish priorities,

develop operational definitions and improve

the final research design.(Shields &

Rangarajan, 2013) It does not seek to arrive

at a conclusion. Conclusive Research can be

classified into descriptive and causal.

Descriptive research tries to answer

questions such as what and How? While,

Causal research tries to establish the cause-

effect relationships among the variables of

the research.

Table 2

Major Differences Between Exploratory And Conclusive Research Design

Research project

components

Exploratory research Conclusive research

Research purpose General: to generate insights about a

situation

Specific: to verify insights and aid in

selecting a course of action

Data needs Vague Clear

Data sources Ill defined Well defined

Data collection form Open-ended, rough Usually structured

Sample Relatively small; subjectively selected to

maximize generalization of insights

Relatively large; objectively selected to

permit generalization of findings

Data collection Flexible; no set procedure Rigid; well-laid-out procedure

Data analysis Informal; typically non-quantitative Formal; typically quantitative

Inferences/

Recommendations

More tentative than final More final than tentative

Source: Pride-Ferrell,(2006)

I. DATA COLLECTION TECHNIQUES

AND HOW TO CHOOSE ONE

Using a mix of both Qualitative and

Quantitative methods can be most beneficial.

The most widely used data collection

techniques are Interviews and

Questionnaires. Interviews may be one to

one or in groups. The Questionnaire is

developed with the research question in

mind. But it is very difficult to determine if

the participant is lying or not. Hence

reliability is a problem here.

Here are a few tips on developing

effective survey Questionnaires:

1. Ensure that that the length of the survey

questionnaire does not run to more than

five minutes.

2. Avoid complicating the Questionnaire

by using questions which may refer to

Copyright © 2020 Statswork. All rights reserved 3

answers of previous questions. For

instance, ‘If your answer was yes to Q.

No 3 then…’.

3. Take care to see that the Questions don’t

look biased. ‘You would not refer XYZ

Baby oil to your friend. Would you?’

4. Ensure that you keep the Demographics

in mind and use uncomplicated words.

5. Make sure that the questions do not

carry conflicting ideas, such as ‘Which

is the best and cheapest restaurant in

town?’ The best restaurant need not be

the cheapest.

Using Data Collection tools such as

‘Device Magic’ which helps you to pre fill

form data. ‘Fulcrum’ allows for custom

maps with geo location while ‘Fast Field’

enables exporting to word and pdf. ‘Magpi’

has features for interactive data collection.

‘Zapier’ helps automate the Data Collection

process.

II. DATA ANALYSIS

Probability and non-probability

methods are used in Data Analysis.

Probability sampling uses random or semi-

random methods to select a sample from

among the given population and it uses

Statistical generalization with a margin for

error as no sample will exactly reflect the

population exactly.

Random Sampling uses a simple

process where there is equal likelihood of

every member from the sample being chosen.

Stratified Random Sampling uses a method

of segregating the sample into mutually

exclusive groups and then selecting simple

random samples from a stratum.

Example:

Strata1: Gender Strata2: Income Strata3: Occupation

Male <1 lakh Self-employed

Female 1 to 2 lakhs Clerical

2-5lakhs Professional

In the above sample we can choose

females with income range of 1 to 2 lakhs

using simple random sampling. We are now

able to make inferences across these 3 strata.

After stratifying the population, simple

random sampling is used to generate the

complete sample.

Among non-probability sampling

methods Purposive sampling is used where

particular cases which are information-rich

are selected with a view to drawing

inferences about the population.

Convenience sampling is used only in cases

of insufficient data.

Mixing methods can improve

credibility of the research findings as each

data source possesses its own limitations and

advantages and triangulating data from

different sources or integrating different

collection methods will help answer the

research question more accurately.

Some methods of Numerical analysis

are given below:

Copyright © 2020 Statswork. All rights reserved 4

Table 3. Some Methods Of Numerical Analysis

Numeric analysis

Analysing numeric data such as cost, frequency or physical characteristics. Options include:

Correlation: a statistical technique to determine how strongly two or more variables are

related.

Cross tabulations: obtaining an indication of the frequency of two variables (e.g.,

gender and frequency of school attendance) occurring at the same time.

Data and text mining: computer-driven automated techniques that run through large

amounts of text or data to find new patterns and information.

Exploratory techniques: taking a ‘first look’ at a data set by summarizing its main

characteristics, often through the use of visual methods.

Frequency tables: arranging collected data values in ascending order of magnitude, along

with their corresponding frequencies, to ensure a clearer picture of a data set.

Measures of central tendency: a summary measure that attempts to describe a whole

set of data with a single value that represents the middle or centre of its distribution.

Measures of dispersion: a summary measure that describes how values are distributed

around the center.

Multivariate descriptive: providing simple summaries of (large amounts of) information

(or data) with two or more related variables.

Non-parametric inferential: data that are flexible and do not follow a normal distribution.

Parametric inferential: carried out on data that follow certain parameters. The data

will be normal (i.e., the distribution parallels the bell curve); numbers can be added,

subtracted, multiplied and divided; variances are equal when comparing two or more

groups; and the

sample should be large and randomly selected.

Summary statistics: providing a quick summary of data, which is particularly

useful for comparing one project to another, before and afterwards.

Time series analysis: observing well defined data items obtained through

repeated measurements over time.

Textual analysis

Analysing words, either spoken or written, including questionnaire responses, interviews and

documents. Options include:

Content analysis: reducing large amounts of unstructured textual content into manageable

data relevant to the (evaluation) research questions.

Thematic coding: recording or identifying passages of text or images linked by a common

theme or idea, allowing the indexation of text into categories.

Narratives: construction of coherent narratives of the changes occurring for an

individual, a community, a site or a programme or policy.

Timelines: a list of key events, ordered chronologically.

Source: Peersman,(2014)

Copyright © 2020 Statswork. All rights reserved 5

REFERENCES

[1] Peersman, G. (2014). Overview: Data Collection and

Analysis Methods in Impact Evaluation:

Methodological Briefs-Impact Evaluation No. 10.

Retrieved from

https://ideas.repec.org/p/ucf/metbri/innpub755.html

[2] -Ferrell. (2006). Foundations of marketing. McGraw-

Hill Education London. Retrieved from

http://www.shermanchui.com/upload/file/20161020/

1476955790263897.pdf

[3] Shields, P. M., & Rangarajan, N. (2013). A playbook for

research methods: Integrating conceptual

frameworks and project management. New Forums

Press. Retrieved from

https://www.researchgate.net/publication/263046108

_A_Playbook_for_Research_Methods_Integrating_

Conceptual_Frameworks_and_Project_Management