Research Design and MethodsData Collection and Analysis

Martin Ganco

BPS Dissertation Workshop, 2011

Planning dissertation research:

Empirical design Paths to take:

Secondary data & large sample analysis Primary data & large sample analysis Primary data & small sample - qualitative analysis Combination of the above

Common student traps:

Developing many new pieces at the same time (theory, data, method, measures).

Having beautiful and polished theory and starting to collect data very late.….and end up with theory which cannot be tested or

cannot be tested well. Be “too” data/context-driven. Hand-collected and novel/interesting but poor

dataset. Suffering from mistakes made early. Expending all energy in one area (e.g.

identification at the expense of theory).

How to avoid the traps?

Limit the number of novel components Start data collection and analysis early (year

2-3) but be open to restart the process. Build/obtain a strong dataset! Theory building and empirical testing is an

iterative process The data will limit your degrees of freedom in the

theorizing Match theory to data – measures need to reflect

the mechanisms

How to avoid the traps?

Measure twice cut once Very path dependent process – choices that you make

early in the process, when you lack skills, may haunt you later.

Be careful and spend a lot of time in the design stage. Obtain feedback from different people. Stay flexible. Be open to revising

theory/design/data/method after feedback (welcome to the review process).

“Hourglass” approach Start broadly (focus on a high level theoretical relationship) Narrow down to 1-2 key variables Extensively analyze and obtain a set of well identified

baseline results Expand to include new variables or interactions

Methods: endogeneity and the “identification revolution”

Do not despair! Even though the identification is critically

important, it should not be at the expense of a theoretical contribution.

Be explicit about the endogeneity concerns and alternative explanations.

Understand the patterns in the data and be creative. Good identification doesn’t necessarily mean an instrument or a natural experiment.

…and feel free to violate any of these rules if you feel that it is the right thing to do.

Because…

Q&A

Which theory to test?

Ease ofempiricalimplementation

Formalization

Analytical (math) equilibrium-based models

Verbal theorizing

(Agent-based) simulations

Tradeoffs in theory building

Methodological progress

Ease ofempiricalimplementation

Formalization

Tradeoffs in theory building

Which theory to test?

Ease ofempiricalimplementation

Formalization

Level of abstraction

Tradeoffs in theory building

Which theory to test?