MIXED METHODS: backgrounds, findings and challenges

A systematic review

Joca Zurc

Leuven, December 2011

Mixed methods are….

The third research group in science (Teddlie and Tashakkori, 2009)

The third methodological moment (Teddlie and Tashakkori, 2003)

The third research path (Gorard and Taylor, 2004)

The third research paradigm (Johnson and Onwuegbuzie, 2004)

Theoretical backgrounds• MM are new dimension, new quality in science• MM established from dichotomy between QUAN in QUAL tradition • Positivism/ post-positivism, numeric data• Constructivism, narrative data • Pragmatism, combined data, in “adolescence”• MM researches working with pragmatic paradigm, interested in both,

narrative and numeric data and their analyses in the same research design (Teddlie and Tashakkori, 2009).

QUAL + QUAN quan + QUAL,

QUAN QUAL QUAL quan

• From methodological dichotomies to methodological continua - Multidimensional Model of Research Methodology (A integrated set of continua between qualitative and quantitative methodology) (Niglas, 2010).

• social, behavioural, education and health areas

• specific sub-population (rural areas, pregnant women, minority groups, addicted people)

• Life style and health (health promotion, social politics)

• Sensitive questions (conflicts, family relations)

• Online research ( participants, expenses)

• The basic goal of mixed methods research is not to replace either quantitative or qualitative approach but rather to draw from strengths and minimize weakness of both in single research study (Johnson and Onwuegbuzie, 2004).

• Concrete research problem provides the research design selections and not the methodological orientation of the researcher.

Theoretical backgrounds

• methodological approach must be suited in the research propose and leads to the answers on the research questions

• MM allows the researchers to draw a more holistic picture of the research problem

• knowledge of different MM research designs

• Multimethodological approach is not MM research approach; study of the research problem with different (only QUAN/QUAL) research methods

Mixed methods research designs

Classifications dimensions (Johnson and Onwuegbuzie 2004):

• mixing inside and between different research stages

• dominancy of the paradigms

• QUAN in QUAL phase time sequences

• the level of the integration

MM research design typology defines data collection, analysis, interpretation and report procedures in the research study (Plano Clark in Creswell 2007).

MM RESEARCH DESIGNS TYPOLOGIES

Morgan (1998) – dominancy and sequence

Teddlie and Tashakkori (2009) – 5 typology families of the MM research designs

Greene (1989, 2007) – purpose for mixing

Creswell and Plano Clark (2007) – 12 classifications developed in the 4 basic MM research designs; sequence, dominance and way of mixing.

The best selected MM research design corresponds to the research problem.

THE AIM OF THE EMPIRICAL RESEARCH STUDY– a systematic review

To analyse the ongoing research problems and purposes in MM research, MM designs, results and efficiency of MM approaches in different scientific fields in the last 5 years (2006–2011).

Research questions

1. In which scientific areas taking place MM research?

2. What are the main research problems and aims of the MM research in the resent 5 years?

3. What kind of the research designs have been used in the studied MM research (Teddlie and Tashakkori, 2009; Morse, 1991; Greene et al., 1989; Greene, 2007; Creswell and Plano Clark, 2007)?

4. What are the results of this research, or support or denied the use of MM approach in science?

MethodsSystematic review study

Inclusion criteria:• original scientific article, journal article, research• published in the last 5 years (July 2006 – July 2011)• peer review journals• Journal publication in the international research data

bases (WebSurvey Methodology, EBESCOhost (CINAHL, MEDLINE), PubMed)

• research or at least evaluation of the mixed methods use

• full text of the article available• English language

Table 1: Research searching history (N=30, N=21)

Key words WebSM EBESCOhost PubMed

matches

selected

matches

selected

matches

selected

mixed methods 76 14 413 / 3667 /

MM, research methodology

/ / 50 9 743 /

MM, research method / / 42 2 993 /

MM, statistics research / / 7 2 409 /

All key words / / / / 116 3

Number of selected article

14 13 3

Results ARTICLEARTICLE SCIENCESCIENCE

Revilla, 2010; Millar and Dillman, 2011; Gigliotti, 2011; Dillman et al., 2010; Potaka, 2008; Lobe and Vehovar, 2009; Link et al., 2009; McMorris et al., 2009; Deutskens et al., 2006; Matsumoto, 2007.

Research methodology (N=10)

Andrew and Halcomb, 2006; Nigg et al., 2009; Hopson and Steiker, 2008; Baker et al., 2010; McCabe et al., 2006.

Health sciences (N=9)

Mortenson and Oliffe, 2009; Dunning and LeMasters, 2009.

Occupational therapy

Bushy, 2008. Nursing

Schreiber et al., 2009. Physiotherapy

Curtis et al., 2009 Sociology (N=2)

Bryman et al., 2008 Social politics

MM RESEARCH PURPOSE ARTICLE

Mixed mode designs (N=11)

(multimethodological approach)

Revilla, 2010; Millar and Dillman, 2011; Gigliotti, 2011; Dillman et al., 2010; Nigg et al., 2009; Link et al., 2009; Dunning and LeMasters, 2009; Potaka, 2008; Lobe and Vehovar, 2009; McCabe et al., 2006; McMorris et al., 2009.

Mixing QUAN and QUAL research techniques (triangulation, complementarity) (N=5) (applied research)

Bushy, 2008; Hopson and Steiker, 2008; Schreiber et al., 2009; Curtis et al., 2009; Baker et al., 2010.

Utility and value of MM research (N=1)

Andrew and Halcomb, 2006.

Quality criteria in MM research (N=2) Bryman et al, 2008; Deutskens et al., 2006.

Implication of MM survey developed abroad (N=1)

Matsumoto, 2007.

Diversity of definitions and applications of MM approach (N=1)

Mortenson and Oliffe, 2009.

TIPOLOGY ARTICLE

Explanatory design (Creswell and Plano Clark, 2007) (N=2)

Hopson and Steiker, 2008; Curtis et al., 2009

Exploratory design (Creswell and Plano Clark, 2007) (N=1)

Lobe and Vehovar, 2009

Triangulation design (Creswell and Plano Clark, 2007; Greene, 2007) (N=4)

Deutskens et al., 2006; Schreiber et al., 2009; McCabe et al., 2006; Baker et al., 2010

Complementarity design (Greene, 2007) (N=2)

Hopson and Steiker, 2008; Curtis et al., 2009

Expansion design (Greene 2007) (N=1)

Lobe and Vehovar, 2009

Parallel design (Teddlie and Tashakkori, 2009) (N=3)

Deutskens et al., 2006; Schreiber et al., 2009; McCabe et al., 2006

Sequential design (Teddlie and Tashakkori, 2009) (N=3)

Hopson and Steiker, 2008; Curtis et al., 2009; Lobe and Vehovar, 2009

Conversion design (Teddlie and Tashakkori, 2009) (N=1)

Baker et al., 2010

MM APPROACH EFFECTIVENESS

All research (21)

ATTENTION - justification of the MM use regarding research problems (Mortenson and Oliffe, 2009),- equal position of the qualitative findings,- research survey tradition, culture, survey application to the society (Matsumoto, 2007),- qualitative research impact (Hopson and Steiker, 2008; Curtis et al., 2009),- online research (lower expenses, higher respondents).

Further research questions erased from the results of the mixed methods systematic review research (N=9)

Applications and further development of findings: Applications and further development of findings: • research repeat on similar or contrast populations (Revilla, 2010)• measurement instrument further development (Potaka, 2008)• MM in online research (McCabe et al., 2006)

MM in specific science area:MM in specific science area:• health programs systematic evaluation (Hopson and Steiker, 2008)• research in clinical practice (Schreiber et al., 2009)

MM basic research concepts investigations:MM basic research concepts investigations:• MM in occupational therapy (Mortenson and Oliffe, 2009), community nursing (Andrew and Halcomb, 2006), rural health and nursing (Bushy, 2008)

Qual + Quan - Systematic Review (Meta-analysis)2001-2011Slovenia Nursing Review, Slovenia Journal of Public Health

Qual - Expert Interviews:editors, methodology educators in higher education system

Survey: mixed methods researchers.Qual + Quan - values, attitudes, obstaclesQuan – test knowledge MM, demographic variables.

coplementarity

EXPANSION

Qual + Quan - Systematic Review (Meta-analysis)2001-20111 international nursing and 1 international public health journal

development

development

coplementaritydevelopment

Conclusions

• MM in methodology and health sciences

• Research problems: data collections, MM purposes

• 7/21 research used MM research design:

• QUANT dominant approach (Palinkas et al., 2011; Niglas, 2009)

• 50% complementarity, sequential, explanatory, exploratory design (Collins et al., 2007 – 34%)

• 50% triangulation, parallel design (Collins et al., 2007 – 66%)

• 21 supported MM approach use in science (Plano Clark et al., 2008)

• 9 emphasis further research questions:

• Health practice development

• Basic concept/ paradigm of MM investigation

• MM in online surveys (Koller and Sinitsa, 2009),

• MM research models in research practice in different scientific disciplines and researchers attitudes to the MM approach (Bryman, 2007).

Thank you for your attention!