handbook of mixed methods in soc and beh reserach_tasshakkori_2003scan

8/3/2019 Handbook of Mixed Methods in Soc and Beh Reserach_tasshakkori_2003scan

1/19

D A T A C O L L E G I IO 'N S T R A T E G IE SI N M 1 X E D M E T H O D S R E S E f \R C H

~

~~i . . -~r ('. . ... -, ~ . : . :---- -

A mbO JiS of th e p rev io us c ha pters Inthis handbook Jiave discussedrna ny nf th e : i ssues 1)U rroundin gmixed methods res ea rch, i: tr1 Jc .~ud. ingthephilosophy of pragmetism, wriring re-search questions, desi g~ ing tn u ) I " , d meth-o ds s tu die s, a nd s.1l!.mp~ing, Now j we moveinto [he specific methods of dam collec-tion tlut are used .ill empi rical resea rehand how these rneth mis can he mixed andmatched in one or more research studies,W eFocus on the six m ajor m ethods of datacollection used in the social and behav-io ra l s cien ces and ~u)w~hey may be mixed,

tOllmed by crossingrwn dl i''111~11I~~nns. Till;'first dimension is th e rese tem;h@pproachdi -mens ion, P ll_ :req ll< l1 [t< '1 t! veand pere quae-tim . tiv c res ea rc h a reth e p nllf;;';\lf t.~ is first c l i . "mension, In this chapter, pure quaUta~ivereseard: is defined < 1 1 . 8 exploratory, indue-tive, Uln$ttl!llC;:tu[~cl~ open-ended; naruralis-tic, and f re e~ kn 'ifin g resesrch r.h:a~n:su~tr>in qualitarive da ra, PHI'e qwmt#atflJe re-~earc.bi~ defi ned 3S confirmatory, deduc-ti ve, structured j closed-ended) cora trol led,and linear research that results 1mquant i -tarive data. The mixed anchor on the re-sea rdUllppFOaC~di rne[Lsiurl n;i=m;Se-'lll~.s re-sea rch that mixes ebeapproaches or uses aless extreme versicnthan the pure quelira-


2/19

299. ME1HODOLOCICAL AND ANAlYTICAt 1SSUES

1 2. 5Jj

11

2, Interviews3, Foeus grou p s4. Tests

D , Securulory dilla (e,g,. personal' erncalled "method triangulation' (e.g.,De'l~l ' i~l> ~989). lnream ethod m Ux lng musti,[ilclude either


3/19

TAB LE11 . . 2 Data C oUedion Styles fhr Mixed Ceil isT y p p 2l~ jW ~ly pe IlTW e 11'lpc' f4

M1Xll l f \L~ of ~~fX'n anrl d'DSr.d~nderl ilQm~ on on (' or t1I10F(' qur~ tionll ~ irE'~Mb:1u re 01 dEpl h ~nd bre.:tcltih interviewi ngMhlUI"e of "a prior!" and ffemergent/flowing" focus groUI,) SitmteJ!,ie~Mi::d.u~ of ~ t,ln da rd i.::e d o pe n- a lrld d os ed -e nd ed p re -m a de Lest~MhclUrp o r sl,"nd


4/19

va iidity types that were ori g i" [ < 1 II Y devel-oped ro[ qualitative research. Th e firsttype, called descriptwe validity, refers e othe factual accuracy of < In account as re -p m!Jr:d by the rcsea rch e r. It is essenti a 1r h a tthe researcher carcfim lly collect and cor-roborare descriptive information duringthe process of data collection [0ensure itsaccu:rJC :y. The second type, ca Hed in t~rprc tiue y ,aUc! it' y, rc fe rs ro the degree to w h i c hd: lc researcher acclj[ardy portrays the par-t ici panrs' meani ngs a bOLU what is beingstu died. [ 1 [ 1 , other wo rds, th e researcherInusr collect d ara th< IJtwill shedl L gln on th ~"ernie" {or ,.,nsider" or "native") v iew s ofresearchparticipants. The goal heeeis [:0Unde rstan d the resea rch pa rticipa nrs'(r;uher~ha.n the re~e:;]rcher'~} vicw~ amiways of thinking , The rhird rype, calledtheor.et icat!Jal idi ty , refers to the degree [0which ,1 theo retical ex plaearien dev elop ;edby the research ( ; : 1 " fit~ the d ala (see aboGlaser &. Strauss, l ~'!(:i7; Streuss &C or bin, 1 .998). S om e specific sr ra eegiesforobtaining Maxwell's (aJllJdother} types atvalidity include member 'r.:h~,kung, reflex-~vity; negative case al l ' l< 'L~Ys~s~r iangular inn(method, data investigaroe, and [hM.rytria ngu lation], extended f~eldwork, per-sisrenr ebservsriou, paneru matching,

300. METHODOLOGJCAL AND f\NALYHCAL [SSUES

qL1


5/19

It).) and the" threars" to these, are coveredill v i r n . m n y every research methods text-hook i.n the social ::LmGhehavioral sciencesI reg". Babhie, 2001 ; Chrisreusen, 2001;Franklort-Nach mi a s & Nachmias, 2000;Johnson! & C hristensen, 20 00 ~. InternalI".ali.rl~tystraddrionally defined as whaem~y be called "causal validity" OJ! ' one'siusrificaruon in making a c am , al in fer en cef rum one's d a ra, C a usal valid i[ 1 ' (reg:ud"l es so f what it is c aB ed) C311 be importarrrrn ' il JaJ1:i~~lati1o '" '[ : .qualitative, and mixedmetho ds resea n:h (Yin, 1 99'4 ), E xterna l\a!idi l Y is rrad itionally defin ed as the de-gree to wh ich on e can gfnC;lI;lI i7',(;a. re-se~n;:~finding ro other people, places, ser-lings, and times. Stake n990, ] 995)offered a n ice m iddle grou ndfor g~rlcli..l-iziHg when he recommended makingthO.L\ghtfuI a rid "ua ru ra listie genera Iiza-tions" (i.e., g.eneriliizing based on simjlar-il r oi people. circu mssances, cpera rion a Ii-zations, etc.), The conceprs of internal\~~!idill'.uH I ex te rna ! V O l ! ~ ~liyw ere (~ rigi~lIlaHy3 pr~ ied to quanriranvc research. Se'\i'-l"ta I. authors, however, have discussed howthese rypes of va Iidity 'C,1I1 be expa ndedand appllicd in qualltative and mixedmethods reseerch ~e.g., LeCompse &Preissle,]993. johnson & Christensen,1000; Onw~,(;ghuzil; j in pr(:~~; Schensul,S d K " 1 J ] ~ul,& teCom.pu:~ 1999; Tashakkori& Teddlie, ]998).

Although 111 a ny of C am phe rl\ rhres 1;$

should aJs(~ attempt to monitor -changestaking, plaice in research participants dueto 11arnra Im(U~~r.atinn .IS well as ch a nges intheir scores duero SMfl,stic.al regresst(mwl~(;:1] ,. extreme" groups are being S,[ ud ~eel.The po inr is that n.:5 .{ ,:"[1chers 8h OU Id hecognizant of each of Ca mpbcB\~hreat~du ring dsta collection (as. well as duringthe other s~tges of resea reh] and 5 1 1 nuld at-temp r to min imize the unfh]ence of thesethreats because of [heir potential impacton data quality.Another set of valid i[y issues sorrou nds

measurement v < J : l i d t t y (and reliabilirytharis ]] ecessa ry for val idity)" When (,:01h~(, :t~ngdara, researchers mlJSf consider 'IV herherthe],' are measuring what they planned tomeasure" This is tHLC when


6/19

30 2 . M ETHODOL ,OGICAL AND ANALYTICAl. ISSUES

eflecls. refers to error resulting from therespondenrs' awareness that they are [hetargets of study. Reactive effects may in-dude several specific types: (a) th e guineapig effect (i.e., rile participants are awarethat the camera is on them and react in aninaccurate, defensive, or di~~ho[]~Mway) ,(b) ro le selection (i.e, the participantsselect from "the many 'true' selves or'proper' behaviors" [p, 16J during the

':~h~~lr ~b l7lo;Cln '1 .1 i" I .!'IY"'P,'COI log" r r "Ur)Ab~ 'f ~,'\ \(.,measurement as ch;~lngcagenr (i..e., the ini-tial measurement activity can producechanges ill what' is being measured evenwhen the re;;pondents a rt' try ing tobe can-duel), and (d) response sets {i.e., th e parrici-pa 1 1 rs ten d E O respond in srcrcorypedw ayssuch a~ acquiescence and "yes-saying" urprefer extreme responses rather than mud-erare ones}.The second m ajor source of error oc-cu rring dlllt"ing data collection is called in -l/{!S{lxattJI' ef frcts and refers EO errors re-sulling from the mvcsrigator (\'l;febb er al.,1%6/2000). There are tWO s pe cific ty pe s:(a) interviewer effects [i.e., participantsart: influenced h y characteristics of the i l l -tcrviewer such as his or her age, class ..andgender) and (h.1change in the research in-srrurnenr (i.e., changes occurwhen the re-

The fourth source of erro r iden tified b)We:bh ct "I. is called access to content cr -m rs and refers to the resea rcher's ina b iii[).to obtain perfect samples of relevant COLl,rent. This problem includes (;l!) rcstri.:tlOII~on COIHC~t (i.e, Some metho ds pro vide at-cess eo content not available rhreugh theuse of o ther m erhods such as observationnor providing direct information. aboutmotivation}, (h ) , stab ility of content m er.~--Va I i t - 1 , '1.t':; UHffC']][' or 'ICOII cc.\e(i data lllJ)'

V,I ry ov r time due to factor!'. not relatedro the research q uestion), and ( c ) ~rabil-ill ' of content over area (i.e., content l ~ fdata collected in cliJfcn:nt areas may var~ 'due to fac to rs nor relarcd to rhe researchqucsrion ),The fifrh, rind last, source of error idcn-

rifled by W ebb et al. is th e operating easeand validity checks avaiiahlefor differentrncrhods. First, th e dross rate (0 .(.:". theamount of irrelevant murerial collectedwhen using u particular set of data) willvilry


7/19

Ji\Bm .1 E1 ,.3 P rin d pies o f Questiu.n nai re CllIll'IIs~ru:ctionPrinciple 1. Make Su re bhilt llie questionnaire items match YOU f rese nt ~ y pe ~ o f re ~p iln ro c a:te go rle '5 .w ailaM e fo r d05oo"en c.h~d

questiennaise i terns.

Principl~111. Use mU~{Lpr items tv meas ur e , lIj ~ raC l C 01 ~ 51r uc tJ ;.F 'tii'lc ip le 'I 2 . D ~~ lf lp a q ucsHol lH ' lo1 i ~ Ih.; it t,~~ "~ Y f m ( h e p ,:l'r i! id piliIT t5 to u s e.Plin~ple 13" A l w i 9 Y S pilot-test your qu estionnal r:.:_SOU I KE : I -\d ap l{ 'c i ~ m ll! l J oh ! l'-O ! t a nd (,'hrist\:'l 'K{'il 12000 ) .

~l'.compensated in 11 S Ingle well-des igneduudy, o r i[rna ytake 1 T J . < 1ny s tud ie s [ ' [ H I I e i[l glip a resea rch I I i:rel'". '~tUl n: ro corn pensa W fmrhe va rio,u s errors .. n dro dcm onsrratc ("011-\"eIgingcorroborariee of a research f~nd-lng, In r he next section. we discussthe dif"f e r e n t methods of data collection that


8/19

FRf \MEWORKF O R ANIALYliNGT A IN M IX E D M E T H O D S R E S E f \ R C H

prev ious c~a.pu:n of rh i s s ect ion oft1rr~book, she a IL ] ,~hoFShave discussedsstages of the mixed methods, re-process. Specificallj', these stages,were presented in seq uemial 0eder,

id e n r iJy i n g fh~ m i X I : , d merhodspu.rpolSieand determ ining re-

questions that match rhe purpose- 6 ) , outli ning the mixed methodsdtsign (ChaptC'rs 7-9), ,~~~c,riU'lg

samplc(s) [Chaprer 10), and cellecring(Chapter 11).

dl lC pn int at wh ieh eh e d:;lm a~ < I ~ j'sis hegi rr s. \ ' lndends depends. on the type of dara eel-~ecred, which in turn depends on the S , 1 ) m-ple size, which in tum depends on the re-search design, which m turn depends 011t11.~ [~.~(;!~ rchpurpose.

Fo r f 'x tlJmpl. e. ][i[ 3pa r~Jl~J : ls . imu! .mneous'mixed methods d.es]gn. in which quanrira-rive and qualitarive data are collected atthe K


9/19

l}EC ISION Sl ltFI...AnO.NS H1P~)'BETWEEN QUANTITAl'WE .ANn'QUAUTATWE DitTA lYrES

1-

If" On the other hand, both quanri ~ativeand qualitative daratypes are used.thentwo decisions emerge. lFiHt, the dats .. ana-l y st m ast decide whether the qUlantitativcd~"'" ' 'W iOli;\iJi1l~ r..:'N'C ''~ ' ''P/Jt&",,,",,,~ ,iU he liseappn)ximOl!tdy equ.i ! i~y [ i.e .., (Hl~f1'.v(J.lstatus design) or whet hc.r eirher rhe qum

ritative or the quaht:a[ivf fee hn iq ues shouh i ; ' domilllant (i.e., domin(lrd-1c5S domnam desIgn) (Cn:::~,,\'d!,1994,; Tashakko& Teddlie, 1998). Clearly, this decisiod.~p.r :nds (HI the resea n.:hpurpme (MN("wmal1l et al., Chapter 6, this vol u me

d

n


10/19

Manicas& Kruger, 1976). A f f > noted byMiller (Chapn;r H. this volumej, elim-i03[iveimluction i s extremely appropriatefor m ix ed rn ethod s stud ies,

A M odel for the MixedMethQJ.s Da ta A1~aly$is .ProassOncethe m ix edm ethods data analyst hasmade[he 12 sets of decisions presented int h e prev icu s sec tion. he or she is ready toandlp~ the data. f.igmc 13.lpmvi des amode lo f the m ix ed m ethods resea rc h p ro -cess,Ir ca Ii! be seen from th~ s d iagra m th a td ~ F ~ analysis [5 pan uf1l! fO~I.["-cnm.[H~t'I.~rn,interactive, cyc!]ca1 research process ]l'["relv in gd ata eo llec rio n, d ata a na ly sis , datamterpreta ti on j a nd ~egitirna dun" Thesetuur mmponefllts are represented in F igure1J. l b y dli psoids,Once clara haw been collected and the

l 2 a fo~emC '[ lJfioned s crs o f decisionsha vcb een made, the mixed methods data analy-5 i ~ beg ins, Data ~n.aly~usconsists oftht: fol-lowing s ev en stag es; cla ra red uction (SeageI~ , dam dlisp Jay (S ta ge 2 ), data transfer-mar ion (S t ilg .e 3), da ta cone] ar ion (Stage41~Jal i lconsolidatien (Su.g~ 5), dsta com-parison {S~age 6). and d 1 1 [a inregrarion( S t a g e 7). These datil analysis stages arerepresenecd ill F igure 13 .] by rectangles, 1 1 1 d are defined in Table 13.4 ..Jrshould be rioted, however, thatwhile

S T A G E 1'"~.D AT A R E .m J CT JO NTheill1Jalys]s stage first in v'O llv es re o

duetion of whatever forms. o f f data wen;gathered at the da r a e co] lecti 011 stage, forquantitative ciata, clara reduction incl udescompura tionef descriptive starisrics (e.g.,measures of central tendency, measares ofdispersion/va riabiliry), Other ways of re-dm;ilrug ] ' \~~merka. :1ata


11/19

Oncethe m ix edm crhods dara analysr hasmade Iht ] 1 sers of decisions presented inth r p r ev ious sec ti on . he Of she is ready rcIn.1~}'1:e rhe data , Figm~ IJ. ~provides amodel of rhe .m ix ed rili::'thods rcsea rch pro-~5 . lr c an be seen from rh is d~a gT~nn thatdUJ < Illil! l'y "isis p art o f a fnur-compmu:rn,IlItrr:acrivc, C y d L C ~ ' 1 1 research process in -\'tJlvL rngam collect jo n. dam analysis, dataln1crpreIJJrion, and leg iti m arion. Thesef~mr,ompnnerlJts arc represented in Figure111 b y e ll ip so i c is .o nee data have been co llected and the11 J fOT~ml:nrimiH:d sets 0 f deci sions havebernmade, [he m ix ed m ethods dina analy-~I~begins. Data a na lysis consists ofthe fol-l {lw ing s ev en s ta ge s. data reduction (Sl ;LgeI}. dsra display ~Stage 1). dara [mnsfior-m:llion ($rJg,e J). d ata c orre la rio n (Sit


12/19


13/19

1 I : :d :i~.(""J~~ir;..",.~!1,r.) I r-~. !1eI:1~!~L~':-i-J:I!-I1o.. .;-:.11....n: ...,.t..Ii7 ~IT/.... :J-I }JI~I.;;p)JIII.lIlvrt rI.rr~ll-r'{:l. r..'i1..ir.lI.mylrurr.Jir ~~ ... ITLU,.JI,*

~-..:h:11'l: rpur.il' ...........l:rLll~ . 1~1lH. IT.ll~ .. .Jrllll I..I.....ILII.JI.~i:I~ n..'"'IlJ. .{Ii ..II ... 9L111h . . jli"lll'"JI~ Itltl. IIJ" P.,!,. 'l i!:1)1i..I i~II,!~~:-I

I ;'IL.~I" 11(01~IIIII -:pLI:'II~LilI" IL~,j ).:-I-i- I-'I~~: LrrI: vi ~"'IL"LI. . . . .-. . ::J-. ~.'-I-iu-.J:IJr~~k.- ~i"!:."I!rtli;l

(l.,.t.I~h~l:vih I~ ... lltp"" In rll-cl ... ....."..:-IClI1"IIl1kLlIL..J".,...;Ill." rr Ih'il....u

I(II''''''II~.:.II ~ II~.J ._.. . , ~ . ~ i I I I l. I-.ilr.-.r r . . . . . . . . . . --.Jr~II'"".r~-..II~. q!L...,..lill~,..1....-1.,LI:.II"li .."'" ;:.I11':'-mm IJ.hI.....

IE" IJI~11!!.Ib::i!-i liI':'l J . . . " i~rpl n:[!l1'b:41l;.T1ilTJll!ftll:l!d)'~in .....illllllll.ILI"-:II"'_1I1)I,ILiII"r ~ t .. r foo r .u.il[l] I!..n.]lt~i~ ' : . I i l r , o . T " ; . .

r;k'l1:~k!:'1I lin UII r:~rJ1:"'illn"';"'Ni mmri~II.~., l'mlClpill!: x '1 ht~ '1'I\11f1:oi1 "r . 1 1 1irr:r.u.nranh."lItIllJl:ri:o:: [i.e . ,P:lnt"'lllJIlIl,JriUnn DI.miKI ~n,J!; !Do:I~rmr-.i!' !hl; hi,;lr:u .


14/19

3 76 \:tF THOnO I.O C [CA L AND ANAL YTK A.L rSSUf .S

constructed COIH ~11"i'si rig < .1 en IT 1 bi ria t~nn 0 fo ~ and 1s. Th e-~t;.'l:HUT~CCS C:;!IlJ then be-usedIn determi ne fn:qu ency t mall ifesU ehemsize1>tind intensity (m anifest! effect SiZt'5forthe meta-themes (see Table 13.1 LFor example, ill the \Xri~dH~ret al,

f200 I) studj described earlier in whichpreservice reachers' perceptions of eharac-rer is tics of coffeeti ve reach e I'~ we re to : xa rn-ifled, t he resea rchers ful,ilnd ~h a l thl.: s U Nt::rm:~r~..:r 111cme~ ~st [I,J enr -Ct' nrcrcdness,t 'nrh ll.~iasm for reach ing , erh icalnes,cIJ~sroom and beha vi0r m ana gem ent,teaching methodology, and knowledge of~u h i ccr] su bcl~vi,cit'd into rhc fo I J ow in! ?;fum rn t'm - [b t'nH :~ : classroom armosphere,su bi eCI an J s a u dent, e [h iC;1!ness, JnJireaching methodology, noth the latent ef-fect si 7.eS f p mponions of var LUKe 'L;"X-p la in ed ) < I nltl rn a n I h:~ t dm.:r s i1 Jt's [ en -dorsemc nr p ropou ion) associ 1 1 red wi r~these meta -[he Ines we re moderate [0I : l I T g C . Th ' r : thematic structure i,~di:-;piayt:din F~ . ~un~ IJ.L blJ~h~s fLgur .. :,~h t' larenref-feu l> ize~perm iu rothe proportion of nui-ance explained by each meta-theme.whereas [be manifest cHeet size indicatesth e prupn rt inn of IIn it~ idcnt u f~cd permct a-theme.][ should be noted [hat caution should

themy rhnll!~hout the social andioral sciences (Onwut:gbuzu::,any case" i f iris inapp ro priateduct facror analyses on inter- .lndrespondent m atrices, then i t is alsopwpr ia rc to ~ulldu,"~ facto r anas cu r es r 'L : pn :: li< t: 'm ing cogn iti n: andmeasures. Second, according IOand Teng (191:)9), dicJ,o to l1m u~J~~~ lilk.dy to ykld Mfuhcts u~mganalyrleal techniques rhan arecaTegury le.g., Likerr-type] items.

]f on ~}' one type I)f data. i5then rhe data transformation leadsdam inregrari 0 rl J ~tilg~ L I l w h ich a! Iu l 1 l J L : " g r a . n : d into a coht:Ul. "r tl whole.ever, i F borh ["ype~ of data arcthen the nexr step might be the dJ1J~(l .d~m Magt:" As. d ~ ~ erib (;'d I ; 'JrherW ilcher t:rr al., 20 0 I), ~his , s tageccrrelaring [he q uam irarive data wid!quanti,tized (qua lirative) data. lheto un derra ke a correl ati 0'[1 e:l(~Sh i fdam t} 'pL :"~fin: collected for eachrn em ber, The dam corte L u~ oI! ~ l:lgrpropriate i f [he purpose of rhe


15/19

A framervork fut Analyzirig Dat l1+ ' " 3 7 7

l'....l D . ! :"! ! f~iO!f~

~~liI';;r.'I1!i1!JIiA. ~ ~(!!!I!ll.llliitl

I I ! " . . . . . Wg:ool~~~.Oa! iS I . . . . 1 1 1l..iiIli!l'i1i!!ff~LSiB.jON 1 L o , . . . EIl!ct Si . .al~I"

~t.nlfBI E o f f o c t= E ~ ~,L.aJm ~"'L Si ..

. IU


16/19

378 METHODOLOGICAL ANn ANAI.Y

rion, 1;\1 h e r e by inferences are m ad e. A snoted by Miller (Chapter 15, this volume).

The nature of (he im p! il" ': -'"--"T"'. ,r~{~, ,111l\:nl mniln,h)lng\' hc~ ;1I011g;H 1e,1rour dirnL'Il~1(111


17/19

DlSARU.1TY SCHO.LARSPeople \~ '.i[hdi!;abihlles have been .1~ 'edth rough \ ' as a p un is hmentfor sin oras a m eans of insp iring or redeem ing orh-ers, The medical model ' i C C ! o > the d i M I ~ i~l~~l~ , : 1 1 problem ur a measu ra b le deUec[ l o -ca ted in the im:!ividu 3.1rhar needs a cu re or~ ' l l 1 e \ 'I ~ . ~ ~ i o n rhar c a n h~ provided by mcdi-cal experts, ]n (he disahi~irYC'Olnmu.nity,Seelm sn (199!:l) descri bed a S I~it awa~ 'frn m [h e rn edi c~ I model til a r cssenti [lily fu-cu ~ cd 011 the in1 pa im CI'!:IH as a s ie kness ~I1J


18/19

ponst.:W indiv iduals who deviate fromrura l standards. i=~nhumo:fe, disabil-I~ viewed ;IS Or1~ d~mt:J]sion o : f hlJlmrln

. crence. Acrurdiflg [Q Gil~ (U999), the1 for people with disabilities is n1nt n~icare rhei r sicknes ~ b ut insrea d w cele-te thci r dist i n c r n e s s , purs L 1 e their equalace in fm lf.rica n soci ery, a nd ack nowl-d ,::u their dufocn:nwl::ss i~ nut defec-bur rather valua ble,


19/19

handbook of mixed methods in soc and beh reserach_tasshakkori_2003scan

Documents