validity and reliability of questionnaires

8/19/2019 Validity and Reliability of Questionnaires

1/38

UNIT 3 CONTD…..

VALIDITYAND

RELIABILITY OFQUESTIONNAIRES

BY – JASKARAN SINGH

JASKARAN SINGH


2/38

CONTENTS

Introduction

Steps in questionnaire designing

Validity Concept o !alidity

"ypes o !alidity

Steps in questionnaire !alidation

Relia#ility "ypes and $easure$ent o relia#ility

Conclusion

JASKARAN SINGH


3/38

INTRODUCTION

%uestionnaire& I$portant $et'od o datacollection used e(tensi!ely

Ad!antages o questionnaire )ess e(pensi!e

*+ers greater anony$ity

,isad!antages Application is li$ited

Response rate is lo-

*pportunities to clariy issues is lac.ing

JASKARAN SINGH


4/38

Ideal requisites o a questionnaire& S'ould #e clear and easy to understand )ayout is easy to read and pleasant to eye

Sequence o questions easy to ollo- S'ould #e de!eloped in an interacti!e style Sensiti!e questions $ust #e -orded

e(actly

JASKARAN SINGH


5/38

Steps in questionnaie!esi"nin"

JASKARAN SINGH


6/38

Va#i!it$

JASKARAN SINGH


7/38

T%e &on&ept o' (a#i!it$

Validity is t'e a#ility o an instru$ent to $easure -'at it isintended to $easure/

,egree to -'ic' t'e researc'er 'as $easured -'at 'e 'as

set out to $easure 0S$it'1 23324

Are -e $easuring -'at -e t'in. -e are $easuring50Kerlinger1 23674

8(tent to -'ic' an e$pirical $easure adequately re9ectst'e real $eaning o t'e concept under consideration0Ba##ie1 23:34

JASKARAN SINGH


8/38

;'y !alidity 5

Validity is done $ainly to ans-er t'e ollo-ingquestions&

Is t'e researc' in!estigation pro!iding ans-ers to t'eresearc' questions or -'ic' it -as underta.en5

I so1 is it pro!iding t'ese ans-ers using appropriate$et'ods and procedures5

JASKARAN SINGH


9/38

%uestions to ponder0"'in.a#out4

In!estigatorReaders oreport

8(perts in t'e


10/38

Lo"i&a# t%in)in"

Justi


11/38

Statisti&a# po&e!ues

By calculating coe>cient o correlations#et-een questions and outco$e!aria#les

JASKARAN SINGH


12/38

T$pes o' (a#i!it$

Validity

Content!alidity

@ace !alidity

Criterionrelated

Concurrent redicti!e

Construct!alidity

JASKARAN SINGH


13/38

CONTENT VALIDITY

ses logical reasoning and 'ence easy to apply

8(tent to -'ic' a $easuring instru$ent co!ers a

representati!e sa$ple o t'e do$ain o t'easpects $easured

;'et'er ite$s and questions co!er t'e ull range

o t'e issues or pro#le$ #eing $easured8(a$ple1 i -e -ant to test .no-ledge on

A$erican Geograp'y it is not air to 'a!e $ostquestions li$ited to t'e geograp'y o 8ngland/

JASKARAN SINGH


14/38

@ace Validity

"'e e(tent to -'ic' a $easuring instru$entappears !alid on its surace

8ac' question or ite$ on t'e researc' instru$ent$ust 'a!e a logical lin. -it' t'e o#=ecti!e

JASKARAN SINGH


15/38

CRITERION VALIDITY

"'e e(tent to -'ic' a $easuring instru$entaccurately predicts #e'a!iour or a#ility in a gi!enarea/

"'e $easuring instru$ent is called criteria?

It is o t-o types&

redicti!e !alidity Concurrent !alidity

JASKARAN SINGH


16/38

redicti!e !alidity

I t'e test is used to predict uture peror$ance

8g& 8ntrance e(a$ / / / / eror$ance o t'ese

tests correlates -it' later peror$ance inproessional college

8g& ;ritten dri!ing test

JASKARAN SINGH


17/38

Concurrent !alidity

I t'e test is used to esti$ate presentperor$ance or person?s a#ility at t'e presentti$e not atte$pting to predict uture outco$es

roessional college e(a$

8g& dri!ing test1 pilot test

JASKARAN SINGH


18/38

CONSTRUCT VALIDITY

Dost i$portant type o !alidity

Assesses t'e e(tent to -'ic' a $easuringinstru$ent accurately $easures a t'eoreticalconstruct it is designed to $easure

Deasured #y correlating peror$ance on t'e test-it' peror$ance on a test or -'ic' construct!alidity 'as #een deter$ined/

"'is can #e done e(peri$entally1 e/g/1 i -e -antto !alidate a $easure o an(iety/ ;e 'a!e a'ypot'esis t'at an(iety increases -'en su#=ectsare under t'e t'reat o an electric s'oc.1 t'ent'e t'reat o an electric s'oc. s'ould increase

an(iety scores/ JASKARAN SINGH


19/38

Anot'er $et'od is to s'o- t'at scores o t'e ne-test di+ers across people -it' di+erent le!els ooutco$es #eing $easured/

JASKARAN SINGH


20/38

Su**a$ o' Va#i!it$

CONTENT CRITERION CONSTRUCT

C*NCRR8N "

R8,IC"IV8

;'at it$easures

;'et'er t'etest co!ers a

representati!esa$ple o t'edo$ains to #e$easured

"'e a#ility ot'e test to

esti$atepresentperor$ance

"'e a#ility ot'e test to

predict utureperor$ance

"'e e(tent to-'ic' t'e

instru$ent$easures at'eoreticalconstruct

Ho- it isacco$plis'ed

As. e(perts toassess t'etest toesta#lis' t'att'e ite$s arerepresentati!eo t'eoutco$e

Correlateperor$anceon t'e test-it' aconcurrent#e'a!iour

Correlateperor$anceon t'e test-it' a#e'a!iour inuture

Correlateperor$anceon t'einstru$ent-it' aperor$anceon anesta#lis'edinstru$ent JASKARAN SINGH


21/38

Steps inquestionn

aie

(a#i!ation JASKARAN SINGH


22/38

FACE VALIDITY

8!aluate in ter$s o&

Reada#ility

)ayout andstyle

Clarity o-ording

@easi#ility

JASKARAN SINGH


23/38

CONTENT VALIDITY

"-o p'ases

Researc'er& ConceptualiEationand do$ain analysis0


24/38

+o, !o e-pets e(a#uate(a#i!it$ Det'od 2& A!erage Congruency ercentage

0AC4 Fop'a$1 236:

8(perts co$pute t'e percentage o questionsdee$ed to #e rele!ant or t'e$

"a.e t'e a!erage o all e(perts

I t'e !alue is 3 / / / Valid

8g& e(perts / / 08(pert 22L1 8(pert :L4

"'en AC M 3L

JASKARAN SINGH


25/38

CONSTRUCT VALIDITY

"o e(a$ine e$pirically t'e interrelations'ip a$ong ite$s andto identiy clusters o ite$s t'at s'are su>cient !ariation to =ustiy t'eir e(istence as a actor or construct to #e $easured#y t'e instru$ent

Det'od& @actor analysis

Various ite$s are gat'ered into co$$on actors

Co$$on actors are synt'esiEed into e-er actors and t'enrelation #et-een eac' ite$ and actor is $easured

nrelated ite$s are eli$inated

JASKARAN SINGH


26/38

Re#iai#it$

JASKARAN SINGH


27/38

RELIABILITY

,e


28/38

Relia#ility $easured inaspects o&

• , o n e t o e n s u r e t ' a t s a $ e r e s u l t s a r e o # t a i n e d - ' e n u s e d c o n s e c u t i ! e l y o r t - o o r $ o r e t i $ e s

• " e s t K r e t e s t $ e t ' o d i s u s e d

S"ABI)I"Y

• " o e n s u r e a l l s u # p a r t s o a i n s t r u $ e n t $ e a s u r e t ' e s a $ e c ' a r a c t e r i s t i c 0 H o $ o g e n e i t y 4

• S p l i t K ' a l $ e t ' o dIN"8RNA)

C*NSIS"8NC Y

• B s e d - ' e n t - o o # s e r ! e r s s t u d y a s i n g l e p ' e n o $ e n o n s i $ u l a t a n e o u s l y

• I n t e r K r a t e r r e l i a # i l i t y

8%IVA)8NC8

JASKARAN SINGH

T t R t t #i i#it


29/38

Test/Retest e#iai#it$0'o stai#it$1

"est ad$inistered t-ice to t'e sa$e participant atdi+erent ti$es

sed or t'ings t'at are sta#le o!er ti$e

8asy and straig'tor-ard approac'

seul or questionnaires1 c'ec.list1 rating scales etc

,isad!antages ractice e+ect 0$ainly or tests4

"oo s'ort inter!als in #et-een 0e+ect o $e$ory4

So$e traits $ay c'ange -it' ti$e JASKARAN SINGH


30/38

Statistical calculation

Ad$inistration o instru$ent to a sa$pleon t-o di+erent occasions

Scores co$pared and calculated #yusing correlation coe>cient or$ula0pearson4

JASKARAN SINGH


31/38

Correlation coe>cient

Deasures t'e degree o relations'ip #et-een t-osets o scores

Can range ro$ 2 to O2

indicates a#sence o any relations'ips

Coe#ation &oe2&ient Sten"t% o'e#ations%ip

O /6 to 2/ Strong

O /7 to /P3 Doderate

O / to /3 None to -ea.

JASKARAN SINGH

S #it % # #i i#it


32/38

Sp#it %a#(es e#iai#it$0%o*o"enit$1

8specially appropriate -'en t'e test is !ery long

Split t'e contents o t'e questionnaire into t-o equi!alent'al!esQ eit'er odde!en nu$#er or


33/38

R? M r2Or

0r M coe>cient o split 'al1 R? Mcoe>cient o entire test4

JASKARAN SINGH

I t t #i i#it


34/38

Inte/ate e#iai#it$0Equi(a#en&e1

InterRater Relia#ility reers to statistical $easure$entst'at deter$ine 'o- si$ilar t'e data collected #ydi+erent raters are/ A rater is so$eone -'o is scoringor $easuring a peror$ance1 #e'a!ior1 or s.ill in a

'u$an or ani$al/

sed -'en a single e!ent is $easured si$ultaneouslyand independently #y t-o or $ore trained o#ser!ers

R M Nu$#er o agree$ents

Nu$#er o agree$ents O Nu$#er o disagree$ents

JASKARAN SINGH


35/38

Su$$ary o Relia#ility

TESTRETEST

SLIT +ALF INTERRATER

;'at it

$easures

Sta#ility o!er

ti$e

8qui!alency o

ite$s

Agree$ent

#et-een raters

Ho- it isacco$plis'e

d

Ad$inistert'e sa$e test

to t'e sa$epeople at t-odi+erentti$es

Correlateperor$ance or

a group opeople on t-oequi!alent'al!es o sa$etest

Ha!e $ultipleresearc'ers

$easure sa$einstru$ent anddeter$inepercentage oagree$ent#et-een t'e$

JASKARAN SINGH


36/38

Conusion

Validated questionnaire It is one -'ic' 'as undergone a !alidation

procedure to s'o- t'at it accurately $easures-'at it ai$s to1 regardless o -'o responds1-'en t'ey respond1 and to -'o$ t'ey respondor -'en selad$inistered and -'ose relia#ility'as also #een e(a$ined t'ere#y&

Reducing #ias and a$#iguities

Better quality o data and credi#le inor$ation

JASKARAN SINGH


37/38

In a nuts%e## . . . .

A questionnaire can be reliablebut invalid . . .

But a valid questionnaire isalways reliable . . /

JASKARAN SINGH


38/38

T+AN4

YOU . .JASKARAN SINGH

validity and reliability of questionnaires

Documents