Upload File

poster 0764-2017: multiple-group calibration in sas ...0 10 20 30 40 50 60 0 1 . -0 5 0 . -0 0.00...

  • Home
  • Documents
  • Poster 0764-2017: Multiple-Group Calibration in SAS ...0 10 20 30 40 50 60 0 1 . -0 5 0 . -0 0.00 0.05 0.10 0.15 0.20 Form Y / Discrimination Parameters Item s a Bi 0 10 20 30 40 50
4
Multiple-Group Calibration in SAS®: PROC IRT and SAS/IML® SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies.

Upload: others

Post on 23-Jul-2020

0 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Poster 0764-2017: Multiple-Group Calibration in SAS ...0 10 20 30 40 50 60 0 1 . -0 5 0 . -0 0.00 0.05 0.10 0.15 0.20 Form Y / Discrimination Parameters Item s a Bi 0 10 20 30 40 50

Multiple-GroupCalibrationinSAS®:PROCIRTandSAS/IML®

SASandallotherSASInstituteInc.productorservicenamesareregisteredtrademarksortrademarksofSASInstituteInc.intheUSAandothercountries.®indicatesUSAregistration.Otherbrandandproductnamesaretrademarksoftheirrespectivecompanies.

Page 2: Poster 0764-2017: Multiple-Group Calibration in SAS ...0 10 20 30 40 50 60 0 1 . -0 5 0 . -0 0.00 0.05 0.10 0.15 0.20 Form Y / Discrimination Parameters Item s a Bi 0 10 20 30 40 50

Multiple-GroupCalibrationinSAS®:PROCIRTandSAS/IML®Kyungyong Kim1,Seohee Park1,Jinah Choi1,andHongwook Seo2

1UniversityofIowaand2ACT

ABSTRACT• Itemresponsetheory(IRT)hasbeengainingpopularityinthefieldofeducationalandpsychologicalmeasurement.• IRTassumesthattheprobabilityofacorrectresponsetoanitemisafunctionofpersonanditemparameters.• InIRT,manyapplicationsrequireacommonabilityscale.However,theoriginandunitofmeasurementoftheability

scaleareundetermined.• IRTestimationprogramstypicallychoosetheabilityscalesothatmeanandSDofthepersonparametersare0and

1forthegroupathand.• Whenestimatingparametersfortwodifferenttestformswithnonequivalentgroups,thiscommonprocedureyields

parameterestimatesthatareontwodifferentabilityscales.• Multiple-groupcalibration,whichissupportedbytheIRTprocedureinSAS/STAT®(SASInstituteInc.,2013),isone

approachthatisoftenusedtohandlethisissue.• Toconductmultiple-groupcalibration,thetwotestformsmustsharesomecommonitems.• ThepurposeofthispaperistocomparetheperformanceofPROCIRT,amultiple-groupcalibrationprogramwritten

usingSAS/IML®,andacommercialsoftwareflexMIRT®(Cai,2013)intermsoftherecoveryofitemparameters.

METHODS• AsimulationstudywasconductedtocomparetheperformanceofPROCIRT,SAS/IML®,andflexMIRT®usingthe

twoparameterlogistic(2PL)model(Birnbaum,1968):𝑃"# 𝜃" 𝑎#, 𝑏# = 1/(1 + exp[1.7𝑎#(𝜃" − 𝑏#)]),

where𝑎# and𝑏# arethediscriminationanddifficultyparametersforitem𝑗,and𝜃" istheabilityparameterforperson𝑖.

• Thestudyfactorsforthesimulationstudywereasfollows:

• Therecoveryoftheitemparameterswereevaluatedusingthreestatistics:bias,standarderror(SE),androotmeansquarederror(RMSE).

RESULTS

SASandallotherSASInstituteInc.productorservicenamesareregisteredtrademarksortrademarksofSASInstituteInc.intheUSAandothercountries.®indicatesUSAregistration.Otherbrandandproductnamesaretrademarksoftheirrespectivecompanies.

Par. ProgramFormY Form X

Bias SE RMSE Bias SE RMSE

𝑎SAS®PROCIRT .027 .047 .055 .026 .045 .053

SAS/IML® .002 .048 .048 .000 .048 .048flexMIRT® -.012 .047 .049 -.016 .047 .050

𝑏SAS®PROCIRT -.225 .046 .232 -.260 .045 .266

SAS/IML® -.005 .050 .050 -.007 .051 .051flexMIRT® .009 .049 .052 .007 .050 .054

StudyFactor FormY/Group1(Reference) FormX/Group2

TestLength 40unique items+20commonitems 40unique items+20commonitems

Sample Size 3,000 3,000

AbilityDistribution N(0, 1) N(0.5,1)

Program SAS® PROCIRT SAS/IML® flexMIRT®

EstimationMethod MarginalMLE MarginalMLE Marginal MLENumericalIntegration Adaptive Gauss-Hermite Compositemidpoint Compositemidpoint

AbilityDistribution (BothGroups) Normal Empirical EmpiricalOptimizationMethod Quasi-Newton Newton-Raphson Newton-Raphson

Table1.Programcomparison

Note:ForSAS/IML®andflexMIRT®,thedistributionsofabilityforbothgroupsareestimatedempiricallywithoutassuminganyshapes.

Table2.Simulationresults

• Overall,thevaluesofbiasfortheitemparameterestimatesobtainedwithSAS®PROCIRTtendedtobelargerthanthoseobtainedwithSAS/IML®andflexMIRT®.

• Thistendencywasmorenoticeableforthe𝑏-parametersthanthe𝑎-parameters.• Asaresult,SAS®PROCIRTyieldeditemparameterestimateswiththelargestvaluesofRMSE.• ItemparameterestimatesobtainedwithSAS/IML®andflexMIRT®werecomparableintermsofallthreeevaluation

criteria.

Page 3: Poster 0764-2017: Multiple-Group Calibration in SAS ...0 10 20 30 40 50 60 0 1 . -0 5 0 . -0 0.00 0.05 0.10 0.15 0.20 Form Y / Discrimination Parameters Item s a Bi 0 10 20 30 40 50

RESULTSCONTINUED CONCLUSIONS/LIMITATIONS

REFERENCES

SASandallotherSASInstituteInc.productorservicenamesareregisteredtrademarksortrademarksofSASInstituteInc.intheUSAandothercountries.®indicatesUSAregistration.Otherbrandandproductnamesaretrademarksoftheirrespectivecompanies.

Multiple-GroupCalibrationinSAS®:PROCIRTandSAS/IML®Kyungyong Kim1,Seohee Park1,Jinah Choi1,andHongwook Seo2

1UniversityofIowaand2ACT

• TheperformanceofSAS/IML®andflexMIRT areverysimilarintermsoftherecoveryof𝑎-parameters.• SAS®PROCIRTyieldsinaccurate𝑏-parameterestimates.

è Formultiple-groupcalibration,themaindifferencebetweenSAS®PROCIRTandtheothertwoprogramsisthespecificationoftheabilitydistributionsduringtheestimationprocess.SAS®PROCIRTassumesabilitiesforbothgroupsfollowanormaldistribution,whereasSAS/IML®andflexMIRT®estimatetheabilitydistributionsconcurrentlywiththeitemparameters.

• SAS®PROCIRTrequiressignificantlymorecomputationtimethantheothertwoprograms.

Note:TheSAS/IML®programwrittenforthisstudyonlyprovidestheitemparameterestimatesasthefinaloutput.• InmostIRTestimationprograms,priordistributionscanbeassumedfortheitemparameterstoguarantee

convergence(i.e.,marginalizedBayesianestimation;Mislevy,1986).However,SAS®PROCIRTonlysupportsthemarginalmaximumlikelihoodestimationmethod(BockandAitkin,1981).

• Becauseofthesmallnumberofconditionsexaminedinthesimulationstudy,thisstudyislimitedinitsabilitytogeneralizethefindingstomeasurementconditionsthatarenotincludedinthesimulationstudy.

Birnbaum,A.(1968).Estimationofanability.InF.M.Lord&M.R.Novick (Eds.),Statisticaltheoriesofmentaltestscores(pp.423-479).Reading,MA:Addison-Wesley.Bock,R.D.,&Aitkin,M.(1981).Marginalmaximumlikelihoodestimationofitemparameters:ApplicationofanEMalgorithm.Psychometrika,46,443-459.Cai,L.(2017).flexMIRT®version3.51:Flexiblemultilevelmultidimensionalitemanalysisandtestscoring[Computersoftware].ChapelHill,NC:VectorPsychometricGroup.Mislevy,R.J. (1986).Bayesmodelestimationinitemresponsemodels.Psychometrika,51,177-195.SASInstituteInc.(2013).SAS/STAT®13.1User’sGuide:TheIRTprocedure[Computersoftware].Cary,NC.

ContactInformation:[email protected]

Figure1.Conditionalbias,SE,andRMSEvaluesforthe𝒂- and𝒃-parameters.Note:Inallfigures,theresultsforSAS®PROCIRTaredepictedwithorange,theresultsforSAS/IML® aredepictedwithgreen,andtheresultsforflexMIRT aredepictedwithred.

• SAS®PROCIRTsystematicallyoverestimatesthe𝑎-parametersandunderestimatesthe𝑏-parametersforallitemsinFormsXandY.

• Betweenthetwoitemparameters,SAS®PROCIRTproducedsignificantlylessaccurate𝑏-parameterestimatesintermsofboththeconditionalbiasandRMSEstatistics.

• SAS/IML®andflexMIRT yieldednearlyunbiasedestimatesforboththe𝑎- and𝑏-parameters.

0 10 20 30 40 50 60

-0.10-0.05

0.00

0.05

0.10

0.15

0.20

Form Y / Discrimination Parameters

Item

Bias

0 10 20 30 40 50 60

0.00

0.05

0.10

0.15

0.20

Form Y / Discrimination Parameters

Item

SE

0 10 20 30 40 50 60

0.00

0.05

0.10

0.15

0.20

Form Y / Discrimination Parameters

Item

RMSE

0 10 20 30 40 50 60

-0.4

-0.2

0.0

0.2

0.4

Form X / Difficulty Parameters

Item

Bias

0 10 20 30 40 50 60

0.0

0.1

0.2

0.3

0.4

Form X / Difficulty Parameters

Item

SE

0 10 20 30 40 50 60

0.0

0.1

0.2

0.3

0.4

Form X / Difficulty Parameters

Item

RMSE

Program SAS®PROCIRT SAS/IML® flexMIRT

ComputationTime 3minutes 0.75seconds 0.84seconds

Page 4: Poster 0764-2017: Multiple-Group Calibration in SAS ...0 10 20 30 40 50 60 0 1 . -0 5 0 . -0 0.00 0.05 0.10 0.15 0.20 Form Y / Discrimination Parameters Item s a Bi 0 10 20 30 40 50

SASandallotherSASInstituteInc.productorservicenamesareregisteredtrademarksortrademarksofSASInstituteInc.intheUSAandothercountries.®indicatesUSAregistration.Otherbrandandproductnamesaretrademarksoftheirrespectivecompanies.


Title 0:40

US/UK World Magnetic Model - Epoch 2015.0 Main Field ... · kj 8 0 6 0 40 20 0-20-40-60 - 8 0 6 0 4 0 20 0-2 0-4 0-6 0-8 0 8 0 6 0 4 0 2 0 0 - 2 0 - 4 0 - 6 0 60 40 20 0 - 0-4 0-6

0764 IPX Voice 220-00 (System Specifications) V1-3 2012-04-02

LITTLE FLOWERS PUBLIC SR.SEC. SCHOOLkumar shilpi 40 0 0 40 117 295 darsh mudgal 25-11-15 male mayank sharma akanksha 295 darsh mudgal 25-11-15 male sharma 40 0 0 40sharma40 0 0 40

VASANT KUNJ, NEW DELHI · 10 2186 aadit sinha rajesh ranjan amrita kumari sinha 40 0 0 0 0 40 11 346 aaditi beniwal sunil beniwal nupur kataria 40 0 0 0 0 40 12 1736 aaditri mondal

2020 PAGLA East Directives · This is the correct ref fee for the officials for 4 teams at the Round Robin. Ref 1 Ref 2 Ref 3 Ref 4 Total Team 1 $40 0 0 0 $40 Team 2 0 $40 0 0 $40

53k) 0:40 1 :oo 0:50 0:05 0.6km 0:40 0:20 1 :oo o:ð7 I .4km 4km … · 2018. 7. 26. · 53k) 0:40 1 :oo 0:50 0:05 0.6km 0:40 0:20 1 :oo o:ð7 I .4km 4km 1.4km 1.7km TEL025-784-3326

EJU Junior Ranking List · 41 90 ROU EJU TOMESCU Mircea 2000 40 0 40 40 42 103 ITA EJU BASIRICO Liborio 1998 40 0 40 0 2 16 2 0 2 0 40 43 104 ITA EJU ZELLI Cesare 1998 39 0 39 20

DELHI PUBLIC SCHOOL · 3 169 aadhir gupta mr. gaurav gupta mrs. meghna gupta 40 0 0 0 0 40 ... 8 1188 aadhya nagpal jatin nagpal anjali chauhan 40 0 0 0 0 40 ... 110 760 aarjav gupta

Delhi Public School...NEELAM BISHT RAWAT 40 0 0 0 0 40 82 1928 AARAV SINGH TOMAR NARSINGH TOMAR KERVEE TOMAR 40 0 0 0 0 40 83 2825 AARAV THAKUR MUKESH KUMAR KAMLA THAPA 40 0 0 0 0

Numbers 0 40

Greenpoint and COMMUNITY BROOKLYN DISTRICT 1 Williamsburg · 2020-07-15 · 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 0 500 1000 1500 2000 0 40 80 120 160 200 Asian Black

No. 4-10-0764 IN THE APPELLATE COURT OF ILLINOIS FOURTH

[XLS] · Web view3 138 0 138 129 9 0 138 0 138 3 103 0 103 0 103 0 103 0 103 3 84 0 84 0 84 0 84 0 84 3 40 0 40 38 2 40 0 40 0 3 141 0 141 0 141 0 141 0 141 3 106 0 106 0 106 0 106

Track Facts for Fair Grounds...23 George Leonard, III 3 2 0 0 66% 66% $21,690 24 Miguel Rodriguez 5 2 0 0 40% 40% $103,000 25 Paul J. McGee 5 2 0 0 40% 40% $55,800 26 Jason H. Meaux

0$+$-$1 2*34/$15)3+$€¦ · v"*+%"0?2"7%17 %43.&".40%1/%&0:"e60%1"0+'0="31'?" '"(.4%" 612a"0+%"&%401% "26"0+%"&.1&(% "02"0+%"72.40"26" .40%1/%&0.24

14 1 02 0 100 80 6 40 0 2 0 40 6 0 8 140 160 180 80 60 40 ... · 16.02.2014 · 1 0° 100° 80° 6 °40°2 0° 2 0° 40° 6 0° 8 120° 1 0 0° 80° 60° 40° 20° 6 14 °2 140°

0 .0. IN 40 XH

MAIN CONCOURSE DIRECTORY - National Football League€¦ · 1 0 2 0 3 0 4 0 5 0 4 0 3 0 2 0 1 0 10 20 30 40 50 40 30 20 10 1 0 2 0 3 0 5 0 4 0 3 0 2 0 1 0 4 0 10 20 30 40 50 40 30

Vol. 4, Issue 5, May 2015 Effectiveness of Ginger …a.12 – 13 yrs b.13 – 14 yrs c. 15 – 16 yrs 0 0 0 0 0 0 2 2 1 40 40 20 2 4 4 20 40 40 5 8 2 33 54 33 0 0 0 0 0 0 0 0 0 0 0

Tampa Yacht Manufacturing LLC. 44FCI - South Boats · arneson surface drives twin man r6-800 40 50 speed 0 10 70 50 speed 0 10 40 30 rpm x 100 0 10 40 30 rpm x 100 0 40 50 speed 0

Planning Permission: CR/2014/0764 – Condition 11 ... Permission: CR/2014/0764 – Condition 11: Recommendations within Generic Quantitative Risk Assessment (All Parcels) Technical

Track Facts for Fair Grounds...21 George Leonard, III 3 2 0 0 66% 66% $21,690 22 Miguel Rodriguez 5 2 0 0 40% 40% $103,000 23 Paul J. McGee 5 2 0 0 40% 40% $55,800 24 Jason H. Meaux

1 2 3 USNG 0 0.25 0.5 1 Miles 0 21 EU 40 39 40 41 0. 5 12

(800) 807-0764 2019 · VSP® Vision Care (800) 807-0764 AN ationwide PPO Vision Plan 2019 VSP Vision Care is available nationwide and overseas. Enrollment options for this plan:

ï890-0053 I PROFILE 6:00 ï899-4332 F) TEL.099-822-0764 8 ... · ï890-0053 i profile 6:00 ï899-4332 f) tel.099-822-0764 8:00 ï893-0009

AP Stats BW 9/17 1)Which set has the largest standard deviation? The smallest? a. 0 10 20 30 40 b. 0 0 20 40 40 c. 0 19 20 21 40 2)Without calculating,

INTERNATIONAL CONFERENCE CONTAMINATED SITES 2018 · 5 100 100 100 80 100 100 100 80 40 10 100 90 40 40 100 100 20 20 20 20 100 80 20 0 100 60 0 0 0 30 90 70 10 0 40 40 0 0 0 It has

IEC UNIVERSITY DEPARTMENT OF CHEMISTRY of...Inorganic Chemistry – IV 3 0 0 20 40 0 60 40 100 4 2. HBCH 502 Organic Chemistry-IV 3 0 0 20 40 0 60 40 100 4 3. HBCH 503 Physical Chemistry

MAIN CONCOURSE DIRECTORY · 2018-10-09 · club level directory 1 0 2 0 3 0 4 0 5 0 4 0 3 0 2 0 1 0 10 20 30 40 50 40 30 20 10 1 0 2 0 3 0 5 0 4 0 3 0 2 0 1 0 4 0 10 20 30 40 50 40

,;'#EO:)+@ "#$&+$:/&7#'%/&(#)1(&/#$&40@@$%$+7$&0+&#)1&;$);2$&'%$&'77$((0+3&/#$&$-.0/,&0+&/#$0%&#)*$(& /)4',&'(&7)*;'%$4&/)&/#$&;$%0)4&2$'40+3 " href="https://vdocument.in/-mn0eo-714070127703-000.html">storage.googleapis.com...M"#N0'#,=#>,;'#EO:)+@ "#$&+$:/&7#'%/&(#)1(&/#$&40@@$%$+7$&0+&#)1&;$);2$&'%$&'77$((0+3&/#$&$-.0/,&0+&/#$0%&#)*$(& /)4',&'(&7)*;'%$4&/)&/#$&;$%0)4&2$'40+3

UNIVERSIDAD NACIONAL MAYOR DE SAN ... - sistemas.unmsm… · 1 40 alumnos plan 2009 y plan 2014 1 0 1 0 40 40 0 0 g1 g2 g3 g3 40 0 0 g1 g2 g3 40 40 0 horario g1 1 1 40 40 0 1 1 40

0764) Volume 05 Issue 01, January 2016 Intelligent Cloud ... · International Journal of Computer and Information Technology (ISSN: 2279 – 0764) Volume 05 – Issue 01, January

ANNUAL GENERAL MEETING OCTOBER 2015€¦ · FY17 0.0 0.2 Forward rate 0.805 0.674 ... Q3 40% 85% Q4 40% 85% Q5 0% 50% Q6 0% 50% Q7 0% 40% Q8 0% 40% . Shareholder Returns Sustained

2014...matching Dental Shelf available. ** Shown in a steel door. Not available in fiberglass. PRIVACY 7 0764 / 2515* / 0764 Glass size: 2264 Door: Fiberglass DRS00 Kasumi Clear Bevels