planning for a conversion to computer-based testing: avoiding the
TRANSCRIPT
![Page 1: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/1.jpg)
Planning for a Conversion to Computer-Based Testing:
Avoiding the Technology Trap
Presentation for theAssociation of Test Publishers
![Page 2: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/2.jpg)
Types of CBTLinear
Fixed form (page turner)Random item selection
Computer adaptive testing (CAT)
Testing on demand vs. fixed schedules
Basic Concepts
![Page 3: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/3.jpg)
Test center– Educational institutions
– Franchise centers
– Fully owned centers
Internet– Home/Office
– Other approved sites
Delivery Systems
![Page 4: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/4.jpg)
Convenient scheduling
Convenient location
More comfortable testing environment
Ability to retake the test sooner
Immediate test results
Benefits of CBT to Customers
![Page 5: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/5.jpg)
Distributed workload
Improved security
Easier to alter test forms
Ability to present a variety of item stimuli
Collection of candidate comments electronically
More precise measurement (CAT)
Benefits to Sponsor
![Page 6: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/6.jpg)
Increased item exposure
Expansion of the item bank to ensure a sufficient number of items of acceptable quality, difficulty and content
Scheduling of administrations
Higher program costs that may not be transferable to candidates
Challenges for the Sponsor
![Page 7: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/7.jpg)
Examination development
Program operations
Customer service
Marketing and education
Financial resources
Technical support
Factors to Consider in Planning a Conversion
![Page 8: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/8.jpg)
High vs. low stakes
Size of item pool
Quality of item pool
Content and statistical coverage
Volunteer resources needed
Linear vs. CAT
Examination Development
![Page 9: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/9.jpg)
New policies and procedures for: Registration
Examination security
Examination exposure
Record keeping and database management
Quality control and management
Program Operations
![Page 10: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/10.jpg)
More candidate and user inquiries
Different customer concerns
Staff training in new system
Loss of tight control - numerous sites and administrations
Increase in outsourcing, but service problems reflect on the sponsor
Customer Service
![Page 11: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/11.jpg)
Redo of marketing planEducational plan
Marketing and Education
![Page 12: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/12.jpg)
Startup costs - obvious and hidden
Maintenance costs
How to cover costs
Impact of conversion on volume (revenue)
Pricing strategies
Cash flow
Financial Resources
![Page 13: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/13.jpg)
Feasibility study
Expansion of item bank
Conversion of item bank
Staff planning and training
Piloting
Research and evaluation
Marketing/Candidate education
Customer service
Start-up Costs
![Page 14: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/14.jpg)
Trying to do too much at onceChanging psychometricians and introducing CBT at the same timeParallel administrations (CBT and P&P concurrently)Unrealistic timetablesAssuming volumes will increase or even stay the sameNot networking with sponsors that have taken the leap
Common Mistakes
![Page 15: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/15.jpg)
Initial planning and researchGather background information
Conduct market research
Perform feasibility analysis
Create business plan
Create marketing plan
Planning for CBT
![Page 16: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/16.jpg)
Select CBT vendor
Implement marketing plan
Restructure test development
Perform comparability studies
Conduct pilot testing
Develop policies and procedures
Orient staff and volunteers
Conduct post-launch analyses
Program Implementation
![Page 17: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/17.jpg)
Considering conversion:Automotive Service Excellence
Pharmacy Technician Certification Board
Undergoing conversion:National Council for Therapeutic Recreation Certification
Oncology Nursing Certification Program
Case Studies
![Page 18: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/18.jpg)
Complete Conversion:American Production and Inventory Control Society
Commission on Dietetic Registration
National Council of Architecture Registration Boards
Case Studies (cont’d.)
![Page 19: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/19.jpg)
Considering conversion:Use pilot test as a trial balloon
Do not put all your eggs in one vendor’s basket - keep your options open
Make sure your customers can afford CBT
Inform your Board of the pro’s and con’s
It’s good to be the late adopter
Lessons Learned
![Page 20: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/20.jpg)
Undergoing conversion:The first year is the hardest
Don’t believe everything your market tells you
Make sure you have healthy reserves
Be conservative in volume estimates
Lessons Learned (cont’d)
![Page 21: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/21.jpg)
Have completed conversion:Don’t offer delivery options
Make sure the system is tried and tested
Don’t take the vendors’ word for it -perform due diligence
Have quality control and management systems in place
Plan four years in advance
Lessons Learned (cont’d.)
![Page 22: Planning for a Conversion to Computer-Based Testing: Avoiding the](https://reader033.vdocument.in/reader033/viewer/2022052706/586cac4b1a28ab09738b4c0e/html5/thumbnails/22.jpg)
Plan carefully, build reserves and allow sufficient time
Understand what customers really want AND what they fear
Evaluate whether CBT can meet customer needs cost-effectively
Determine added value
Educate customers and market CBT
Keys to Success