broad-based testing
DESCRIPTION
Broad-Based Testing. Los Angeles County Office of Education PERSONNEL COMMISSION. The Problem. Many related classifications Differ on specific job content and levels of responsibility Many obstacles to class consolidation - logistical, political, fiscal Each class has a unique examination - PowerPoint PPT PresentationTRANSCRIPT
Broad-Based TestingBroad-Based Testing
Los Angeles County Office of EducationLos Angeles County Office of EducationPERSONNEL COMMISSIONPERSONNEL COMMISSION
The ProblemThe Problem
Many related classificationsMany related classifications Differ on specific job content and levels of responsibilityDiffer on specific job content and levels of responsibility Many obstacles to class consolidation - logistical, political, Many obstacles to class consolidation - logistical, political,
fiscalfiscal Each class has a unique examinationEach class has a unique examination
Variably content and qualityVariably content and quality Not consistently updatedNot consistently updated 2-3 month recruitment & examination cycle time2-3 month recruitment & examination cycle time
No overall competency architecture across the classesNo overall competency architecture across the classes Core competencies and competency progressionCore competencies and competency progression Pass point paradoxes (lower pass points for higher level Pass point paradoxes (lower pass points for higher level
jobs based on selection ratios)jobs based on selection ratios)
Purpose and ObjectivesPurpose and Objectives
Improve testing quality, reliability, and Improve testing quality, reliability, and efficiencyefficiency
Improve coherence of clerical examination Improve coherence of clerical examination plansplans
Reduce redundant testingReduce redundant testing Reduce time to place candidates on eligibility Reduce time to place candidates on eligibility
listslists Ensure eligibility lists are quickly availableEnsure eligibility lists are quickly available
StrategyStrategy
Core test battery for as many classifications as Core test battery for as many classifications as possiblepossible
Annual calendar of assessment opportunities (e.g., Annual calendar of assessment opportunities (e.g., quarterly)quarterly)
Computer-based testingComputer-based testing Score banking and certification to different exam plansScore banking and certification to different exam plans Different weights and cut-off scores for different Different weights and cut-off scores for different
classifications based upon relevancy and level neededclassifications based upon relevancy and level needed College-Board Model (SAT, GRE, etc.)College-Board Model (SAT, GRE, etc.)
Score Data FlowScore Data Flow
Appointment to Take Clerical
Battery(10 scores)
Exam Plan For Battery Score
Bank
Take Battery – CBT and Scanton
Apply to Job Title
Exam Plan For Job Title A 1
(weights and pass points for each
score)
Eligibilty List A
Electronic Certification of Scores to each Exam Plan3
Electronic Score Transfer 2
Exam Plan For Job Title B 1
(weights and pass points for each
score)
Eligibilty List B
Development of the Test BatteryDevelopment of the Test Battery
Reviewed job analyses from Agency-Wide Reviewed job analyses from Agency-Wide Classification Study - 2004-2007Classification Study - 2004-2007
Re-stated duties into “generalized work activity” (GWA) Re-stated duties into “generalized work activity” (GWA) statements with no reference to specific job content.statements with no reference to specific job content.
Map GWAs to each classificationMap GWAs to each classification Identify relevant assessment tools for GWAsIdentify relevant assessment tools for GWAs Via GWAs, map assessment tools to ClassificationsVia GWAs, map assessment tools to Classifications Derive relevancy and weight of assessment tools to Derive relevancy and weight of assessment tools to
classifications. classifications.
Matrix of GWA with ClassificationsMatrix of GWA with Classifications
ClassClass
GWAGWA
Typist ClerkTypist Clerk SecretarySecretary AccountingAccounting
ClerkClerk
ReceptionistReceptionist
Greet, interact, and Greet, interact, and provide information to provide information to others in person and others in person and on the telephoneon the telephone
11 22 11 22
Verify, input, and Verify, input, and correct data and correct data and informationinformation
22 11 22 00
Receive, sort, and Receive, sort, and categorize information categorize information and materialsand materials
22 22 22 11
Understand, interpret, Understand, interpret, explain, and follow explain, and follow directions, rules, and directions, rules, and regulationsregulations
22 11 22 11
Competencies Assessed - TestsCompetencies Assessed - Tests
1.1. Following Instructions (MC)Following Instructions (MC)
2.2. English Usage & Grammar (MC)English Usage & Grammar (MC)
3.3. Microsoft Outlook (MC)Microsoft Outlook (MC)
4.4. Microsoft Excel (Performance, OPAC)Microsoft Excel (Performance, OPAC)
5.5. Microsoft Word Skills (Performance, OPAC)Microsoft Word Skills (Performance, OPAC)
6.6. Telephone - Customer Service (SJ, OPAC)Telephone - Customer Service (SJ, OPAC)
7.7. Sequencing and ordering speed and accuracy (timed)Sequencing and ordering speed and accuracy (timed)
8.8. Computational speed and accuracy (timed)Computational speed and accuracy (timed)
9.9. Checking and comparing speed and accuracy (timed)Checking and comparing speed and accuracy (timed)
10.10. Data Entry speed and accuracy (timed, OPAC)Data Entry speed and accuracy (timed, OPAC)
Matrix of Tests with GWAsMatrix of Tests with GWAs
TestTest
GWAGWA
Telephone Telephone Customer Customer ServiceService
Data Entry Data Entry speed and speed and accuracyaccuracy
Sequencing Sequencing and ordering and ordering speed and speed and accuracyaccuracy
Following Following Written Written InstructionsInstructions
Greet, interact, and Greet, interact, and provide information to provide information to others in person and others in person and on the telephoneon the telephone
xx
Verify, input, and Verify, input, and correct data and correct data and informationinformation
xx xx
Receive, sort, and Receive, sort, and categorize information categorize information and materialsand materials
xx
Understand, interpret, Understand, interpret, explain, and follow explain, and follow directions, rules, and directions, rules, and regulationsregulations
xx
ClassClass
TestTest
Typist Typist ClerkClerk
SecretarySecretary AccountingAccounting
ClerkClerk
ReceptionistReceptionist
Telephone Telephone Customer Customer ServiceService
11 22 11 22
Data Entry Data Entry speed and speed and accuracyaccuracy
22 11 22 00
Sequencing and Sequencing and ordering speed ordering speed and accuracyand accuracy
22 22 22 11
Following Following Written Written InstructionsInstructions
22 11 22 11
Matrix of Tests with ClassificationsMatrix of Tests with Classifications
ResultResult
Linkage of tests-to-tasks-to-classifications to establish Linkage of tests-to-tasks-to-classifications to establish job relatednessjob relatedness
Derived weighting scheme for each classificationDerived weighting scheme for each classification Relevant testsRelevant tests Relevant weightsRelevant weights
Job specific competencies identified in the job analyses Job specific competencies identified in the job analyses are assessed in specialized examination partsare assessed in specialized examination parts Min QualsMin Quals TETE InterviewInterview PerformancePerformance
Applicable ClassificationsApplicable Classifications
1.1. ClerkClerk2.2. Intermediate ClerkIntermediate Clerk3.3. Senior ClerkSenior Clerk4.4. School ClerkSchool Clerk5.5. Senior School ClerkSenior School Clerk6.6. Data Control ClerkData Control Clerk7.7. Senior Data Control ClerkSenior Data Control Clerk8.8. Typist ClerkTypist Clerk9.9. Intermediate Typist ClerkIntermediate Typist Clerk10.10. Temporary Office WorkerTemporary Office Worker11.11. Senior Typist ClerkSenior Typist Clerk12.12. Department Assistant, DanceDepartment Assistant, Dance13.13. Department Assistant, MusicDepartment Assistant, Music
14.14. Department Assistant, TheaterDepartment Assistant, Theater15.15. Department Assistant, Visual ArtsDepartment Assistant, Visual Arts
16.16. ReceptionistReceptionist17.17. ReaderReader18.18. Information Resources SpecialistInformation Resources Specialist
19.19. Media Dispatching ClerkMedia Dispatching Clerk20.20. SecretarySecretary21.21. Division SecretaryDivision Secretary22.22. Legal SecretaryLegal Secretary23.23. School Administrative SecretarySchool Administrative Secretary
24.24. Senior Division SecretarySenior Division Secretary25.25. Executive Legal SecretaryExecutive Legal Secretary26.26. Executive AssistantExecutive Assistant
Affected PositionsAffected Positions
ClassificationClassification # LACOE Positions# LACOE Positions % of Clerical Positions% of Clerical Positions
Senior School ClerkSenior School Clerk 110110 27%27%
SecretarySecretary 7878 19%19%
Division SecretaryDivision Secretary 4646 11%11%
Senior Typist ClerkSenior Typist Clerk 3535 9%9%
School Administrative School Administrative SecretarySecretary
3030 7%7%
Intermediate Typist ClerkIntermediate Typist Clerk 2828 7%7%
Senior Division SecretarySenior Division Secretary 2424 6%6%
TOTALTOTAL 351351 87%87%
Administrative PracticesAdministrative Practices
Score LifeScore Life: : 2 years 2 years
Test Administration:Test Administration: Every 3 monthsEvery 3 months
Test RetakeTest Retake: : Once every 6 monthsOnce every 6 months
Retake Scope:Retake Scope: Whole Battery – not individual tests for logistical reasonsWhole Battery – not individual tests for logistical reasons
Weighted or Qualifying-OnlyWeighted or Qualifying-Only:: Qualifying Pass/Fail-only for Promotional ExamsQualifying Pass/Fail-only for Promotional Exams Weighted for Open recruitment Weighted for Open recruitment
Effective ScoresEffective Scores:: Only the candidate’s most recent score, not the highest Only the candidate’s most recent score, not the highest
score, can be certified to a list. score, can be certified to a list. For candidates whose scores enable them to be placed For candidates whose scores enable them to be placed
on an eligibility list (pass the exam), the certified scores on an eligibility list (pass the exam), the certified scores stands for the duration of eligibility on the respective list. stands for the duration of eligibility on the respective list.
• Scores already certified are not updated by retakesScores already certified are not updated by retakes• One’s score (and relative ranking) on a list can not be improved One’s score (and relative ranking) on a list can not be improved
until eligibility expires (one year).until eligibility expires (one year). Multiple (10) Pass PointsMultiple (10) Pass Points
Low individual test pass pointsLow individual test pass points Reduced Adverse ImpactReduced Adverse Impact 3 apparent factors: 3 apparent factors:
• Computer skills, speed and accuracy, verbalComputer skills, speed and accuracy, verbal
Administrative PracticesAdministrative Practices
Continuous offering of the BBCContinuous offering of the BBC Continuous recruitment for all affected classificationsContinuous recruitment for all affected classifications
Establishes a pre-tested pool of potential candidates for Establishes a pre-tested pool of potential candidates for the affected classesthe affected classes
For most classifications, limited or no public posting for For most classifications, limited or no public posting for job specific recruitmentsjob specific recruitments Notifications sent to pre-tested pool, based on test cut-off Notifications sent to pre-tested pool, based on test cut-off
score score Invitations to apply to take the specialized exam parts for Invitations to apply to take the specialized exam parts for
the classificationthe classification
RecruitmentRecruitment
No need to go to the general population for No need to go to the general population for each recruitmenteach recruitment
General
Population
Recruit To Take BBC
PRE-TESTEDBBC POOL
Recruit for Job B
Recruit for Job A
Recruit for Job C
Benefits Received and ExpectedBenefits Received and Expected
Test quality improvement throughTest quality improvement through Data monitoring and psychometric analysisData monitoring and psychometric analysis Empirical validation researchEmpirical validation research Development of alternate formsDevelopment of alternate forms
Administration efficiencies throughAdministration efficiencies through Managed efficiencies in test administration and score Managed efficiencies in test administration and score
managementsmanagements Minimizing retesting of the same candidates for different Minimizing retesting of the same candidates for different
classificationsclassifications Improved communications with candidatesImproved communications with candidates
Recruitment enhancement throughRecruitment enhancement through Maintaining a pre-tested pool of applicantsMaintaining a pre-tested pool of applicants Having something to offer at job faresHaving something to offer at job fares Reduced eligibility list production cycle timeReduced eligibility list production cycle time
Broad Based Administrative ClassesBroad Based Administrative Classes(Round 1)(Round 1)
ADMINISTRATIVE ANALYSTADMINISTRATIVE ANALYST ADMINISTRATIVE ADMINISTRATIVE
ASSISTANTASSISTANT ADMINISTRATIVE AIDEADMINISTRATIVE AIDE ASSISTANT ASSISTANT
ADMINISTRATIVE ANALYSTADMINISTRATIVE ANALYST RESOURCE & RESOURCE &
DEVELOPMENT ANALYSTDEVELOPMENT ANALYST PROJECT COORDINATORPROJECT COORDINATOR COMPENSATION ANALYSTCOMPENSATION ANALYST RESEARCH ANALYSTRESEARCH ANALYST LEGISLATIVE ANALYSTLEGISLATIVE ANALYST
BUDGET ANALYSTBUDGET ANALYST HEAD START PROGRAM HEAD START PROGRAM
RESULTS SPECIALISTRESULTS SPECIALIST HUMAN RESOURCES AIDEHUMAN RESOURCES AIDE HUMAN RESOURCES HUMAN RESOURCES
ANALYSTANALYST HS PROGRAM HS PROGRAM
DEVELOPMENT DEVELOPMENT SPECIALISTSPECIALIST
ASSISTANT HUMAN ASSISTANT HUMAN RESOURCES ANALYSTRESOURCES ANALYST
LABOR RELATIONS LABOR RELATIONS SPECIALISTSPECIALIST
Broad Based Administrative TestsBroad Based Administrative TestsComputer Written SJT TE
Gathering & Analyzing Data X X X
Attention to Detail X X
Critical Thinking & Problem Solving X
Mathematical Agility X
Using Technology X X X
Reading Comprehension X
Writing X X
Oral Communication* X
Informing X
Professional Impact X
Relationship Building X
Managing Performance X X
Developing Others X X
Project Management X X
Leadership X
Group Facilitation
Position Relevant Oral Exam (role-play exam).Presentation Skill
Thank you!Thank you!