student occupational competency assessment - pacta · student occupational competency assessment...

Pennsylvania Inspired Leadership Program

Student Occupational Competency

Assessment

Analyzing Data for Student Achievement

Pennsylvania Association of Career & Technical Administrators

October 7, 2008

Pennsylvania Association of Career & Technical Administrators Pennsylvania Inspired Leadership Program

Student Occupational Competency Assessment

Presentation Outline

1. Review and Analysis of NOCTI Results (12:30 p.m. – 1:30 p.m.) Page #

a. Begin with the End in Mind ................................................................................................. 3 b. Data Collection

i. Participant Score Report........................................................................................ 8 ii. Test Coordinator’s Score Report, Section 1 .......................................................... 9 iii. Test Coordinator’s Score Report, Section 2 ........................................................ 10

c. Data Compilation i. Annual Spreadsheet

1) Program .................................................................................................. 11 2) Student.................................................................................................... 12

ii. Historical Spreadsheet 1) Program .................................................................................................. 13 2) Student.................................................................................................... 14

d. Data Analysis i. Program Level Analysis

1) Local Norm v. National Norm.................................................................. 15 2) Percentage of Students by Proficiency Level ......................................... 16

ii. Competency Level Analysis 1) National Score ........................................................................................ 17 2) Local Score 3) Standard Error 4) Analysis Range—Assignment #2.......................................................... 18

e. Data Presentation i. Tables .................................................................................................................. 19 ii. Charts................................................................................................................... 21

2. Identification of Program Strengths & Deficiencies (1:30 p.m. – 2:15 p.m.)

a. Program Level Analysis i. Local Norm to National Norm

1) Above ...................................................................................................... 23 2) Below ...................................................................................................... 24 3) Long-term Trend—Rising, Declining, Up-and-down............................... 25

ii. Percentage of Students by Proficiency Level ...................................................... 26 b. Competency Level Analysis

i. Deficient Competencies ....................................................................................... 27 ii. Declining Competencies ...................................................................................... 27

c. Competency Analysis Relative to NOCTI Blueprint Percentages .................................... 28

Break (2:15 p.m. – 2:30 p.m.)

Page 1 of 40

Pennsylvania Inspired Leadership Program Student Occupational Competency Assessment Presentation Outline

3. Root Cause Analysis of Strengths and Deficiencies (2:30 p.m. – 3:15 p.m.) Page #

a. Techniques........................................................................................................................ 29 i. Ask ‘Why’ Five Times ii. Cause-and-Effect Diagram (Fishbone) iii. Interrelationship Diagram

b. Potential Sources of Variance (Deficiency) i. Student-Related Variables ................................................................................... 30

1) Number of Students 2) Number of Students with Disabilities

ii. Staff-Related Variables ........................................................................................ 31 iii. Curriculum-Related Variables

1) Alignment with NOCTI Blueprint a. Competency to Competency ..................................................... 33 b. Scope to Scope.......................................................................... 35

2) Sequence................................................................................................ 38

4. Strategies to Address Curricular & Instructional Deficiencies (3:15 p.m. – 4:00 p.m.)

a. Strategies Related to Curriculum i. Expand Curriculum—Drafting & Design ii. Reduce Curriculum—Drafting & Design iii. Re-sequence Curriculum—Health Assistant iv. Discount Deficiencies—Facilities Maintenance v. Change CIP—Electrical Engineering................................................................... 39

b. Strategies Related to Instruction i. Review ‘Older’ Competencies ii. Alter Instructional Methods

1) Projects 2) Group Activity 3) Theory or Laboratory 4) Integrated Academics

iii. Instructional Resources 1) Guest Speakers 2) Field Trip 3) Textbooks 4) Work-based Activity

5. Creating the Strategic Initiative for Each School (4:00 p.m. – 4:45 p.m.)

Page 2 of 40

Auto Body RepairWritten

1999

Dat

a, 4

5.5

1999

Dat

a, 4

7.7

2000

Dat

a, 5

1.7

2000

Dat

a, 4

7.8

2001

Dat

a, 5

4.1

2001

Dat

a, 4

7.8

2002

Dat

a, 5

4.5

2002

Dat

a, 4

8.1

2003

Dat

a, 5

4.5

2003

Dat

a, 4

8.6

2004

Dat

a, 5

2.5

2004

Dat

a, 4

8.6

2005

Dat

a, 5

9.2

2005

Dat

a, 4

8.9

2006

Dat

a, 6

0.2

2006

Dat

a, 4

8.9

2007

Dat

a, 6

4.9

2007

Dat

a, 5

9.9

2008

Dat

a, 5

2.2

2008

Dat

a, 5

5.8

0

10

20

30

40

50

60

70

Local Norm National Norm

NOCTI Scores

Local norms above national

norms andrising over time!

Page 3 of 40

Automotive TechnologiesWritten

-10

0

10

20

30

40

50

60

70

80

90

100

Competencies Tested

Perc

ent S

core

Group Norm--2006 54.9 54.7 53.1 58.0 53.2 78.5

Sig. Diff.--2006 0 0 0 0 0 0

Group Norm--2007 60.1 55.4 55.8 56.4 50.6 75.0

Sig. Diff.--2007 0 0 0 0 0 0

Group Norm--2008 61.5 90.0 85.0 76.0 73.9 84.2

Sig. Diff.--2008 0 0 0 0 0 0

Engine Repair Suspension Brakes Electrical Engine Perf Customer Rels

Competency scoreson the rise

and above the national norm

Page 4 of 40

Erie County Technical SchoolNOCTI Analysis

Percent of Students At or Above the National Norm

83%

82%

71%

100%

89% 93%

62% 71

%

89%

43% 50

%

50%

43%

82%

65%

56%

33%

0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Auto Bod

y Rep

air

Automoti

ve Tec

hnolo

gies

Early C

hildh

od E

duca

tion

Art & D

esign

for B

usine

ss

Compu

ter In

formati

on S

ystem

s

Constr

uctio

n Trad

esCos

metolog

y

Culina

ry Arts

(Tes

t #09

90 A

CF)Draf

ting &

Des

ign

Electric

al Eng

ineeri

ngElec

tronic

s

Facilit

y Main

tenan

ce Tec

hnolo

gies

Graphic

Com

munica

tions

Health

Ass

istan

t

Hospit

ality

Manag

emen

tMeta

l Fab

ricati

on

Network

ing Tec

hnolo

gies

Tool &

Die

Programs

Perc

ent o

f Stu

dent

s

Page 5 of 40

Drafting & DesignWritten

-10

0

10

20

30

40

50

60

70

80

90

100

Competencies Tested

Perc

ent S

core

Group Norm--2007 72.6 62.8 56.3 68.9 54.7 45.1

Sig. Diff.--2007 0 0 0 0 0 0

Group Norm--2008 72.6 62.8 56.3 68.9 54.7 45.1

Sig. Diff.--2008 0 0 0 0 0 0

Prepare to Draw- Basic Dimensioning

24%

Geometric Construction19%

Drawing Techniques13%

Applied Mathematics9%

Dimensioning Skills21%

Supplementary Views14%

Lowest Performing Competency Area

Supplementary Views

Page 6 of 40

Participant Written Assessment Score Report - Score reports are provided for each test taker and include the individual’s subpart/job scores and total scores along with comparative data for the group, site cumulative, site, state and nation. While separate reports are provided for written and performance assessments, both provide the same data.

2/1/2007 Michigan Test Taker A

A B C

3

D E F G

SecondaryNOCTI Test Site, 9999

Page 8 of 40

Test Coordinator’s Written Report - The Test Coordinator Report is a summary of the assessment results for all test takers within a particular group. Participants are listed alphabetically with individual sub-scores and total scores displayed. This report contains a summary for the written assessment results for the designated group of test takers.

Test Taker A

Test Taker B

Test Taker E

Test Taker D

Test Taker C

A

B C D

2/1/2007

NOCTI Test Site, 9999

Michigan

5

Secondary

Page 9 of 40

Test Coordinator’s Written Assessment Comparative Data Report - The comparative data report provides a wealth of information that can be helpful in making inferences about the observed assessment results/scores. The report displays group, sitecumulative, site, state and nation user-based normative information for both the written and performance assessments.

G

F

E

D

C

B

A

NOCTI Test Site, 9999

7

2/1/2007

Secondary Michigan

Page 10 of 40

Program Analysis Erie County Technical SchoolNOCTI Analysis

Class of 2008

9/24/2008

Program 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 TotalAuto Body Repair NEW TEST FOR 2007

Written Test n= 6Group Norm 50.6 54.6 39.6 70.6 53.9National Norm 56.9 55.2 46.1 67.1 56.3Std. Error 4.8 5.3 5.2 5 5.1Range--Low 52.1 49.9 40.9 0 51.3Range--High 61.7 60.5 51.3 72.1 61.4Within or Above Range FALSE TRUE FALSE TRUE TRUEVariance from Mean -6.3 -0.6 -6.5 3.5 -2.5

Sig. Diff.--National -6.3 0 -6.5 0 0.0

Automotive TechnologiesWritten Test n= 11

Group Norm 61.5 90 85 76 73.9 84.2 78.4National Norm 62.5 51.9 46.7 46.5 52.8 50.6 51.8Std. Error 4.9 5.9 5.2 5.1 5.9 5.7 5.5Range--Low 57.6 46 41.5 41.4 46.9 44.9 46.4Range--High 67.4 57.8 51.9 51.6 58.7 56.3 57.3Within or Above Range TRUE TRUE TRUE TRUE TRUE TRUE TRUEVariance from Mean -1.0 38.1 38.3 29.5 21.1 33.6 26.6

Sig. Diff.--National 0 0 0 0 0 0 0.0

Early Childhood Education NEW TEST FOR 2007Written Test n= 17

Group Norm 75 90 70.8 81.3 83.8 75.7 75.7 75 86.3 79.2 77.5 67.3 90.6 79.1National Norm 75.4 81.5 58.1 78 78.4 70.7 69 71.7 78.3 69.1 67.9 68.6 84 73.1Std. Error 4.1 4.3 5.7 3.9 4.7 4.9 5.6 6.1 5.7 6.7 6.7 6 6.2 5.4Range--Low 71.3 77.2 52.4 74.1 73.7 65.8 63.4 65.6 72.6 62.4 61.2 62.6 77.8 67.7Range--High 79.5 85.8 63.8 81.9 83.1 75.6 74.6 77.8 84 75.8 74.6 74.6 90.2 78.6Within or Above Range TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUEVariance from Mean -0.4 8.5 12.7 3.3 5.4 5.0 6.7 3.3 8.0 10.1 9.6 -1.3 6.6 6.0

Sig. Diff.--National 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Competency Area Number

Page 1Page 11 of 40

Student Analysis Erie County Technical SchoolNOCTI Analysis

Class of 2008

9/24/2008

ProgramStudent Scores Written Written Written Written Written Written Written

1 56.9 79.2 82.1 63.4 32.6 48.3 70.9 2 52.0 59.2 71.1 77.7 65.1 64.9 74.6 3 22.8 60.8 73.7 79.4 45.0 70.9 60.5 4 57.7 70.0 54.7 74.9 62.8 76.2 79.9 5 61.8 44.2 81.1 74.3 62.8 54.3 76.9 6 61.8 58.3 83.7 73.7 62.0 68.2 59.7 7 61.7 89.5 69.1 53.5 72.2 70.9 8 66.7 86.8 63.6 60.3 59.0 9 50.8 80.5 58.9 55.6 73.9

10 65.0 71.6 26.5 76.9 11 53.3 67.4 47.7 80.6 12 87.4 65.6 56.0 13 77.9 66.2 45.5 14 68.4 62.9 15 68.4 16 74.7 17 67.9

ADV 65.0 54.0 83.6 65.7 48.2 71.0 73.2 COM 60.0 49.0 78.6 60.7 43.2 66.0 68.2 BAS 50.0 39.0 68.6 0.1 33.2 56.0 58.2

Local Norm 52.2 60.8 75.7 73.2 56.3 60.0 68.1National Norm 55.8 51.1 73.9 63.3 50.7 54.8 69.3Std. Error 4.4 4.5 3.9 4.2 4.7 4.3 3.7Range--Low 51.4 46.6 70.0 59.1 46.0 50.5 65.6Range--High 60.2 55.6 77.8 67.5 55.4 59.1 73.0n= 6.0 11.0 17.0 7.0 9.0 14.0 13.0

# At or Above Nat. Ave. 5.0 9.0 12.0 7.0 8.0 13.0 8.0# Advanced 0 8 4 6 7 2 6# Competent 2 2 3 1 1 3 2

# Basic 4 1 10 0 1 9 5Advanced & Competent 2 10 7 7 8 5 8

% Above Nat. Ave. 83% 82% 71% 100% 89% 93% 62%

Auto Body Repair (2083)

Automotive Technologies

(4008)

Early Childhood Education

(3016)Art & Design

(3019)Construction Trades (3015)

Computer Information

Systems (3023)Cosmetology

(2082)

Page 1Page 12 of 40

Program Analysis Erie County Technical SchoolNOCTI Analysis--2008

9/24/2008

Program 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Competency Area Number

Auto Body Repair

Ref

inis

hing

Det

ailin

g

Cor

rosi

on

Safe

ty/E

nviro

n

Written TestGroup Norm--2006 53.6 71 70 59 57.1 84.6 40 37.5 40 70National Norm--2006 48 59 48.1 49 44.8 62.4 48 40.4 22.8 44.3Sig. Diff.--2006 0 0 0 0 0 0 -8 0 0 0Group Norm--2007 65.8 48.8 54.5 78.1National Norm--2007 54 42.2 47.7 64.7Sig. Diff.--2007 0 0 0 0Group Norm--2008 50.6 54.6 39.6 70.6National Norm--2008 56.9 55.2 46.1 67.1Sig. Diff.--2008 -6.3 0 -6.5 0


Engi

ne R

epai

r

Susp

ensi

on

Bra

kes

Elec

trica

l

Engi

ne P

erf

Cus

tom

er R

els

Written TestGroup Norm--2006 54.9 54.7 53.1 58.0 53.2 78.5National Norm--2006 55.3 51.5 52 55.6 46.2 74.1Sig. Diff.--2006 0 0 0 0 0 0Group Norm--2007 60.1 55.4 55.8 56.4 50.6 75.0National Norm--2007 55 51.5 52.2 55.2 45.7 74.6Sig. Diff.--2007 0 0 0 0 0 0Group Norm--2008 61.5 90.0 85.0 76.0 73.9 84.2National Norm--2008 62.5 51.9 46.7 46.5 52.8 50.6Sig. Diff.--2008 0 0 0 0 0 0

Page 13 of 40

Student Analysis Erie County Technical SchoolNOCTI Analysis--2008

Percentage of Students Performing 'At or Above' the National Norm

9/24/2008


Art & Design for Business

Auto BodyRepair


Construction Trades

Early Childhod Education Cosmetology


2004 DataLocal Norm 52.5 75.3 70.5 48.8 68.6 74.1National Norm 48.6 70.4 0.0 46.9 63.5 68.3

# At or Above Nat. Ave. 5 0 8 10 7 15 10% At or Above Nat. Ave. 83% 89% n/a 78% 88% 91%

2005 DataLocal Norm 59.2 55.2 70.8 61.7 53.0 81.0 67.8National Norm 48.9 52.8 70.3 62.5 45.8 63.4 68.3

# At or Above Nat. Ave. 8 10 8 7 10 18 9% At or Above Nat. Ave. 100% 77% 80% 78% 83% 100% 75%







Page 14 of 40


1999

Dat

a, 4

5.5

1999

Dat

a, 4

7.7

2000

Dat

a, 5

1.7

2000

Dat

a, 4

7.8

2001

Dat

a, 5

4.1

2001

Dat

a, 4

7.8

2002

Dat

a, 5

4.5

2002

Dat

a, 4

8.1

2003

Dat

a, 5

4.5

2003

Dat

a, 4

8.6

2004

Dat

a, 5

2.5

2004

Dat

a, 4

8.6

2005

Dat

a, 5

9.2

2005

Dat

a, 4

8.9

2006

Dat

a, 6

0.2

2006

Dat

a, 4

8.9

2007

Dat

a, 6

4.9

2007

Dat

a, 5

9.9

2008

Dat

a, 5

2.2

2008

Dat

a, 5

5.8

0

10

20

30

40

50

60

70


NOCTI Scores

Page 15 of 40

Erie County Technical SchoolNOCTI Analysis

Percent of Students At or Above the National Norm

83%

82%

71%

100%

89% 93%

62% 71

%

89%

43% 50

%

50%

43%

82%

65%

56%

33%

0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Auto Bod

y Rep

air

Automoti

ve Tec

hnolo

gies

Early C

hildh

od E

duca

tion

Art & D

esign

for B

usine

ss

Compu

ter In

formati

on S

ystem

s

Constr

uctio

n Trad

esCos

metolog

y

Culina

ry Arts

(Tes

t #09

90 A

CF)Draf

ting &

Des

ign

Electric

al Eng

ineeri

ngElec

tronic

s

Facilit

y Main

tenan

ce Tec

hnolo

gies

Graphic

Com

munica

tions

Health

Ass

istan

t

Hospit

ality

Manag

emen

tMeta

l Fab

ricati

on

Network

ing Tec

hnolo

gies

Tool &

Die

Programs

Perc

ent o

f Stu

dent

s

Programs PerformingAt or Above

Your Benchmark

Page 16 of 40

Assignment #2

More than Numbers: Working with the Score Report

Competency Area Titles

(Record below)

% of NOCTI Test for

Competency Area

Nation Mean for

Competency Area

Standard Error

for Competency

Area

Nation Mean minus

Standard Error

Nation Mean plus

Standard Error

Range for Competency

Area

Group Mean for

Competency Area

Analysis: Group Mean

Within, Above or

Below Range?

Practice #1 (from Workplace Readiness Sample) Communications

24%

Practice #2 Problem Solving and Critical Thinking

19%

Competency 1:

Competency 2:

Competency 3:

Page 18 of 40

Program Analysis Erie County Technical SchoolNOCTI Analysis--2008

9/24/2008

Program 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Competency Area Number

Auto Body Repair

Ref

inis

hing

Det

ailin

g

Cor

rosi

on

Safe

ty/E

nviro

n

Written TestGroup Norm--2006 53.6 71 70 59 57.1 84.6 40 37.5 40 70National Norm--2006 48 59 48.1 49 44.8 62.4 48 40.4 22.8 44.3Sig. Diff.--2006 0 0 0 0 0 0 -8 0 0 0Group Norm--2007 65.8 48.8 54.5 78.1National Norm--2007 54 42.2 47.7 64.7Sig. Diff.--2007 0 0 0 0Group Norm--2008 50.6 54.6 39.6 70.6National Norm--2008 56.9 55.2 46.1 67.1Sig. Diff.--2008 -6.3 0 -6.5 0


Engi

ne R

epai

r

Susp

ensi

on

Bra

kes

Elec

trica

l

Engi

ne P

erf

Cus

tom

er R

els

Written TestGroup Norm--2006 54.9 54.7 53.1 58.0 53.2 78.5National Norm--2006 55.3 51.5 52 55.6 46.2 74.1Sig. Diff.--2006 0 0 0 0 0 0Group Norm--2007 60.1 55.4 55.8 56.4 50.6 75.0National Norm--2007 55 51.5 52.2 55.2 45.7 74.6Sig. Diff.--2007 0 0 0 0 0 0Group Norm--2008 61.5 90.0 85.0 76.0 73.9 84.2National Norm--2008 62.5 51.9 46.7 46.5 52.8 50.6Sig. Diff.--2008 0 0 0 0 0 0

Page 19 of 40

Student Analysis Erie County Technical SchoolNOCTI Analysis--2008

Percentage of Students Performing 'At or Above' the National Norm

9/24/2008


Art & Design for Business

Auto BodyRepair


Construction Trades

Early Childhod Education Cosmetology


2004 DataLocal Norm 52.5 75.3 70.5 48.8 68.6 74.1National Norm 48.6 70.4 0.0 46.9 63.5 68.3

# At or Above Nat. Ave. 5 0 8 10 7 15 10% At or Above Nat. Ave. 83% 89% n/a 78% 88% 91%









Page 20 of 40

Construction Trades--Written

1999

Dat

a, 5

6.2

1999

Dat

a, 8

1.2

2000

Dat

a, 4

9.4

2000

Dat

a, 4

9.0

2001

Dat

a, 6

5.6

2001

Dat

a, 6

3.1

2002

Dat

a, 6

3.8

2002

Dat

a, 6

3.3

2003

Dat

a, 7

3.1

2003

Dat

a, 6

3.4

2004

Dat

a, 6

8.6

2004

Dat

a, 6

3.5

2005

Dat

a, 8

1.0

2005

Dat

a, 6

3.4

2006

Dat

a, 7

0.6

2006

Dat

a, 6

3.2

2007

Dat

a, 4

8.2

2007

Dat

a, 4

3.2

2008

Dat

a, 6

0.0

2008

Dat

a, 5

4.8

0

10

20

30

40

50

60

70

80

90


NOCTI Scores

Page 21 of 40

Facilities Maintenance TechnologiesWritten

-30

-20

-10

0

10

20

30

40

50

60

70

80

Competencies Tested

Perc

ent S

core

Group Norm--2006 42.1 79.0 65.0 60.0 55.7 66.7 72.5 73.3 44.3

Sig. Diff.--2006 -9.5 0 0 0 0 0 0 0 0.0

Group Norm--2007 36.7 48.5 56.7 38.3 33.3 38.9 40.0 51.1 36.4

Sig. Diff.--2007 -14 0 0 -19.7 -9.9 -20.3 -14.5 -8.7 -11.8

Group Norm--2008 46.2 71.1 58.0 55.6 42.9 63.6 60.4 65.3 43.8

Sig. Diff.--2008 0 0 0 0 0 0 0 0 0

General Maintenance- Welding, Glazing Electrical Plumbing Painting/Wall Covering

HVAC/Sheet Metal; Power Plant/Boiler Masonry

Grounds & Equipment Maint;

Hydraulic/PneumaticSafety and Tools Carpentry

Page 22 of 40


1999

Dat

a, 4

5.5

1999

Dat

a, 4

7.7

2000

Dat

a, 5

1.7

2000

Dat

a, 4

7.8

2001

Dat

a, 5

4.1

2001

Dat

a, 4

7.8

2002

Dat

a, 5

4.5

2002

Dat

a, 4

8.1

2003

Dat

a, 5

4.5

2003

Dat

a, 4

8.6

2004

Dat

a, 5

2.5

2004

Dat

a, 4

8.6

2005

Dat

a, 5

9.2

2005

Dat

a, 4

8.9

2006

Dat

a, 6

0.2

2006

Dat

a, 4

8.9

2007

Dat

a, 6

4.9

2007

Dat

a, 5

9.9

2008

Dat

a, 5

2.2

2008

Dat

a, 5

5.8

0

10

20

30

40

50

60

70


NOCTI Scores

Local NormAbove

National Norm

Page 23 of 40

Networking Technologies--Written

2005

Dat

a, 5

5.9

2005

Dat

a, 6

2.9

2006

Dat

a, 4

6.3

2006

Dat

a, 6

3.2

2007

Dat

a, 4

6.3

2007

Dat

a, 6

3.2

2008

Dat

a, 5

3.5

2008

Dat

a, 5

7.7

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0


NOCTI Scores

Local NormBelow

National Norm

Page 24 of 40

Drafting & Design--Written

1999

Dat

a, 6

2.0

1999

Dat

a, 7

5.7

2000

Dat

a, 5

2.4

2000

Dat

a, 5

5.0

2001

Dat

a, 5

7.1

2001

Dat

a, 5

6.2

2002

Dat

a, 5

8.0

2002

Dat

a, 5

5.1

2003

Dat

a, 5

2.7

2003

Dat

a, 5

4.7

2004

Dat

a, 5

7.0

2004

Dat

a, 5

4.6

2005

Dat

a, 5

4.0

2005

Dat

a, 5

4.6

2006

Dat

a, 5

9.6

2006

Dat

a, 5

4.5

2007

Dat

a, 6

0.5

2007

Dat

a, 5

4.5

2008

Dat

a, 6

8.8

2008

Dat

a, 5

9.6

0

10

20

30

40

50

60

70

80


NOCTI Scores

Up-and-DownLong-term

Trend

Page 25 of 40

Erie County Technical SchoolProficiency Level Percentage

2007-2008

0%

25%

50%

75%

100%

Auto B

ody R

epair

(208

3)

Automoti

ve Tec

hnolo

gies (

4008

)

Early C

hildh

ood E

duca

tion (

3016

)

Art & D

esign

(301

9)

Compu

ter In

formati

on S

ystem

s (30

23)

Constr

uctio

n Trad

es (3

015)

Cosmeto

logy (

2082

)

Culi

nary

Arts (4

136 L

evel

2) KIP

S

Culi

nary

Arts (0

990)

ACFF

Draftin

g & D

esign

(303

8)

Electric

al Eng

ineeri

ng (3

030)

Electro

nics

(303

5)

Facilit

ies M

ainten

ance

Techs

(201

2)

Graphic

Com

munica

tions

(414

2)

Health

Ass

istan

t (305

8)

Touris

m & H

ospit

ality

Mgmt. (

2080

)

Metal F

abric

ation

(307

2)

Network

ing Tec

hnolo

gies (

2414

)

Tool &

Die-

PRECIS

ION M

ACHININ

G

Technical Training Programs

Perc

ent P

rofic

ient

% Advanced % Competent % Basic

Page 26 of 40


-30

-20

-10

0

10

20

30

40

50

60

70

80

Competencies Tested

Perc

ent S

core

Group Norm--2006 42.1 79.0 65.0 60.0 55.7 66.7 72.5 73.3 44.3

Sig. Diff.--2006 -9.5 0 0 0 0 0 0 0 0.0

Group Norm--2007 36.7 48.5 56.7 38.3 33.3 38.9 40.0 51.1 36.4

Sig. Diff.--2007 -14 0 0 -19.7 -9.9 -20.3 -14.5 -8.7 -11.8

Group Norm--2008 46.2 71.1 58.0 55.6 42.9 63.6 60.4 65.3 43.8

Sig. Diff.--2008 0 0 0 0 0 0 0 0 0





Declining Competency

Deficient Competencies

Page 27 of 40

Drafting & DesignWritten

-10

0

10

20

30

40

50

60

70

80

90

100

Competencies Tested

Perc

ent S

core

Group Norm--2007 72.6 62.8 56.3 68.9 54.7 45.1

Sig. Diff.--2007 0 0 0 0 0 0

Group Norm--2008 72.6 62.8 56.3 68.9 54.7 45.1

Sig. Diff.--2008 0 0 0 0 0 0

Prepare to Draw- Basic Dimensioning

24%

Geometric Construction19%

Drawing Techniques13%

Applied Mathematics9%

Dimensioning Skills21%

Supplementary Views14%

Competency AreasRelative to

NOCTI Blueprint Percentages

Page 28 of 40

Root Cause Analysis

• Ask “Why?” Five Times • Cause-and-Effect Diagram (Fishbone)

• Interrelationship Diagram

You are looking for ‘causes of variance’

from the norm or the expected

Page 29 of 40


-30

-20

-10

0

10

20

30

40

50

60

70

80

Competencies Tested

Perc

ent S

core

Group Norm--2006 42.1 79.0 65.0 60.0 55.7 66.7 72.5 73.3 44.3

Sig. Diff.--2006 -9.5 0 0 0 0 0 0 0 0.0

Group Norm--2007 36.7 48.5 56.7 38.3 33.3 38.9 40.0 51.1 36.4

Sig. Diff.--2007 -14 0 0 -19.7 -9.9 -20.3 -14.5 -8.7 -11.8

Group Norm--2008 46.2 71.1 58.0 55.6 42.9 63.6 60.4 65.3 43.8

Sig. Diff.--2008 0 0 0 0 0 0 0 0 0





Potential Sources of Deficiency--

Student-Related Variables

Page 30 of 40

Drafting & Design--Written

1999

Dat

a, 6

2.0

1999

Dat

a, 7

5.7

2000

Dat

a, 5

2.4

2000

Dat

a, 5

5.0

2001

Dat

a, 5

7.1

2001

Dat

a, 5

6.2

2002

Dat

a, 5

8.0

2002

Dat

a, 5

5.1

2003

Dat

a, 5

2.7

2003

Dat

a, 5

4.7

2004

Dat

a, 5

7.0

2004

Dat

a, 5

4.6

2005

Dat

a, 5

4.0

2005

Dat

a, 5

4.6

2006

Dat

a, 5

9.6

2006

Dat

a, 5

4.5

2007

Dat

a, 6

0.5

2007

Dat

a, 5

4.5

2008

Dat

a, 6

8.8

2008

Dat

a, 5

9.6

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0


NOCTI Scores

Potential Sources ofDeficiency--Staff-Related

Variables

Health Condition

Page 31 of 40

Tool & Die--Written

1999

Dat

a, 4

8.6

1999

Dat

a, 5

1.6

2000

Dat

a, 6

0.0

2000

Dat

a, 5

1.9

2001

Dat

a, 6

1.7

2001

Dat

a, 5

1.6

2002

Dat

a, 5

8.4

2002

Dat

a, 5

2.1

2003

Dat

a, 6

6.5

2003

Dat

a, 5

2.6

2004

Dat

a, 6

7.9

2004

Dat

a, 5

2.6

2005

Dat

a, 6

7.9

2005

Dat

a, 5

3.1

2006

Dat

a, 7

4.4

2006

Dat

a, 6

7.6

2007

Dat

a, 6

5.6

2007

Dat

a, 6

7.6

2008

Dat

a, 4

8.2

2008

Dat

a, 6

0.8

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0


NOCTI Scores

Potential Sources ofDeficiency--Staff-Related

VariablesNew Teacher

Page 32 of 40

Collision Repair And Refinishing Technology

Specific Competencies and Skills Tested in this Assessment:

Structural

• Select, setup, and adjust welders • Describe various types and uses of

welding processes • Use adhesive bonding procedures • Analyze primary and secondary

damage • Replace and/or repair structural

components Non-Structural

• Identify automotive plastics and proper repair procedures • Remove and replace automotive glass and trim • Remove, install, replace, and align non-structural panels • Remove, install, and replace components

Painting and Refinishing

• Identify and demonstrate surface preparation techniques • Identify and demonstrate paint and materials preparation techniques • Identify and demonstrate paint and materials application techniques • Identify painting and refinishing safety/environmental requirements

Mechanical and Electrical Systems

• Identify, remove, and install/replace components • Identify basic suspension geometry • Describe the terms voltage, current, and resistance (fundamental

electrical theory) • Perform basic and advanced electrical diagnostic operations

Safety and Environmental Issues

• Demonstrate knowledge of safety/environmental requirements in the collision repair industry

• Identify proper safety techniques for the use of shop equipment

Page 1 of 5 NOCTI Job Ready Assessment Copyright © NOCTI 2007 All Rights Reserved Page 33 of 40

Assignment #4 How Close Are They- NOCTI and my Curriculum?

Aligning Your Program with NOCTI Competencies

Auto Body Technology Test Code 2083 Check the NOCTI competencies below against your Duty/Task List; if your program's tasks are numbered, indicate the corresponding number in the column to the left of the NOCTI competency. Highlight any NOCTI competency on this list that you can not find in your curriculum. Specific Competencies and Skills Tested in this Assessment: Structural Select, setup, and adjust welders Describe various types and uses of welding processes Use adhesive bonding procedures Analyze primary and secondary damage Replace and/or repair structural components Non-Structural Identify automotive plastics and proper repair procedures Remove and replace automotive glass and trim Remove, install, replace, and align non-structural panels Remove, install, and replace components Painting and Refinishing Identify and demonstrate surface preparation techniques Identify and demonstrate paint and materials preparation techniques Identify and demonstrate paint and materials application techniques Identify painting and refinishing safety/environmental requirements Mechanical and Electrical Systems Identify, remove, and install/replace components Identify basic suspension geometry Describe the terms voltage, current, and resistance (fundamental electrical

theory) Perform basic and advanced electrical diagnostic operations Safety and Environmental Issues Demonstrate knowledge of safety/environmental requirements in the collision

repair industry Identify proper safety techniques for the use of shop equipment

Page 34 of 40

Collision Repair And Refinishing Technology

Written Assessment:

Administration Time: 3 hours Number of Questions: 150 Areas Covered:

Page 2 of 5 NOCTI Job Ready Assessment Copyright © NOCTI 2007 All Rights Reserved Page 35 of 40

Revision Date: February 2008

AUTO BODY

REPAIR

Program Guide

Content

Area Course Title Course Number Course Hours

00 Intro to Auto Body Orientation and Safety ABR101 45 Principles of Design Construction Lifting

and Supporting ABR102 45 Hand Tools ABR103 30 Power Tools ABR104 30 Mathematics for Automotive Body

Repair I ABR105 15 Vehicle Handling in the Shop ABR106 30 Mathematics for Automotive Body

Repair II ABR107 15 Applied Math II ABR201 15 Applied Math III ABR301 15 Leadership ABR109 15 Business Principles ABR209 15 Total Quality Concepts ABR309 15

10 Laboratory Practices Applied Practices I ABR111 15 Applied Practices II ABR112 15 LII Laboratory Practices I ABR211 30 LII Laboratory Practices II ABR212 30 LII Laboratory Practices III ABR213 30 LII Laboratory Practices IV ABR214 30 LIII Laboratory Practices I ABR311 30 LIII Laboratory Practices II ABR312 30 LIII Laboratory Practices III ABR313 30 LIII Laboratory Practices IV ABR314 30

20 Welding LI MIG Welding I ABR121 30 LI MIG Welding II ABR122 30 LII MIG Welding I ABR221 30 LII MIG Welding II ABR222 30 LII MIG Welding III ABR223 30

Page 36 of 40

Revision Date: February 2008

LIII MIG Welding I ABR321 30 LIII MIG Welding II ABR322 30 LIII MIG Welding III ABR323 30

30 Panel Repairs Sheet Metal Repair I ABR131 30 Sheet Metal Repair II ABR132 15 LII Panel Repair I ABR231 30 LII Panel Repair II ABR232 30 LII Panel Repair III ABR233 30

LII Panel Repair IV ABR234 30 LIII Panel Repair I ABR331 30 LIII Panel Repair II ABR332 30 LIII Panel Repair III ABR333 30 LIII Panel Repair IV ABR334 30

40 Paint and Refinishing Surface Prep ABR141 30 Sanding Techniques I ABR142 30 Refinish Equipment and Masking

Techniques ABR143 30 LII Refinish I ABR241 30 LII Refinish II ABR242 30 LII Refinish III ABR243 30 LII Refinish IV ABR244 30 LIII Refinish I ABR341 30 LIII Refinish II ABR342 30 LIII Refinish III ABR343 30 LIII Refinish IV ABR344 30

80 Business Concepts Estimating ABR181 30

Page 37 of 40

AUTO BODY REPAIR ECTS QUARTERLY COURSE SEQUENCE

Revision Date: December 2007 JK

YE

AR

I

1st Quarter ABR101 45 Orientation and Safety ABR102 45 Principles of Design, Construction Lifting and Supporting ABR103 30 Hand Tools TOTAL:120

2nd Quarter ABR104 30 Power Tools ABR121 30 MIG Welding I ABR131 30 Sheet Metal Repair I ABR105 15 Math for Automotive Body Repair I ABR109 15 Leadership TOTAL: 120

3rd Quarter ABR111 15 Applied Practices I ABR122 30 MIG Welding II ABR132 15 Sheet Metal Repair II ABR141 30 Surface Prep ABR142 30 Sanding Techniques TOTAL:120

4th Quarter ABR112 15 Applied Practices II ABR107 15 Math for Automotive Body Repair II ABR 106 30 Vehicle Handling in the Shop ABR143 30 Refinish Equipment and Masking Techniques ABR181 Estimating 30 TOTAL:120

YE

AR

II

1st Quarter ABR211 30 LII Laboratory Practices I ABR221 30 LII, MIG Welding I ABR241 30 LII Refinishing I ABR231 30 LII Panel Repair I TOTAL:120

2nd Quarter ABR212 30 LII Laboratory Practices II ABR201 15 Applied Math II ABR209 15 Business Principles ABR242 30 LII Refinishing II ABR232 LII Panel Repair II 30 TOTAL:120

3rd Quarter ABR213 30 LII Laboratory Practices III ABR222 30 LII MIG Welding II ABR243 30 LII Refinishing III ABR233 LII Panel Repair III 30 TOTAL:120

4th Quarter ABR214 30 LII Laboratory Practices IV ABR223 30 LII MIG Welding III ABR244 30 LII Refinishing IV ABR234 LII Panel Repair IV 30 TOTAL:120

YE

AR

III

1st Quarter ABR311 30 LIII Laboratory Practices I ABR321 30 LIII MIG Welding I ABR341 30 LIII Refinishing I ABR331 30 LIII Panel Repair I TOTAL:120

2nd Quarter ABR312 30 LIII Laboratory Practices II ABR309 15 Total Quality Concepts ABR301 15 Applied Math III ABR342 30 LIII Refinishing II ABR332 30 LIII Panel Repair II TOTAL:120

3rd Quarter ABR313 30 LIII Laboratory Practices III ABR322 30 LIII MIG Welding II ABR343 30 LIII Refinishing III ABR333 30 LIII Panel Repair III TOTAL: 120

4th Quarter ABR314 30 LIII Laboratory Practices IV ABR323 30 LIII MIG Welding III ABR344 30 LIII Refinishing IV ABR334 30 LIII Panel Repair IV TOTAL: 120

Page 38 of 40

Electrical EngineeringWritten

-40

-30

-20

-10

0

10

20

30

40

50

60

70

Competencies Tested

Perc

ent S

core

Group Norm--2007 47.5 52.5 35.8 58.6 33.6 51.1 61.4 34.6 22.5 33.8 47.5 40 38.6 37.5 40 35 41.7

Sig. Diff.--2007 -9.8 -19.7 -19.4 -12.7 -18.3 -21.4 0 -16.6 -16.8 -17.6 0 -12.5 -14 -8.4 -8.8 0 -15.9

Group Norm--2008 28.6 40.5 35.3 60.2 29.9 50.8 38.8 27.5 32.1 34.8 39.3 27.5 28.6 25 38.1 42.9 40.5

Sig. Diff.--2008 -29.8 -30.1 -18.2 -12.4 -20.3 -21.7 -19.5 -22 0 -14.2 0 -24.5 -23.9 -18.9 -8.8 0 -12.1

General Information Safety

Meters, Measurements, Testing

Identification of Tools,

Materials, &

National Electric Code

Blueprints, Drawings, Diagrams,

Planning, Layout, & Estimating

AC Theory, Conductors, Magnetic

ElectronicsDC: Basic

Electric/Electron

Low Voltage Systems

Wiring, Circuits &

Installation

Raceway & Conduit Bending

Transformers

Motors, Motor

Control Lighting

Basic Computer

Skills

Page 39 of 40

Creating a

Strategic Improvement

Plan

to improve student achievement

Page 40 of 40

student occupational competency assessment - pacta · student occupational competency assessment...

Documents