u.s. department of labor employment and training administration transforming wia youth programs...

U.S. Department of LaborU.S. Department of Labor

Employment and Employment and Training AdministrationTraining Administration

Transforming WIA Transforming WIA Youth Programs Youth Programs

through through Performance and Performance and

ReportingReporting

2

AgendaAgenda

• Review ETA’s vision for serving youthReview ETA’s vision for serving youth

• Clarify statutory & common measuresClarify statutory & common measures

• Review key terms and conceptsReview key terms and concepts

• Calculate youth common measuresCalculate youth common measures

• Youth program design implicationsYouth program design implications

3

New Strategic VisionNew Strategic Vision

• Out-of-school youth (and those most at risk Out-of-school youth (and those most at risk of dropping out) are an important part of the of dropping out) are an important part of the new workforce “supply pipeline” needed by new workforce “supply pipeline” needed by businesses to fill job vacancies in a businesses to fill job vacancies in a knowledge-based economy knowledge-based economy

• WIA-funded youth programs will provide WIA-funded youth programs will provide leadership by serving as a catalyst to leadership by serving as a catalyst to connect youth with quality secondary and connect youth with quality secondary and post-secondary educational opportunities post-secondary educational opportunities and high-growth employmentand high-growth employment

4

The New Youth VisionThe New Youth Vision

• Built on 4 pillars:Built on 4 pillars:– Meeting the demands of business Meeting the demands of business

(be demand-driven) (be demand-driven)– Alternative educationAlternative education– Focus on the neediest youth—special Focus on the neediest youth—special

targeted populations targeted populations – Focus on improved performanceFocus on improved performance

THE NEW YOUTH

VISION

5

What is Demand-Driven?What is Demand-Driven?

• Using labor market information and other Using labor market information and other economic data to assess employment economic data to assess employment opportunities in high growth industriesopportunities in high growth industries

• Awareness of skills and competencies needed Awareness of skills and competencies needed to satisfy employer requirements to satisfy employer requirements

• The strategic investment of public workforce The strategic investment of public workforce system resources to meet employer needssystem resources to meet employer needs

• Forming strategic partnerships to develop Forming strategic partnerships to develop effective solution setseffective solution sets

• Catalyst to promote focused economic growth Catalyst to promote focused economic growth through talent developmentthrough talent development

6

Why Focus on Alternative Education? Why Focus on Alternative Education? • 3.8 million dropouts, not employed3.8 million dropouts, not employed• Negatively impact state economies through lost Negatively impact state economies through lost

tax revenue as well as costs to other systems tax revenue as well as costs to other systems such as TANF and JJsuch as TANF and JJ

• The labor force needs these potential workers & The labor force needs these potential workers & 80% of new jobs will require some type of post- 80% of new jobs will require some type of post- secondary trainingsecondary training

• Youth workforce system is intermediary Youth workforce system is intermediary responsible for reconnecting youth to educational responsible for reconnecting youth to educational opportunitiesopportunities

• Without reconnecting them to educational Without reconnecting them to educational opportunities, youth will not be adequately opportunities, youth will not be adequately prepared to participate in new, knowledge-based prepared to participate in new, knowledge-based economy economy

7

Serving the Neediest YouthServing the Neediest Youth

• Youth in foster care or aging out of foster careYouth in foster care or aging out of foster care• Children of incarcerated parentsChildren of incarcerated parents• Court involved youth or those at risk of Court involved youth or those at risk of

involvementinvolvement• Migrant youthMigrant youth• Youth with disabilitiesYouth with disabilities• Native American youthNative American youth• Homeless and runaway youthHomeless and runaway youth• Out-of-school youthOut-of-school youth• High school dropoutsHigh school dropouts

8

Improved PerformanceImproved Performance

• Implement common performance measuresImplement common performance measures• Identify how collaborative efforts can lead to Identify how collaborative efforts can lead to

shared outcomes for all youth-serving agenciesshared outcomes for all youth-serving agencies• Use “real time” data to manage programs and Use “real time” data to manage programs and

work toward program improvementwork toward program improvement

9

Collaboration is KeyCollaboration is Key

• Workforce programs alone cannot achieve this Workforce programs alone cannot achieve this vision vision

• Shared youth vision activities emphasize the Shared youth vision activities emphasize the importance of collaboration at every levelimportance of collaboration at every level

• Federal/State/Local agencies that fund youth Federal/State/Local agencies that fund youth programs should be coordinating efforts in programs should be coordinating efforts in serving youthserving youth

• Critical to engage your partners at the state Critical to engage your partners at the state and local leveland local level

10

Clarifying AccountabilityClarifying AccountabilityYouth WIA 7 & CM 3Youth WIA 7 & CM 3

11

Implementation of PY 2007 ReportingImplementation of PY 2007 Reporting

WIA Statutory MeasuresWIA Statutory Measures

Older youth (19-21)Older youth (19-21) Younger youth (14-18)Younger youth (14-18)

Entered employment rateEntered employment rate Skill attainment rate Skill attainment rate

Retention rateRetention rate Diploma attainment rate Diploma attainment rate

Credential/Certificate rateCredential/Certificate rate Retention rate Retention rate

Earnings changeEarnings change

Youth Common Measures (14-21)Youth Common Measures (14-21)

Placement in employment or educationPlacement in employment or education

Attainment of a degree/certificateAttainment of a degree/certificate

Literacy/numeracy gainsLiteracy/numeracy gains

12

AccountabilityAccountability

• Most grantees are held accountable for Most grantees are held accountable for performance outcomesperformance outcomes on the sevenon the seven WIA youth WIA youth performance measuresperformance measures

– In the absence of a reauthorized WIA, the statutory In the absence of a reauthorized WIA, the statutory measures must be used for accountability purposesmeasures must be used for accountability purposes

• All grantees are required to All grantees are required to report report on the three on the three youth common measures – placement in youth common measures – placement in employment/education, degree attainment and employment/education, degree attainment and literacy/numeracy literacy/numeracy

13

Current vs. Common MeasuresCurrent vs. Common MeasuresIn GeneralIn General

• States should continue to ensure success with States should continue to ensure success with current measures, while transitioning to a focus current measures, while transitioning to a focus on common measureson common measures

• Major difference between the two sets of Major difference between the two sets of measures is the lack of ISY exclusion for measures is the lack of ISY exclusion for placement and certificate common measuresplacement and certificate common measures

– Keep ISY engaged until graduation/GED attainmentKeep ISY engaged until graduation/GED attainment

• Focus on post-secondary education/ Focus on post-secondary education/ occupational skills training, employment and/or occupational skills training, employment and/or literacy/numeracy gainsliteracy/numeracy gains

14

Reviewing Key Terms Reviewing Key Terms and Conceptsand Concepts

15

Program ParticipationProgram Participation

A participant is an individual who is determined eligible to participate in a program and receives a service funded by the program.

The participation date is the date the first service is received after the individual is determined eligible.

16

Program ExitProgram Exit

• Exit occurs when a participant does not Exit occurs when a participant does not receive a service (funded by program or receive a service (funded by program or partner program) for 90 consecutive days and partner program) for 90 consecutive days and no future services are scheduledno future services are scheduled

• There is no more concept of “hard exit” only There is no more concept of “hard exit” only what was a “soft exit”what was a “soft exit”

• The The exitexit datedate is the date of the last service is the date of the last service received by the participantreceived by the participant

• For measures using UI wage records, the For measures using UI wage records, the exitexit quarterquarter is the quarter containing the exit date is the quarter containing the exit date

17

Clarifying Follow-Up ServicesClarifying Follow-Up Services

• Youth are required to receive at least 12 Youth are required to receive at least 12 months of follow-up services, which are months of follow-up services, which are triggered at exittriggered at exit

• With a “soft exit” approach, do you lose the With a “soft exit” approach, do you lose the first 3 months of follow-up? first 3 months of follow-up? NONO

• Begin follow-up after the expected last service Begin follow-up after the expected last service –- if only follow-up services are provided for –- if only follow-up services are provided for next 90 days, record an “exit” next 90 days, record an “exit”

– However, if youth need additional services, this 90 However, if youth need additional services, this 90 days provides a “window” to better ensure days provides a “window” to better ensure success (because no official exit was recorded)success (because no official exit was recorded)

18

ExclusionsExclusions

Circumstances that exclude participants from Circumstances that exclude participants from performance measures (current and common):performance measures (current and common):• Institutionalized Institutionalized • Health/Medical or Family CareHealth/Medical or Family Care• Deceased Deceased • Reservists called to active dutyReservists called to active duty• Relocated to a mandated program (such as Relocated to a mandated program (such as

foster care)foster care)• Invalid or missing SSNInvalid or missing SSN

19

Gaps in ServiceGaps in Service

Participants with a gap in service of greater than Participants with a gap in service of greater than 90 days should not be exited if gap is due to one 90 days should not be exited if gap is due to one of the following:of the following:• Delay before beginning of trainingDelay before beginning of training• Health/medical conditionHealth/medical condition• Temporary move from area that prevents Temporary move from area that prevents

individual from participatingindividual from participating

20

Data SourcesData Sources

• UI wage records and supplemental data UI wage records and supplemental data sources are allowable for employment-related sources are allowable for employment-related measures (placement and retention)measures (placement and retention)

• UI wage records for OY earnings measureUI wage records for OY earnings measure• Administrative records will be data source for Administrative records will be data source for

non-employment placements as well as non-employment placements as well as degree or certificate, skill attainment & degree or certificate, skill attainment & literacy/numeracy measuresliteracy/numeracy measures

21

Calculating Youth Common Calculating Youth Common MeasuresMeasures

22

Youth Common MeasuresYouth Common Measures

• Placement in employment or education Placement in employment or education

• Attainment of a degree/certificateAttainment of a degree/certificate

• Literacy/numeracy gainsLiteracy/numeracy gains

NOTE: The measures apply to youth 14-21; no more

older/younger distinction!

23

Placement in Employment or EducationPlacement in Employment or Education

DefinitionDefinition

Of those not in post-secondary education, Of those not in post-secondary education, employment, or the military at participation, the employment, or the military at participation, the percentage in employment, the military or percentage in employment, the military or enrolled in post-secondary education and/orenrolled in post-secondary education and/oradvanced training/occupational skills training in advanced training/occupational skills training in the 1the 1stst quarter after exit. quarter after exit.

24

CalculationCalculation

Of those not employed, in the military, or in post-Of those not employed, in the military, or in post-secondary education at participation…secondary education at participation…

Number of youth employed, in the military or Number of youth employed, in the military or enrolled in post-secondary education and/or enrolled in post-secondary education and/or

advanced training or occupational skills advanced training or occupational skills training in the 1training in the 1stst quarter after exit quarter after exit

Number of exitersNumber of exiters


25

• Excludes youth in employment, the military or Excludes youth in employment, the military or post-secondary education at participationpost-secondary education at participation

• Employment, military and education status at Employment, military and education status at participation is based on information collected participation is based on information collected from the individualfrom the individual

• Employment and military status in the 1Employment and military status in the 1stst quarter quarter after exit is based on wage records (supplemental after exit is based on wage records (supplemental data allowed)data allowed)

• Education status in the 1Education status in the 1stst quarter after exit is quarter after exit is based on administrative recordsbased on administrative records

• Youth in secondary school at exit are Youth in secondary school at exit are included, included, consistent with ETA’s vision to ensure youth consistent with ETA’s vision to ensure youth successfully complete their secondary educationsuccessfully complete their secondary education


26

Defining Specific TermsDefining Specific Terms

Post-Secondary EducationPost-Secondary Education

A program at an accredited degree-granting A program at an accredited degree-granting institution leading to an academic degree. institution leading to an academic degree. Programs offered by degree-granting institutions Programs offered by degree-granting institutions that do that do not not lead to an academic degree (such as lead to an academic degree (such as certificate programs) do not count as a placement certificate programs) do not count as a placement in post-secondary education but may count as a in post-secondary education but may count as a placement in “advanced training/occupational placement in “advanced training/occupational skills training.”skills training.”

27

Advanced Training/Occupational Skills TrainingAdvanced Training/Occupational Skills Training

An organized program of study that provides An organized program of study that provides specific vocational skills leading to proficiency in specific vocational skills leading to proficiency in performing actual tasks and technical functions performing actual tasks and technical functions required by certain occupational fields at entry, required by certain occupational fields at entry, intermediate or advanced levels. Such training intermediate or advanced levels. Such training should: should:

• be outcome-oriented and focused on a long-term goal be outcome-oriented and focused on a long-term goal as specified in the ISS,as specified in the ISS,

• coincide with exit rather than short-term training, and coincide with exit rather than short-term training, and

• result in attainment of a certificateresult in attainment of a certificate


28

Employed, in the military or in post-

secondary education at participation?

Has a qualifying outcome* in the 1st quarter after exit?

NO

YES

Excluded

YES

YES

Numerator

Denominator

NO

Qualifying Outcomes: employment, military, enrolled in post-secondary education, advanced training, or occupational skills training.


29

Attainment of a Degree or CertificateAttainment of a Degree or Certificate


Of those enrolled in education at or during Of those enrolled in education at or during participation, the percentage who attained a participation, the percentage who attained a diploma, GED or certificate by the end of the 3diploma, GED or certificate by the end of the 3rdrd quarter after exit.quarter after exit.

30


Of those enrolled in education at participation or Of those enrolled in education at participation or any time during the program…any time during the program…

Number of youth who attained a diploma, GED or Number of youth who attained a diploma, GED or certificate by the end of the 3certificate by the end of the 3rdrd quarter after exit quarter after exit

Number of exitersNumber of exiters


31

• Youth in secondary school at exit are Youth in secondary school at exit are included, included, which which is consistent with ETA’s vision to ensure youth is consistent with ETA’s vision to ensure youth successfully complete their secondary educationsuccessfully complete their secondary education

• Diplomas, GEDs or certificates can be obtained Diplomas, GEDs or certificates can be obtained during participation or at any point by the end of the during participation or at any point by the end of the 33rdrd quarter after exit quarter after exit

• Work readiness certificates will Work readiness certificates will not not be accepted for be accepted for this measurethis measure

• Consistent with ETA’s youth vision’s demand- driven Consistent with ETA’s youth vision’s demand- driven approach to ensure training consistent with employer approach to ensure training consistent with employer needsneeds


32

EducationEducation

Participation in secondary or post-secondary Participation in secondary or post-secondary school, adult education programs, or any other school, adult education programs, or any other organized program of study leading to a degree organized program of study leading to a degree or certificate.or certificate.

DiplomaDiploma

Any credential accepted by the State educational Any credential accepted by the State educational agency as equivalent to a HS diploma; also agency as equivalent to a HS diploma; also includes post-secondary degrees.includes post-secondary degrees.


33

Certificate Certificate

Awarded in recognition of an individual’s Awarded in recognition of an individual’s attainment of technical or occupational skills by attainment of technical or occupational skills by specifiedspecified institutions such as the State institutions such as the State educational agency or institution of higher educational agency or institution of higher education. It does education. It does not not include work readiness include work readiness certificates or certificates awarded by local certificates or certificates awarded by local Boards.Boards.


34

Certificates: Awarding InstitutionsCertificates: Awarding Institutions• A state educational agency, or a state agency A state educational agency, or a state agency

administering vocational and technical education administering vocational and technical education • Institution of higher education Institution of higher education • A professional, industry, or employer organization A professional, industry, or employer organization • A registered apprenticeship programA registered apprenticeship program• A public regulatory agency (e.g., FAA aviation mechanic A public regulatory agency (e.g., FAA aviation mechanic

certification)certification)• A program approved by Veterans’ Affairs to offer A program approved by Veterans’ Affairs to offer

education and training to veterans and other eligible education and training to veterans and other eligible persons under the Montgomery GI Billpersons under the Montgomery GI Bill

• Office of Job CorpsOffice of Job Corps• Institutions of higher education which are formally Institutions of higher education which are formally

controlled, or formally sanctioned or chartered by controlled, or formally sanctioned or chartered by the governing body of an Indian tribe the governing body of an Indian tribe

35

Enrolled in education at participation or any

time during participation?

Attained diploma, GED or certificate by the end of

the 3rd quarter after exit?

YES

NO Excluded

YES

YES

Numerator

Denominator

NO


36

Literacy/Numeracy GainsLiteracy/Numeracy Gains


Of those out of-school youth who are basic skills Of those out of-school youth who are basic skills deficient, the percentage who increase one or deficient, the percentage who increase one or more educational functioning levels within one more educational functioning levels within one year of participation.year of participation.

37


Of those out-of-school youth who are basic skills Of those out-of-school youth who are basic skills deficient…deficient…

Number of youth participants who increase one Number of youth participants who increase one or more educational functioning levelsor more educational functioning levels

Number of youth Number of youth who completed a who completed a

year of participation year of participation (based on date of 1(based on date of 1stst youth youth

service)service)

Number of youth Number of youth who exit before who exit before

completing a year completing a year of participationof participation

++


38

• Only common measure that is not exit-basedOnly common measure that is not exit-based• Excludes in-school youth and out-of-school youth Excludes in-school youth and out-of-school youth

who are not basic skills deficientwho are not basic skills deficient• Includes individuals with learning disabilitiesIncludes individuals with learning disabilities• A positive outcome means the youth must advance A positive outcome means the youth must advance

one or more Adult Basic Education or English as a one or more Adult Basic Education or English as a Second Language functioning levelSecond Language functioning level

• Gains can occur in literacy Gains can occur in literacy or or numeracy (can pre-numeracy (can pre-test at different levels in each category)test at different levels in each category)

• Educational levels are consistent with Department Educational levels are consistent with Department of Education’s National Reporting System (NRS)of Education’s National Reporting System (NRS)


39

About the AssessmentsAbout the Assessments• All out-of-school youth must be assessed in All out-of-school youth must be assessed in

basic reading, writing and mathbasic reading, writing and math

• Pre-testing must occur within 60 days of the Pre-testing must occur within 60 days of the first youth program service; can use pre-test first youth program service; can use pre-test from up to six months prior to date of first from up to six months prior to date of first youth serviceyouth service

• The same standardized assessment must be The same standardized assessment must be used for pre- and post-testingused for pre- and post-testing

• Youth should be post-tested by the end of one Youth should be post-tested by the end of one year of participation and compared to pre-test year of participation and compared to pre-test results obtained during initial assessmentresults obtained during initial assessment

40

About the AssessmentsAbout the Assessments

• If a youth continues to be basic skills deficient If a youth continues to be basic skills deficient after the first 12 months of participation, they after the first 12 months of participation, they should continue to receive training in literacy should continue to receive training in literacy and/or numeracy skillsand/or numeracy skills

• Youth should be post-tested and included in the Youth should be post-tested and included in the measure at the completion of the 2measure at the completion of the 2ndnd year only if year only if they complete two full years in the programthey complete two full years in the program

41

Implications of the New Implications of the New Youth Vision and Program Youth Vision and Program

DesignDesign

42

Long-Term ParticipationLong-Term ParticipationProgram Design Implication #1Program Design Implication #1

• Positive outcomes are unlikely to be attained Positive outcomes are unlikely to be attained unless youth participate for extended periods of unless youth participate for extended periods of timetime– In-school younger youth must remain engaged until In-school younger youth must remain engaged until

they graduate or receive a GED and go on to they graduate or receive a GED and go on to employment or post-secondary educationemployment or post-secondary education

– Out-of-school youth who are basic skills deficient will Out-of-school youth who are basic skills deficient will likely need more time to improve one or more EFLslikely need more time to improve one or more EFLs

– Summer employment programs that enroll WIA youth Summer employment programs that enroll WIA youth for the summer only are unlikely to achieve positive for the summer only are unlikely to achieve positive outcomesoutcomes

43

• Keeping youth connected to an organization or Keeping youth connected to an organization or programprogram– Establishing a sense of self-worth through program Establishing a sense of self-worth through program

participation (e.g., they get paid, receive positive feedback; participation (e.g., they get paid, receive positive feedback; their contributions matter, etc.)their contributions matter, etc.)

• Providing incentivesProviding incentives– Opportunities to demonstrate skills to family and peers; field Opportunities to demonstrate skills to family and peers; field

trips and events; stipends, opportunities to serve and lead; trips and events; stipends, opportunities to serve and lead; supportive services; recognitionsupportive services; recognition

• Youth-centered programsYouth-centered programs– Youth-friendly intake procedures; interpersonal support in Youth-friendly intake procedures; interpersonal support in

programs from personnel, parents, peers; honesty and programs from personnel, parents, peers; honesty and authenticity in program (provide what’s promised); culturally authenticity in program (provide what’s promised); culturally competent staff and culturally relevant programscompetent staff and culturally relevant programs

Long-Term ParticipationLong-Term ParticipationEngagement and Retention StrategiesEngagement and Retention Strategies

44

• Since long-term participation is needed to Since long-term participation is needed to achieve outcomes, more than ever, WIA youth achieve outcomes, more than ever, WIA youth must have access to a network of services, must have access to a network of services, not a single program or providernot a single program or provider– Strategic case management strategies will be Strategic case management strategies will be

needed to connect youth to multiple programsneeded to connect youth to multiple programs

– Develop short-term attainable goals for youth and Develop short-term attainable goals for youth and youth programs to support long-term positive youth programs to support long-term positive outcomesoutcomes

Network of ServicesNetwork of ServicesProgram Design Implication #2Program Design Implication #2

45

Network of ServicesNetwork of ServicesStrategic Case ManagementStrategic Case Management

• Case management is keyCase management is key

– Create and maintain positive relationships with youth; Create and maintain positive relationships with youth; have youth see case manager role as connector to a have youth see case manager role as connector to a network of services; use team approach to helping network of services; use team approach to helping youthyouth

46

• For Youth:For Youth:

– Setting intermediate goals (outcomes) helps them see Setting intermediate goals (outcomes) helps them see progress, which contributes to engagement and progress, which contributes to engagement and retentionretention

• Think SMART; Set goals WITH the youth; Make it VISUALThink SMART; Set goals WITH the youth; Make it VISUAL

• For WIA Youth Programs:For WIA Youth Programs:

– WIA outcomes aren’t appropriate to use as program WIA outcomes aren’t appropriate to use as program outcomes for one-year contracts; develop intermediate outcomes for one-year contracts; develop intermediate measures to monitor program successmeasures to monitor program success

• Including participation rates, skill/goal attainment ratesIncluding participation rates, skill/goal attainment rates

Balance outcomes with outputsBalance outcomes with outputs

Network of ServicesNetwork of ServicesShort-Term OutcomesShort-Term Outcomes

47

• When done well, assessment sets the stage for all When done well, assessment sets the stage for all future services across programs; when done future services across programs; when done poorly, it can have a negative impact on individual poorly, it can have a negative impact on individual youth success youth success and and youth program successyouth program success

– Youth friendly assessmentsYouth friendly assessments

• Shorter versions when possible (subtests), pleasant testing Shorter versions when possible (subtests), pleasant testing environments, don’t send for a test right from the startenvironments, don’t send for a test right from the start

– Tell youth why they are being tested and how test will be Tell youth why they are being tested and how test will be used – don’t retest too soonused – don’t retest too soon

– Consider other assessments – style, self-awareness, Consider other assessments – style, self-awareness, learning style – remove the high-stakes test pressurelearning style – remove the high-stakes test pressure

AssessmentAssessmentProgram Design Implication #3Program Design Implication #3

u.s. department of labor employment and training administration transforming wia youth programs...

Documents

reconnecting youth

school youth

new youth vision slide

wia youth programs

new strategic vision

employment opportunities

highgrowth employment

new jobs