1

Program and Compliance ManagementProgram and Compliance Management

Workshop: Workshop:

Documentation Desires, Duties, DilemmasDocumentation Desires, Duties, Dilemmas

V I R T U A L L V I R T U A L L YY

OutlineOutline

• IntroductionIntroduction

• Eligibility vs. Data ValidationEligibility vs. Data Validation

• DV DocumentationDV Documentation

• Documenting Common Measure Documenting Common Measure OutcomesOutcomes

• Best Case Scenario?Best Case Scenario?

2

IntroductionIntroduction

3

First Things FirstFirst Things First

• Documentation is not just paper (Utah, for instance, is entirely paperless)

• Requirements differ across programs and populations and pertain to:

– Who you’re serving …documenting characteristics

– What they’re getting …documenting services

– With what results …documenting outcomes

• “THEY said…” – Make sure what you think is required really is

4

Potential Data SourcesPotential Data Sources

Who You’re ServingWho You’re Serving What They’re GettingWhat They’re Getting With What ResultsWith What Results

Birth CertificateBaptismal Records

Social Security CardResident Card

Driver’s LicenseSchool/Student IDSchool Records

Hospital RecordsPublic Agency Records

DD-214Veterans’ Org.

RecordsAssessment Records

Pay Stubs

[Others]

Attendance SheetsProvider Forms

Provider InvoicesSign-in Sheets

School RecordsActivity Forms

Progress Reports

[Others]

Assessment ResultsCompletion dates

Pay StubsDocumented employer

callSchool Records

Progress ReportsCopy of Diploma

Copy of CertificateSurveys

[Others]

5

The Goldilocks FactorThe Goldilocks Factor

• Not too much. Not too little. Just right.Just right.

– Supports stewardship, accountability

– Leaves audit trail

– Consistent with data validation requirements

6

Issues from Issues from Data Validation ReviewsData Validation Reviews

• Lack of documentation or insufficient documentation to support service provision (particularly for youth services)

• State and/or Local policy includes incorrect sources

• Issues with case files and case notes >>

7

Issues from Issues from Data Validation Reviews (2)Data Validation Reviews (2)

• Case Files– Can’t be located, documents missing– Inconsistencies between file and MIS– Lack of consistent file structure– One file for multiple periods of participation

• Case Notes– Quality (insufficient, illegible, irrelevant); Missing (where

needed or required by policy)– Case notes contain info that differs from other documents

or the MIS (e.g., different dates)

8

Eligibility Eligibility vs.vs.

Data ValidationData Validation

9

Are the Following Statements Are the Following Statements TRUE or FALSE?TRUE or FALSE?

• Federal policy elaborates on the eligibility requirements contained in the statute and regulations.

• Documentation for eligibility and documentation for data validation are the same.

10

EXAMPLE: Youth EligibilityEXAMPLE: Youth Eligibility

• Sec. 101(13) of the WIA states that eligible youth are low income and have at least one barrier:

1. Basic Skills Deficient

2. School Dropout

3. Homeless, Runaway or Foster Child

4. Pregnant or Parenting

5. Offender

6. Youth who requires additional assistance…

11

Documentation Disconnect

Documentation Disconnect

• BSD is defined at Sec. 101(4) as being determined by a “generally accepted standardized test…”

– …So eligibility is based on a standardized test but

– Federal DV policy has school records and case notes in addition to standardized tests (TEGL 28-11, May 9, 2012 (page A-18))

Basic Skills DeficiencyBasic Skills DeficiencyDocumentation DisconnectDocumentation Disconnect

12

• AND… if the youth is BSD as defined, then Literacy/Numeracy requirements must also be followed

– …which means the ‘standardized test’ has to be acceptable for L/N or another test administered

– …which means reporting several WIASRD data elements and following the associated DV documentation requirements

Basic Skills DeficiencyBasic Skills DeficiencyDocumentation Disconnect (2)Documentation Disconnect (2)

13

The 6The 6thth Youth Eligibility Criterion Youth Eligibility CriterionDocumentation DisconnectDocumentation Disconnect

• “…requires additional assistance to complete an educational program or to secure and hold employment as defined by State or local policy. If the State Board defines a policy, the policy must be included in the State Plan.”

• Many states don’t define, leaving it up to locals

• How is it defined in your state?

14

• DV reviews have cited more than lack of documentation for this eligibility item

• Including insufficient case notes to validate

• Area of Concern often included in final report recommending a thorough review of local definitions/policy on this item (e.g., too vague)

The 6The 6thth Youth Eligibility Criterion Youth Eligibility CriterionDocumentation Disconnect (2)Documentation Disconnect (2)

15

Data Validation Data Validation DocumentationDocumentation

16

Types of Allowable Source Types of Allowable Source DocumentationDocumentation

• The document itself (e.g., certificate)

• State MIS (management information system)

• Cross-Match

• Self-Attestation

• Case Notes

17

““State MIS”State MIS”

• Specific, detailed information stored in the state’s management information system that supports an element

– An indicator, such as a checkmark on a computer screen, is not acceptable source documentation in and of itself

• A deeper level of inquiry is required

18

Example of Using “State MIS”Example of Using “State MIS”Validating Validating Date of ExitDate of Exit

• After the exit date is noted in the state MIS, what additional information would you look for in order to pass this data element?

– The service provided and last date received

– No other qualifying services provided within 90 days of the last service

– Individual not scheduled for future service and has no gap in service

19

““Cross-Match”Cross-Match”

• Requires accessing a non-WIA database to find detailed supporting evidence for the data element

– An indicator or presence of an SSN or checkmark in a non-WIA database is not sufficient evidence

• TANF participation can be determined by a cross-match with the state’s public assistance database but it is not sufficient to just find the SSN in the database

20

Example of Using “Cross-Match”Example of Using “Cross-Match”Validating Validating Youth PlacementYouth Placement

• A state uses a cross-match (computerized record) with their Higher Education Administration as the source document for verifying placement into post-secondary education

• What information would you need to pass this data element?

– Name or some other personal identifier

– Information showing enrollment in post-secondary education

– Date that indicates the youth was enrolled in first quarter after exit quarter

21

““Self-Attestation”Self-Attestation”

• When a participant states his/her status for a particular data element, then signs and dates a form acknowledging this status

• Key elements for self-attestation:

– Participant identifying his/her status for permitted data elements

– Signing and dating a form attesting to this self-identification

22

Example of Using “Self-Attestation”Example of Using “Self-Attestation”Validating Validating Displaced HomemakerDisplaced Homemaker Status Status

• What type of information/language must be included on a self-attestation form to be considered an acceptable source document for this data element?

– The individual has been providing unpaid services to family members in the home, and

– Has been dependent on the income of another family member, and

– Is no longer supported by that income, and

– Is unemployed or underemployed, and

– Is experiencing difficulty obtaining/upgrading employment.

– Signature and date

23

““Case Notes”Case Notes”

• Paper or electronic statements by the case manager that identify, at a minimum, the following

– Participant’s status for a particular data element,

– Date the information was obtained (and from who, as appropriate)

– Name of case manager

24

Example of Using “Case Notes”Example of Using “Case Notes”Validating Validating Other Reasons For ExitOther Reasons For Exit

• What type of information must case notes include?

– Participant information (likely moot point given electronic case notes)

– Individual will no longer be able to participate due to [state reason*]

*Institutionalization; Health/Medical; Deceased; Family Care; Reservist called to Active Duty; Relocation to mandated facility for youth

25

Documenting Documenting Common Measure Common Measure

OutcomesOutcomes

26

Data SourcesData Sources

• According to TEGL 17-05, these are the five basic data sources used to demonstrate common measure outcomes

A. Wage Records

B. WRIS, FEDES

C. Administrative Records

D. Assessment Instruments

E. Supplemental Data

27

Documenting OutcomesDocumenting OutcomesUsing Supplemental DataUsing Supplemental Data

• In the absence of wage recordsIn the absence of wage records, can be used to demonstrate employment and retention only (i.e., not earnings)

• All data and methods are documented and subject to audit

– “Allowable sources. . . include case management notes, automated data base systems, One-Stop operating system’s administrative records, surveys of participants, and contact with employers.” (TEGL 17-05, page 26)

• The data and methods to obtain it can be clarified in state policy guidance (e.g., maintaining correct contact information is job one)

28

Documenting OutcomesDocumenting OutcomesUsing Supplemental Data (2)Using Supplemental Data (2)

• Some states use supplemental data more than others (and some states don’t use it at all)

– SD and UT do not use supplemental data

– For the first quarter of PY12, supplemental data accounted for what percentage of the numerator of the Adult EER?

• MT – 6%

• ND – 2%

• WY – 9%

29

Documenting OutcomesDocumenting OutcomesCommon MeasuresCommon Measures

1. Entered Employment Rate

2. Employment Retention Rate

3. Average Earnings

4. Placement in Employment or Education

5. Attainment of Degree/Certificate

6. Literacy/Numeracy

A. Wage RecordsB. WRIS, FEDESC. Supplemental DataD. Administrative

RecordsE. Assessment

Instruments

According to TEGL 17-05, these are the five basic data sources used to demonstrate common measure outcomes (A—E).

Question: Which common measure utilizes four of the allowable data sources?Question: Which common measure utilizes four of the allowable data sources?

30

Best Case ScenarioBest Case Scenario

31

The Ideal ?The Ideal ?

• Use federal DV source documentation as foundation

• State provides additional guidance to clarify federal sources where necessary

• State provides guidance on “other” documentation to ensure statewide consistency

• Local Boards/One-Stop Operators provide additional guidance where needed (consistent with State and federal)

32

The Ideal ? (2)The Ideal ? (2)

• Irrespective of level, documentation policies and procedural requirements are part of staff orientation and general data management

– Consistency

– Accountability

33

34