implementing a risk-based title iv audit program presented by: jarod paulson, compliance manager...

Implementing a risk-based Title IV audit program

Presented by:Jarod Paulson, Compliance Manager

Erin Hage, Audit and Compliance Specialist

2

3

Agenda

• Compliance framework• Identifying and assessing risk• Measuring compliance results• Using audit results to continuously

improve• Capella’s risk based TIV audit

program• Questions

4

Compliance framework

• Maintaining TIV compliance is essential– Non-compliance threatens your institution’s

ability to participate in TIV, can result in fines, and damage to brand.

• The external regulatory environment is fluid– Negotiated rulemaking, Reauthorization

• An audit program should Identify compliance gaps, Measure effectiveness, and provide visibility to needed improvements– Changing behavior vs. providing information

5

Identifying and assessing risk

• Risk Based Approach – Risk Assessment

• Top 10 audit and program review findings• Prior Audit Findings or Program review findings• NASFAA or ED assessment tools• FSA handbook, Bluebook & CFR

– Risk Rating• Determine which rules are most impactful for your

institution and design controls in response to residual risk.

– Sampling Methodology• Sample sizes should be risk based and used to

identify; not control risk. • e.g. 20, 40, 60 risk based, SOX methodology

6

Measuring compliance results

• Individual quality monitoring • Process monitoring• Mock audits• Monitoring schedule for non-

operational requirements

7

8

Using audit results to continuously improve

• Coaching– Individual performance and accountability

• Training– Job aid resources– Policy and Procedures

• Reporting– Provide visibility to process owners and

key stakeholders

• Remediation tracking– Lean methodology

9

Using audit results to continuously improve

• Process Improvement principles– Redesign– Eliminate– Standardize– Integrate– Automate– Off-load– Clarify

10

11

Capella’s risk based TIV audit program

• Individual Audits– Purpose

• Quality Assurance• TIV Compliance• Performance Improvement• Visibility to competencies on an individual level

12

What’s next?

1. Identify tasks2. Risk rate each task3. Divide tasks into groups4. Establish a sample size5. Create testing attributes6. Create goals7. Create reporting8. Create a remediation plan

13

Identify Tasks

• Determine what is considered a task– e.g. Verification, R2T4

• Ensure you have visibility to all potential “tasks” at your school

14

Risk rating

• The following factors can be used to determine the risk of each task:– Audit Risk– Complexity– Business Environment– Volume– Impact

15

Risk rating

• Use a scale of 1-3 for each factor– Example:

Complexity

3 – Complex (multiple steps, multiple people involved, multiple dependencies/inputs

2 – Mildly Complex (less than 3 core steps to the process/handoffs)

1 – Simple (low dependency on other processes)

16

Risk Rating

• Average the 5 risk ratings for each task to determine the final risk rate.

Audit Risk Complexity

Business Environment Volume Impact Average

R2T4 Calculations 3 3 1 1 3 2.2

Awarding TEACH Grant 2 3 3 1 2 2.2

17

Divide tasks into test groups• Find similarities among the tasks

and begin to group similar ones together– e.g. R2T4 and PWD calculations could

be combined into one test group

• Other example test groups may be:– Federal Grants– Application– Award Adjustments

18

Establish Sample Size

• Choose a relevant sample size– 20,40,60

• How many hours can be dedicated to this audit?

19

Create testing attributes

Test Group Attribute 1 Attribute 2 Attribute 3 Attribute 4

Federal Grants

Learner is eligible for the grant awarded

Learner was awarded the correct Grant amount

Experienced Teacher Letter/Test Score is sufficient (if applicable)

If funds were authorized, learner is eligible for the disbursement

• Identify what makes a task correctFor example:• Student meets eligibility requirements• Correct award amount

20

Create goals

• Set an Error Rate or Success Rate goal for individuals and the team

• Delivering a success rate instead of an error rate to individuals may create a more positive feeling surrounding audit results– e.g. 3% error rate = 97% success rate

21

Create Reporting

• What do you want to report?– Employee error rates– Team error rates

• How frequently will you report the results?–Monthly, Quarterly, Annually

22

Create Reporting

• Methods–Monthly result meetings– Charts/graphs– Summary by test group

23

Create Reporting – Monthly Error Rates

Test Group Name Federal Grant Test GroupMonth Name January

Processor Items Failed Items Tested Error RateEmployee 1 0 0 0%Employee 2 0 10 0%Employee 3 1 10 10%Employee 4 0 5 0%Employee 5 0 0 0%Employee 6 0 0 0%Employee 7 0 0 0%Employee 8 0 10 0%Employee 9 0 10 0%Employee 10 0 0 0%Team Totals 1 45 2%

24

Create Reporting – Monthly Error Rate

Test Group Name Federal Grant Test Group Federal Grant Test Group Federal Grant Test Group Federal Grant Test Group

Month Name January February March YTD - 2015

ProcessorTotal Failed

Total Tested

Error Rate

Total Failed

Total Tested

Error Rate

Total Failed

Total Tested

Error Rate

Total Failed

Total Tested

Error Rate

Employee 10 0 0% 0 0 0% 0 0 0% 0 0 0%

Employee 20 10 0% 0 10 0% 0 10 0% 0 30 0%

Employee 31 10 10% 1 10 10% 1 10 10% 3 30 10%

Employee 40 5 0% 0 1 0% 0 5 0% 0 11 0%

Employee 50 0 0% 0 0 0% 0 0 0% 0 0 0%

Employee 60 0 0% 0 0 0% 0 0 0% 0 0 0%

Employee 70 0 0% 0 0 0% 0 0 0% 0 0 0%

Employee 80 10 0% 1 10 10% 0 10 0% 1 30 3%

Employee 90 10 0% 0 7 0% 0 10 0% 0 27 0%

Employee 100 0 0% 0 0 0% 0 0 0% 0 0 0%

Team Totals1 45 2% 2 38 5% 1 45 2% 4 128 3%

25

Create Reporting – Isolating the issue

Federal Grant Test Group R2T4 Test Group Award Adj Test Group

YTD Error Rate - 2015 YTD Error Rate - 2015 YTD Error Rate - 2015Employee 1

0% 0% 0%Employee 2

0% 0% 5%Employee 3

10% 0% 5%Employee 4

0% 5% 4%Employee 5

0% 0% 2%Employee 6

0% 0% 4%Employee 7

0% 0% 5%Employee 8

3% 0% 2%Employee 9

0% 0% 5%Employee 10

0% 0% 5%Team Totals

3% 3% 4%

26

Create reporting – Individual employee

Award Ad-justment

R2T4 Federal Grant Overall error rate0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

5%0%

10%5%

Employee 3 YTD Error Rate

Employee 3 YTD Error Rate

27

Create a remediation plan

• Identify who is accountable for taking action on the data

• Communicate results to individual processors

• Determine if training or coaching sessions need to scheduled

• Track your remediation plan• Report the outcomes

28

Capella’s risk based TIV audit program• Monthly self-audit

– Process based audits– Test frequency and sample size determined

by risk rating– Red, Yellow, Green tracking

• Rolling results, quarterly trending and root cause.

– University wide partnership in remediation– Deficiencies are given high visibility

• Department and university leadership, cross functional compliance committee, BOD audit committee.

• Other Audits (based on annual audit plan)

29

Capella remediation structure

30

31

key learnings

• TIV compliance is an institutional responsibility (partnerships across the university)

• Risk assessment should be ongoing• Devoted resources • Zero findings mindset (you will never

be perfect)• Respectful of people, reward good

results, error rate vs. quality score, explain the why.

32

Results

• Continued and increased investment in the program

• Stronger partnerships with other departments

• Strong Compliance results and reputation

33

Questions?