Exercise

Use the enclosed Preview Worksheet to compose a preview of a presentation of one of the report summaries also enclosed.

The preview when read aloud cannot exceed three minutes in duration.

To add realism to the exercise, the instructor will select participants at random to read previews to the group.

Tips for Improving a Preview

1. Do not write a preview as if it were a statute defining afelony!

2. Use simple sentences written in active voice. Bill hit the ball.NOT The ball was hit by Bill. Never use complex sentenceswith multiple adjectives, adverbs, and prepositional phrases.For example, instead of

o The Bureau of Investigation, which is the subject of ourstaff’s evaluation and presentation this morning,operates within authority provided by G.S. 23-12-56(A) and is housed within the Department of Humanservices building on Stanton Street in Plantersville,except for the office of administration, which is locatedin Jeffersonville near the headquarters of theDepartment of Administration behind the DOAwarehouse on Curtis Street.

Use: o Our staff evaluated the Bureau of Investigation within

the Department of Human Services.

3. Read your opening frame quickly, but not as rapidly as an auctioneer!

4. Then slow down. Use bridging phrases when makingtransitions between the frame, message, summary points andthe transition to the full presentation. Do not hesitate torepeat bridging phrases.

1

This page is intended to be blank.

2

1. Frame the presentationa. Write a brief greeting for the panel that you are briefing, e.g. Good morning, Mr. Chairman and members:

b. State your name, position, and organization, e.g. I am William Jones, a Senior Program Evaluator with the Program EvaluationDivision

c. State the duration of your presentation, e.g. My presentation will take about twenty minutes.

d. State when you will field questions in accordance with what is agreed to by the Chair of the committee, e.g. I will need additionaltime to answer your questions and as the chairman has directed, I will be taking your questions at the conclusion of my remarks.

e. Describe any handouts and show them to the group, e.g. You have before you a copy of my slides, a summary of our report, andsome supporting documents.

f. Write a bridging phrase for transition to your message, e.g. Mr. Chairman and members; the most important consideration is…

2. State the message of the presentation, e.g. types of messages:• Assertion. There are major problems with the system for determining eligibility for Medicaid benefits.• Closure. We have completed a draft bill that addresses the sixteen issues contained in your committee’s interim report to the

legislature.• Cost. The proposal will save $1.5 million in state general funds.

3. Write a bridging phrase to summary points, e.g. Mr. Chairman and members, there are three keyfactors to consider…

3

4. Briefly state supporting points:

__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

5. Write a bridging phrase to the full presentation, e.g. Mr. Chairman and members, the

remainder of my presentation provides more detail about our findings and recommendations. Our first finding is…

4

This page is intended to be blank.

5

PROGRAM EVALUATION DIVISION

NORTH CAROLINA GENERAL ASSEMBLY

Report No. 2008-05-1 May 2008

Consolidating Agricultural Research Facility Management Would Improve Efficiency and Effectiveness

Legislation passed by the NC General Assembly during the 2007 session directed the Program Evaluation Division to evaluate the state’s 18 agricultural research stations. The aim of the evaluation was to identify efficiency savings associated with station structure and management, with particular attention to examining whether or not all 18 stations should be owned and managed by a single entity. The evaluation scope included 11 additional research facilities owned and operated by the state land-grant universities.

Summary

Evaluation findings suggest the current divided management structure hinders planning, monitoring, and accountability across agricultural research facilities. In addition, reducing the number of facilities would allow limited funds to be allocated more strategically. Finally, an effective and efficient system for agricultural research is important to North Carolina’s agricultural industry and citizenry.

The General Assembly should consider legislation that creates a system of all agricultural research facilities managed by the two land-grant universities instead of the NC Department of Agriculture and Consumer Services (NCDA&CS). The legislation also should establish an advisory board that includes representatives from NCDA&CS and both land-grant universities to provide guidance to the system from a much-needed statewide perspective. One of the board’s first tasks should be the appointment of an independent panel to conduct a comprehensive review of facilities, including consideration of the recommendations in this report for streamlining the system. Finally, system management should consistently track data on system inputs, resource utilization, and research outcomes and provide annual reports to the public.

As shown in the following table, implementing these recommendations could save up to $3.7 million in recurring and $54.7 million in non-recurring state funds.

Estimated Estimated Recurring Fiscal Nonrecurring

Impact Fiscal Impact Proposed Legislative Action

$ 500,000 $ - Consolidate central management of agricultural facilities (Recommendation 1)

3,171,413 39,856,546 Close seven research stations, pending review of scientific necessity (Recommendation 3)

- 1,299,780 Sell or transfer discrete parcels attached to research stations but not used for research (Recommendation 3)

- 13,590,175 Sell or transfer NCDA&CS forest management tracts (Recommendation 1)

$ 3,671,413 $ 54,746,501 Total Estimated Fiscal Impact

Source: Program Evaluation Division.

6

PROGRAM EVALUATION DIVISION NORTH CAROLINA GENERAL ASSEMBLY

April 2010 Report No. 2010-04

Selling 25 Underutilized Aircraft May Yield Up to $8.1 Million and Save $1.5 Million Annually

Summary Evaluation Purpose. The North Carolina General Assembly directed the Program Evaluation Division to study the number, use, and effectiveness of state aircraft; consider ways to achieve efficiency; and determine if it is desirable or feasible to sell or transfer aircraft.

In Fiscal Year 2008-09, eight state programs employed 89 full-time staff to operate 72 aircraft located at 23 locations across the state at a cost of $10.8 million.

Of the 72 aircraft operated by state programs, 79% flew fewer than 200 hours per year. Fifty-seven aircraft failed to meet the minimum transportation industry threshold of 200 flight hours per year. Agencies flew 36 aircraft (or 50% of the fleet) less than 100 hours. This number includes eight that did not or could not fly at all during Fiscal Year 2008-09. Based on utilization data, the Program Evaluation Division estimates that at least 25 of 72 aircraft and 5 hangar facilities could be eliminated. In addition, decentralized operations among eight independent programs contribute to mismanagement and inefficiency.

The General Assembly should eliminate 25 aircraft and five facilities. Elimination of these aircraft may result in up to $8.1 million in non-recurring proceeds from the sale of aircraft plus $1.5 million in recurring savings. The elimination of five facilities could result in an additional $26,060 in recurring savings.

In addition, the General Assembly should establish the Aviation Management Authority in the Department of Transportation. The authority would oversee management of all aircraft owned or operated by the state. Over a period of two years, the authority would initiate the following tasks:

• consolidate all aircraft used for passenger transport and photogrammetry missions;

• oversee the implementation of recommendations regarding safety issues at the Division of Forest Resources;

• oversee the elimination of 25 aircraft and five facilities; • develop policies and procedures to guide management oversight of

all state aviation resources; • oversee management of all state aviation resources; and • oversee maintenance operations and information management for

all aircraft owned and operated by the state.

7

PROGRAM EVALUATION DIVISION NORTH CAROLINA GENERAL ASSEMBLY

March 2014 Report No. 2014-02

Performance Measurement and Monitoring Would Strengthen Accountability of North Carolina’s Driver Education Program

Summary This evaluation examines driver education in North Carolina as administered statewide by the Department of Public Instruction (DPI) and conducted by local education agencies (LEAs). In response to a 2010 review by the Program Evaluation Division, the General Assembly passed reforms to the program in 2011. DPI allots over $26 million annually to LEAs in State highway funds for driver education, supplemented by fees.

While comprehensive and generally responsive to the 2011 reform law, the DPI strategic plan for driver education lacks objectives and quantitative performance indicators for measuring program activity and effectiveness. The strategic plan for driver education should have contained statewide measures for North Carolina, for each LEA, and for each high school’s driver education program within each LEA, including inputs, processes, outputs, outcomes, and efficiency measures.

Management deficiencies and lack of accountability stem from State Board of Education delegation to LEAs without sufficient DPI oversight. DPI does not collect sufficient and reliable data to determine the efficiency and effectiveness of driver education, does not have a uniform method to deliver driver education statewide, performs no monitoring of LEA instructors, and failed to conduct a valid pilot project for testing the relative effectiveness of online versus traditional instruction. DPI has not collected sufficient data on costs or student participation and has not had the capacity to use data available for measuring driver education outcomes. From 2007 to 2013, 46% of students failed the DMV test, including those making multiple attempts. LEAs are allowed to use a variety of instructional methods including contracting, yet DPI does not know which methods are cost-effective.

North Carolina’s teen accident and fatality rates have declined since the implementation of graduated driver licensing but remain high. Geographic and demographic conditions contribute to North Carolina’s higher teen traffic fatality rates. Nevertheless, parents and teens need to be aware of the elevated risk of traffic accidents.

To address these findings, the General Assembly should require statewide performance measures for driver education; a data-driven outcome monitoring system for student drivers

completing driver education; a feasibility study on offering uniform online classroom driver

education; and standards established by the School of Government at the University

of North Carolina at Chapel Hill for legislatively-directed pilot projects including but not limited to driver education.

8

PROGRAM EVALUATION DIVISION

NORTH CAROLINA GENERAL ASSEMBLY

November 2016 Report No. 2016-10

Medicaid Program Integrity Section is Not Cost-Effectively Identifying and Preventing Fraud, Waste, and Abuse

Summary The Joint Legislative Program Evaluation Oversight Committee’s 2015–17 Work Plan directed the Program Evaluation Division to examine the effectiveness and efficiency of the state Department of Health and Human Services’ Medicaid Program Integrity (PI) Section. The PI Section contributes to the strategic objective of the North Carolina Medicaid program by detecting and preventing fraud, waste, and program abuse, and by ensuring that taxpayer dollars are used appropriately.

Due in part to a lack of access to valid and reliable claim payment data, the number of fraud referrals made by the Program Integrity Section to the state Department of Justice’s Medicaid Investigations Division (MID) declined by 84% from Fiscal Year 2012–13 to Fiscal Year 2014–15. In addition, MID was limited in its ability to pursue prosecutions of the fraud referrals submitted by the PI Section in Fiscal Year 2014–15.

The lack of a formal risk assessment process and performance management information has limited the cost-effectiveness of the PI Section. In Fiscal Year 2014–15, payments to contractors performing pre-claim and post-claim payment reviews exceeded associated savings to state funding requirements by $3.2 million. The PI Section is also not effectively monitoring Medicaid recipient eligibility determinations performed by county departments of social services. Further, the PI Section is not effectively utilizing available information gleaned from reviews of eligibility determinations and medical service claims.

To address these findings, the General Assembly should amend state law to:

adopt a uniform methodology to identify and measure the severityof Medicaid eligibility and medical service claim errors,

provide incentives for county departments of social services toensure the accuracy of Medicaid eligibility determinations,

require the North Carolina Medicaid program, in partnership withthe Office of Administrative Hearings and MID, to identifyalternatives to increase the amount recouped from identifiedoverpayments and the percentage of fraud referrals accepted byMID for further investigation and prosecution,

develop and incorporate a Progressive Corrective Action processfor providers selected for enhanced oversight, and

produce an annual performance report and work plan thatdocuments results and provides a roadmap to reduce fraud, waste,and abuse.

9

Sept

CoFun

Sum

tember 2011

omparended a

mmary

P

N

1

ed to Oand Its

PROG

NORTH C

Other StPlan Fe

As ProCarepRetfor

1.

2.

3.

4.

5.

6.

GRAM

CAROLIN

tates’ Reatures

directed byogram Evaluarolina’s stateport focuses tirement Systrmer governm

Who makekey role in the featureState contriWhat are thavailable tprovides emthat accounparticipateeligible forHow do thsystems? Tthan other sretirement formula muHow woulsystem? It from a defAssembly cvested to reHowever, laState’s costpersonnel tHow is TSEstate contriemployee cthe state co2010, emp$583 millioHow does systems? Tplans for stfunding staperformancassets need

EVALU

NA GENE

Retirems Are Ty

the North Cation Oversie retirement on the Northtem (TSERS) ment employ

es decisions the operatio

es of the retiribute to TSERhe plan featto members mployees wints for years e in at least tr Social Secue plan featuThere are nostates’ plansare typical a

ultiplier are ld altering this not clear tined benefitould changeeduce the Stawmakers hats and the Stto deliver essERS funded?butions, andcontribution aontribution eaployees contron, and inves TSERS fundThe Programtate employetus. TSERS race on these mded to make

UATIO

ERAL AS

ment Playpical

Carolina Genight Committ system to oth Carolina Te because it cyees. This stu

about TSERon of TSERS. rement beneRS. tures of TSER of TSERS? Tith lifetime re of service atwo supplemurity and Staures of TSERo plan featurs. TSERS’s emand its vestiness generous

he plan typethere would t to a definee plan featurtate’s costs inave to weighate’s ability sential public? TSERS is fu investment iat 6% of coach year as ributed $835stment incoming status c

m Evaluation ees and teacanked sixth omeasures ind future paym

ON D

SEMBLY

ans, TSEor Less

neral Assembtee, this studyther state reteachers’ andcovers the mady addresse

S? Several d The Genera

efit and how

RS and whaTSERS is a deetirement incand salary. Mental retirem

ate Health PlaRS compare tres on which mployee contng period, fis than other

e or features be long-termd contributiores for futuren providing ih the tradeo to recruit anc services. nded from eincome. The mpensation part of the 5.8 million, the contributedompare to oDivision rankchers using thout of 84 pladicates the Stments for be

DIVISI

Report N

ERS is Ws Gene

bly’s Joint Ley compares tirement syst

d State Emploajority of cures six resear

different ental Assembly much emplo

at other beneefined benefcome based Members of Tment plans, aan benefits ito other statTSERS is mo

tribution ratenal average states’ planss of TSERS am savings froon plan. The e hires or thoits retirementoff between nd retain qua

employee coGeneral Assin 1975 andbudget proche State cond $5.7 billioother state reked state rethree key means in 2009.tate is accumnefits accrue

ION

No. 2011-05 Well rous

egislative North tems. This oyees’ rrent and ch questions:

tities play a determines yees and the

efits are fit plan that on a formulaTSERS can and they arein retirementte retirementre generous e and normae salary, ands. ffect the om changingGeneral

ose not yet t benefit. reducing thealified

ntributions, sembly set d determines cess. In tributed

on. etirement tirement easures of . Strong mulating the ed to date.

:

e

a

t. t l

d

e

10

PROGRAM EVALUATION DIVISION NORTH CAROLINA GENERAL ASSEMBLY

October 2012 Report No. 2012-10

North Carolina Should Require NC Railroad Company to Pay an Annual Dividend and Strengthen Reporting

Summary The General Assembly directed the Program Evaluation Division to evaluate the North Carolina Railroad Company (NCRR). The legislation required a review of the corporation’s mission, economic development benefits, tangible and intangible value, and governance and required consideration of whether NCRR should be sold, transferred under the North Carolina Department of Transportation or another state agency, or maintain its corporate structure.

NCRR has benefitted from its unique relationship with the State of North Carolina, the corporation’s sole shareholder, but the State has not profited financially from this relationship. NCRR’s capital has increased by at least $196.3 million since 1998 when the State became the corporation’s sole shareholder. As a result, NCRR is both profitable and solvent. However, the State has not profited from this relationship in the form of receiving direct financial benefits because dividends are required by state law to pay for NCRR capital improvements.

Selling NCRR or the railroad corridor may not be in the best interest of the State because these valuable rail assets and their long-term earnings potential would be lost. However, NCRR property not related to the railroad corridor could be transferred to the State as a dividend and sold to provide one-time revenue.

The State has limited mechanisms for oversight of NCRR. As a private corporation, NCRR has less stringent reporting requirements than publicly-traded corporations. NCRR has not fully met all reporting requirements since 2007 and lacks a comprehensive strategic plan and performance measurement system. Changing NCRR’s corporate structure could strengthen the State’s oversight but requires a lengthy and complicated process. Amending state law would improve the corporation’s reporting to the State and require NCRR to make an annual dividend payment while maintaining its corporate structure.

To address these findings, the General Assembly should

amend Chapter 124 of the General Statutes to strengthen reporting by NCRR;

require NCRR to pay a one-time dividend of $15.5 million and, thereafter, require NCRR to pay an annual dividend equal to 25% of NCRR’s annual income from its trackage rights agreement (estimated at $3.7 million annually) to the General Fund; and

require NCRR to convey to the State 14 properties not directly related to the railroad corridor so these properties can be sold and the proceeds deposited into the General Fund.

11

PROGRAM EVALUATION DIVISION NORTH CAROLINA GENERAL ASSEMBLY

December 2014 Report No. 2014-15

Occupational Licensing Agencies Should Not be Centralized, but Stronger Oversight is Needed

Summary As required by Session Law 2013-413, Section 10.(a), the Program Evaluation Division (PED) examined the structure, organization, and operation of the State’s various independent occupational licensure boards as defined by G.S. 93B-1. PED identified 55 occupational licensing agencies (OLAs) with occupational licensing boards that met this statutory definition. While OLAs are state agencies, they receive no state general revenue and are not subject to legislatively mandated spending restrictions. In Fiscal Year 2013–14, these 55 OLAs expended $67.2 million to regulate 703,870 licensees.

The General Assembly should not transfer regulatory authority or administrative responsibilities from OLAs to a central state agency. The transfer may not result in improved performance and would likely entail significant implementation costs to realize potential gains in efficiency.

There is insufficient state-level oversight to ensure OLAs are efficiently and effectively protecting the public. Occupational licensing is intended to help ensure the public is protected from severe physical or economic harm resulting from the associated activities of an occupation. Currently, there is no state-level entity responsible for ensuring OLA compliance with statutorily-mandated reporting requirements. Statutory requirements also do not ensure that OLAs provide reliable information to enable monitoring and evaluation of efficiency and effectiveness.

As identified in Appendix D, the General Assembly should review the authority of 12 OLAs to issue licenses and consider consolidating 10 individual OLAs with other regulatory entities.

To address these findings, the General Assembly should amend North Carolina General Statutes to:

establish an Occupational Licensing Commission that would not function as a central licensing authority but would assist the General Assembly and OLAs in improving effectiveness and resolving disputes,

ensure that the regulatory entities that are required to comply with the associated statutory requirements are clearly defined and listed,

ensure the complaint process used by each OLA includes specified capabilities and attributes, and

establish a legislative subcommittee to evaluate the continuing need for licensing authority for the 12 identified OLAs.

12

Optional Presentation Preview Topics Prepare your preview on some work product with which you are familiar.

• Bill • Fiscal note • Research memorandum • Recent court decision

13