virginia department of health professions core competency: board performance measured

1Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144

Virginia Department of Health Professions

Core Competency:

Board Performance Measured

October 22, 2008Neal Kauder, President, VisualResearch, Inc.


Two Major Projects Underway

1. Agency-wide Performance Measurement

2. Sanctioning Reference Points (SRPs)


Note: Vertical scales on line charts change, both across boards and measures, in order to accommodate varying degrees of data fluctuation.

CNA - In Q1 2009, the clearance rate was 116%, the Pending Caseload older than 250 business days was 28% and the percent closed within 250 business days was 80%.Q1 2009 Caseloads:Received=138, Closed=160Pending over 250 days=96Closed within 250 days=110

Nurses - In Q1 2009, the clearance rate was 102%, the Pending Caseload older than 250 business days was 18% and the percent closed within 250 business days was 80%.Q1 2009 Caseloads:Received=313, Closed=319Pending over 250 days=118Closed within 250 days=224

Clearance Rate

Nursing - In Q1 2009, the clearance rate was 106%, the Pending Caseload older than 250 business days was 22% and the percent closed within 250 business days was 80%.Q1 2009 Caseloads:Received=451, Closed=479Pending over 250 days=219Closed within 250 days=334

Age of Pending Caseload Percent Closed in 250 Business Days(percent of cases pending over one year)

0%

50%

100%

150%

200%

250%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

10%

20%

30%

40%

50%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

10%

20%

30%

40%

50%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

50%

100%

150%

200%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

50%

100%

150%

200%

250%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%

10%

20%

30%

40%

50%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

90% Goal

90% Goal

90% Goal

25% Goal

25% Goal

25% Goal

100% Goal

100% Goal

100% Goal

Agency-wide Performance Measurement



Dentistry - In Q1 2009, the clearance rate was 202%, the Pending Caseload older than 250 business days was 45% and the percent closed within 250 business days was 81%.Q1 2009 Caseloads:Received=58, Closed=117Pending over 250 days=87Closed within 250 days=75

Clearance Rate

Medicine - In Q1 2009, the clearance rate was 186%, the Pending Caseload older than 250 business days was 26% and the percent closed within 250 business days was 92%.Q1 2009 Caseloads:Received=267, Closed=497Pending over 250 days=96Closed within 250 days=422

Pharmacy - In Q1 2009, the clearance rate was 125%, the Pending Caseload older than 250 business days was 9% and the percent closed within 250 business days was 77%.Q1 2009 Caseloads:Received=59, Closed=74Pending over 250 days=10Closed within 250 days=51


0%

100%

200%

300%

400%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

15%

30%

45%

60%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

10%

20%

30%

40%

50%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

50%

100%

150%

200%

250%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

50%

100%

150%

200%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%

10%

20%

30%

40%

50%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

90% Goal

90% Goal

90% Goal

25% Goal

25% Goal

25% Goal

100% Goal

100% Goal

100% Goal




Counseling - In Q1 2009, the clearance rate was 71%, the Pending Caseload older than 250 business days was 0% and the percent closed within 250 business days was 100%.Q1 2009 Caseloads:Received=7, Closed=5Pending over 250 days=0

Closed within 250 days=4

Clearance Rate

Veterinary Medicine - In Q1 2009, the clearance rate was 144%, the Pending Caseload older 250 business days was 3% and the percent closed within 250 business days was 93%.Q1 2009 Caseloads:Received=34, Closed=49Pending over 250 days=2Closed within 250 days=39

Social Work - In Q1 2009, the clearance rate was 180%, the Pending Caseload older than 250 business days was 17% and the percent closed within 250 business days was 78%.Q1 2009 Caseloads:Received=5, Closed=9Pending over 250 days=2Closed within 250 days=7


0%

100%

200%

300%

400%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

15%

30%

45%

60%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

10%

20%

30%

40%

50%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

50%

100%

150%

200%

250%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

50%

100%

150%

200%

250%

300%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%

15%

30%

45%

60%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

90% Goal

90% Goal

90% Goal

25% Goal

25% Goal

25% Goal

100% Goal

100% Goal

100% Goal





Optometry - In Q1 2009, the clearance rate was 200%, the Pending Caseload older than 250 business days was 0% and the percent closed within 250 business days was 86%.Q1 2009 Caseloads:Received=4, Closed=8Pending over 250 days=0Closed within 250 days=6

Long-Term Care Administrators - In Q1 2009, the board recieved no new cases, and closed 4 cases. No cases were pending over 250 business days, 100 percent of cases cosed within 250 business days.Q1 2009 Caseloads:Received=0, Closed=4Pending over 250 days=0Closed within 250 days=4

Clearance Rate

Psychology - In Q1 2009, the clearance rate was 160%, the Pending Caseload older than 250 business days was 18% and the percent closed within 250 business days was 86%.Q1 2009 Caseloads:Received=5, Closed=8Pending over 250 days=2Closed within 250 days=6

0%

100%

200%

300%

400%

500%

600%

700%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

10%

20%

30%

40%

50%

60%

70%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

10%

20%

30%

40%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

100%

200%

300%

400%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

100%

200%

300%

400%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%

10%

20%

30%

40%

50%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

90% Goal

90% Goal

90% Goal

25% Goal

25% Goal

25% Goal

100% Goal

100% Goal

100% Goal





Audiology - In Q1 2009, the board closed 2 cases and received none. 33% of the Pending Caseload was older than 250 business days; no cases were closed within 250 business days.Q1 2009 Caseloads:Received=0, Closed=2Pending over 250 days=2

Closed within 250 days=0

Funeral - In Q1 2009, the clearance rate was 75%, the Pending Caseload older than 250 business days was 0% and the percent closed within 250 business days was 100%.Q1 2009 Caseloads:Received=4, Closed=3Pending over 250 days=0Closed within 250 days=3

Clearance Rate

Physical Therapy - In Q1 2009, the clearance rate was 100%, the Pending Caseload older than 250 business days was 50% and the percent closed within 250 business days was 100%.Q1 2009 Caseloads:Received=2, Closed=2Pending over 250 days=3Closed within 250 days=2

0%

100%

200%

300%

400%

500%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

20%

40%

60%

80%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

100%

200%

300%

400%

500%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

50%

100%

150%

200%

250%

300%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

90% Goal

90% Goal

90% Goal

25% Goal

25% Goal

25% Goal

100% Goal

100% Goal

100% Goal




Clearance Rate

Medicine - In Q1 2009, the clearance rate was 186%, the Pending Caseload older than 250 business days was 26% and the percent closed within 250 business days was 92%.Q1 2009 Caseloads:Received=267, Closed=497Pending over 250 days=96Closed within 250 days=422


0%

10%

20%

30%

40%

50%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

0%

50%

100%

150%

200%

250%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%

25%

50%

75%

100%

Q1 05 Q1 06 Q1 07 Q1 08 Q1 09

90% Goal

90% Goal

90% Goal

25% Goal

25% Goal

25% Goal

100% Goal

100% Goal

100% Goal

Data will be interactively visual and accessible (downloadable in Excel)

Q1 05 295 160 54%

Q2 05 303 190 63%

Q3 05 308 290 94%

Q4 05 338 204 60%

Q1 06 329 322 98%

Q2 06 334 220 66%

Q3 06 337 260 77%

Q4 06 299 268 90%

Q1 07 316 202 64%

Q2 07 278 237 85%

Q3 07 350 238 68%

Q4 07 294 463 157%

Q1 08 235 329 140%

Q2 08 247 439 178%

Q3 08 285 569 200%

Q4 08 258 558 216%

Q1 09 267 497 186%

Received ClosedClearance

Rate

Q1 05 906 27%

Q2 05 1019 27%

Q3 05 1037 29%

Q4 05 1171 29%

Q1 06 1178 30%

Q2 06 1292 32%

Q3 06 1369 33%

Q4 06 1400 35%

Q1 07 1514 37%

Q2 07 1555 38%

Q3 07 1662 40%

Q4 07 1499 42%

Q1 08 1409 45%

Q2 08 1210 45%

Q3 08 934 32%

Q4 08 626 26%

Q1 09 363 26%

% of Pending

# Pending 250 days or more

Q1 05 147 85%

Q2 05 181 83%

Q3 05 272 81%

Q4 05 178 78%

Q1 06 287 81%

Q2 06 191 77%

Q3 06 231 79%

Q4 06 227 69%

Q1 07 163 74%

Q2 07 207 54%

Q3 07 208 69%

Q4 07 391 67%

Q1 08 259 73%

Q2 08 356 69%

Q3 08 410 50%

Q4 08 479 83%

Q1 09 459 92%

# Closed w/in 250 days

% Closed w/in 250 days

Board of Medicine



Data will be interactively visual and accessible (downloadable in Excel)

• PDF file linked to Excel files that automatically open – we can implement in the

short-term, quickly (L2K reporting to come....)

• All files on a shared drive – accessible by many or restricted access

• A “dashboard” approach

• Management can gauge performance in a paperless, easy to understand

visual format.

• Can also print on PC printers if desired.



Other Performance measure work:

• Narrowing case category codes for more valid and reliable recording of

patient care vs. non-patient case

• 2 KPM measures by stage – enforcement vs. board

• Recommendations related to old case standards

• Examining impact of new agency policies on performance

• Recidivism concept to be explored as an internal management tool



Recidivism as possible internal management measure...

• Multiple measures of recidivism possible

• Measure which respondents return after a violation and sanction

are handed down

• How long to recidivate, and for what type of “case” – similar to

original case?



Purpose

“…to provide an empirical, systematic analysis of board sanctions for

offenses and, based upon this systematic analysis, to derive

reference points for board members and an educational tool for

respondents and the public”

Virginia Board of Health Professions, Work plan, Spring 2001

Sanctioning Reference Points


SRPs are:

• Data driven, historically based worksheets that assess factors that predict

sanction outcomes

• Factors include those board members and staff indicate are important

• Designed so similarly situated respondents receive similar sanctions

• Designed to be modified as history changes, not static

• Voluntary; reasons for departing are recorded



Medicine sample worksheetAdopted August 2004

Sparrow 010199999

60

30

80

60

50



Board Start DateTotal Number

of Cases

Overall Agreement

Rates

BHP Overall 664 80%

Medicine Aug-04 92 70%

Dentistry Jul-06 73 85%

Nursing (Nurses and CNAs) Jun-06 466 82%

Funeral May-07 9 89%

Veterinary Medicine May-07 31 52%

Pharmacy Nov-07 17 82%

Optometry Dec-08 2 50%

*

* November 6th, 2008 – Veterinary Board will be considering adjusting SRP worksheet to include CE as possible sanction at the lowest thresholds. This is consistent with other boards more recent practice as well.

SRP Agreement Rates(through October 1, 2008)



SRP Example of Departure Reasons Cited

Mitigating (less harsh than SRP result):

• Isolated incident

• Closing practice/ No longer practicing

• No patient involvement

• Sincerity/ Remorse

• Corrective action take

• Actions were not the cause of injury

Aggravating (more harsh than SRP result):

• Drug Abuse/ Diversion

• No call/ No show

• Repeat SOC issues

• Sexual abuse with patient harm

• Severity of allegation

• Denial of addiction problem

• Multiple allegations

• Failed drug screens



Optometry was the first Board to incorporate CCAs into the SRP system...



Remaining Boards:

Behavioral sciences

Psychology and Social Work have approved draft WS

Meeting with Counseling Nov. 14

Move forward to draft SRP manual

Train Board members and staff on use

Physical Therapy, Long-Term Care, Audiology & Speech PathologyReviewed limited number of cases

Interviewed board members that hear cases

Preparing draft data collection instrument

Collect data, draft worksheet



Evaluating the Effectiveness of Sanctioning Reference Points

Primary Outcomes

1. Consistency

2. Proportionality

3. Neutrality

4. Board and staff satisfaction

Secondary Outcomes:

1. Transferability of SRP system

2. Case processing time

3. Violation, CCA, or Consent Order rates change

Possible outcomes to measure:



That’s all!

Questions or comments?

virginia department of health professions core competency: board performance measured

Documents