• The Office of Virtual Health (OVH) aims to enable PHSA clinical andacademic programs to reach patients and provide health services remotely.

• Solutions must improve patient outcomes that are clinically led, equitable,adaptable, scalable and sustainable.

• Our evaluation framework intends to measure, monitor, report, andimprove the actions of OVH and the PHSA clinical and academic programson the intended Virtual Health outcomes.

AIM

To describe, monitor, analyze and improve the impact of thePHSA OVH and the approach used on the intended outcomes.

To provide a framework that will result in a consistentapproach and common measures for the Virtual Healthservices that are provided by PHSA programs.

To provide a PHSA-wide approach to monitor, analyze andimprove the impact of both the OVH, and the virtual servicesdelivered by PHSA programs, on the patient, provider andhealth system outcomes.

OBJECTIVES OF FRAMEWORK

PHSA OVH has an evaluation framework that is aligned with quality aspects for clinical service delivery.

• The goal is a consolidated approach for all PHSA programs who are conducting Virtual Health initiatives.

• This will provide the information needed to report the impact of Virtual Health services in the contextof all health services and their alignment with specific goals.

• Improvements can then be made to achieve a system of Virtual Health that is acceptable to patientsand providers, provides improved accessibility to the appropriate services in an equitable way, andimproves the performance of the health system efficiency and effectiveness.

• The evaluation framework for OVH and PHSA clinical programs is currently being used across multiplePHSA VH initiatives.

EVALUATION FRAMEWORK

SAMPLE BALANCED SCORECARD

NEXT STEPS CONTACT US

Office of Virtual Health, Provincial Health Services Authority, BC

Michele Fryer, BsN, MHSA & Kristina Wong, MSc, MBA

A System-wide evaluation framework for Virtual Health in PHSAConnecting

for

health

Michele Fryer, Director, Office of Virtual Health, PHSA michele.fryer@phsa.ca

Office of Virtual Health: officeofvirtualhealth@phsa.ca

Consulted subject matter experts

Conducted literature review

Engaged stakeholders

Aligned with BCPSQC Quality Matrix & PHSA Board metrics

The objectives of the evaluation framework involve reporting on the results ofVirtual Health initiatives. To ensure these initiatives are meeting theirintended goals, robust evaluation using both qualitative and quantitativemetrics are necessary. The goals of this Virtual Health evaluation frameworkare highlighted below:

Intended outcome: A patient centred, clinically led Virtual Health system

Intended impact: Improved patient access, health outcomes and system capacity

Consistent with the PHSA principle of “one system of care,” Virtual Healthservices are not a separate system from in-person health services, andtherefore the evaluation framework also cannot be separate. The resultingframework is based on the quality indicators from BC Health Quality Matrixwith adaptation to make the measures and outcomes specific to Virtual Health.

METHODS

• Implement the evaluation framework across PHSA andreport organization, program and clinic outcomes andimpact of Virtual Health.

• Create and implement improvement plans based on results.

Program level Aggregate of measures within programs

1. OVH evaluation: to assess the structure and processes used by the OVH in leading and facilitating the adoption of Virtual Health services by clinical programs (delivery of the Virtual Health strategy).

2. Clinical program evaluation: to provide a consistent approach and common measures for PHSA clinical programs in delivering Virtual Health services.

PHSA level (PHSA-wide)Aggregate of measures across programs

Clinic levelIndividual clinic measures

DIP MHSUS

Kelowna Victoria Prince George

BC Cancer BCCDC BC Women’s

Benchmarking and comparison

Results

reporting

Widely applicable templates

Report content examples1. Customer impact: dimensions of

quality1.1 Acceptability1.2 Accessibility1.3 Appropriateness1.4 Effectiveness1.5 Safety

2. Internal processes2.1 Efficiency 2.2 Equity2.3 Infrastructure development

3. Feedback on vendor solution3.1 VH-Visits3.2 Digital messaging3.3 Appointment reminders3.4 2-way document sharing

4. Lessons learned5. Financial implications

The average score on overall experience with VH-visits,

indicating high satisfaction from both patients and providers.

8/10

Source: BC Cancer-Victoria, Virtual Health Visits project, Aug 2018

0 2 4 6 8 10 12 14

More

Same

Less

N/A

Disruption and difficulty of VH-visits(n=16)

Ability for family or friends was less/same/more DIFFICULT

Amount of disruption to patient daily routine was less/same/more DISRUPTIVE

No safety concerns

identified by providers during

Virtual Health Visits.

37%

14%

32%

17%

Patient updateJuly - September 2018 (n=117)

Interested (agreementsigned)

Interested (agreementnot signed yet)

Not Interested

Undecided