Development of Evaluation Tool (KPI) for e-Gov Development

Veselin StoyanovManagement consultant, ICB

11 Mar 2011 1BG-KR eGov Experts Workshop: <TITLE>

Main Problems Related to eGov Implementation

Current situation: Clear and well structured e-governance strategy 2011-2015 The achievement of the strategy is not consistently measured. Focus on long-term general KPI. It is not clear how projects are prioritized. Information about project results is difficult to obtain.

Lack of methodology for evaluation of eGov can lead to: Inadequate results in the development of e-government Inappropriate selection of projects for realization Poor project management is not discouraged Lack of trust from stakeholders

What Is the Right Approach for Evaluation of eGov?

1. eGov KPI best practices research (EU, South Korea)

2. Analyze the objectives of Bulgaria for e-governance

3. Determine KPI to measure the objectives

4. Determine project and e-service selection criteria

5. Determine project execution KPI

6. Analyze KPI data sources

7. Propose methodology for calculation of each KPI

Project methodology

• Use proven methodology and tested KPIBalanced Scorecard Strategic Management System (BSC)

• Adapt best practices and add KPIs based on challenges specific for Bulgaria

• Evaluation must be based on S.M.A.R.T. objectives!

Balanced Scorecard (BSC)

Introduced by Harvard Business School professors Norton and Kaplan

One of the most popular and successful strategic management tools

•The strategic objectives and the KPIs are arranged in 4 major perspectives. •The Customer objective has the highest priority in non-profit organizations.

Financial

Human, Information and Organizational Capital

Internal Business Processes

Customer

ObjectiveObjective

ObjectiveObjective

ObjectiveObjective

ObjectiveObjective

General eGov perspective

Clear performance objectivesStrategic Objective

Reduction of time,efforts and price for usageof administrative servicesas well as for search andaccess to personal and

public data

e-service usersatisfaction

Plan value: 90

Tolerance range: 10

Repetitiveusage ofe-service

Plan value: 100

Tolerance range: 20

Number of clicksto perform service

Plan value: 3

Tolerance range: 2

Increase in theusage of e-service

Plan value: 25

Tolerance range: 5

Price reductionof the service

for citizens

Plan value: 20

Tolerance range: 5

Important!The key performance

indicators for measuring the success of the projects are set at the beginning.

KPI

Mistakes:The different projects are evaluated

„ad hoc” and the decision for realization is taken without clear analyses of their strategic importance.

Project X

Avg. total costs: 30000.0 EUR

Project Y

Avg. total costs: 45000.0 EUR

Str

ateg

ic p

roje

cts

Cre

atio

n o

f el

ectr

on

ic 

reg

iste

r in

ag

ency

XIm

ple

men

tati

on

of

elec

tro

nic

 d

ocu

men

t w

ork

flo

w i

n

min

istr

y Y

Imp

lem

enta

tio

n o

f IT

IL

Dev

elo

p e

-go

vern

ance

ed

uca

tio

n c

ou

rses

Pro

ject

x

Pro

ject

y

Perspectives Strategic objectives Performance PrioritizationCustomer Improve quality of e-services 1 6 0.3

Inclusive e-services 0.84 8 0.5Internal Business Processes

Increase paperless document workflow inside and between administrations

0.343 0.8 0.7

Processing time optimization 0.46 12 0.5 0.4 0.3Reuse of existing technology and knowledge

0.339 0.5

Human, Information and Organizational Capital

Improve the administrative capacity0.32

10 0.2 0.4

Improved security and reliability of information and integration systems and the related e-services

0.2811 0.5 0.5 0.6

Financial Planning, control and transparency of administrative costs and project management

0.211 0.3

Performance 0.64 0.727 0.67 0.19 0.42 0Prioritization 3 1 2 5 4 6

Ave

rag

e to

tal

cost

s

30,0

00E

UR

25,0

00 E

UR

25,0

00 E

UR

10,0

00 E

UR

50,0

00 E

UR

12,0

00 E

UR

Selecting Projects for RealizationLinking the expected contribution

of each project with the fulfillment of the strategy

Structured description of

the e-governance strategy

Strategic objectives

Projects that contribute to the strategic objectives

Projects priority

Projectsbudget

Result:The prioritization of projects

allows the most efficient use of financial resources.

Improved quality andaccessibility of e-services.efficiency and transparency of the public administration

High customersatisfaction

Optimal e-processes

Improved datasecurity

Low process costs

Objective 1

Objective 2

Objective 3

Objective 4

Monitoring and Measurement

Result:Objective measurement of the

degree of achievement of the government’s goals.

Reduction of time,efforts and price for

usageof administrative

services...

Plan value: 1Actual value: 0.832

e-service usersatisfaction

Plan value: 90Actual value: 78

Repetitiveusage ofe-service

Plan value: 100Actual value: 101

Number of clicksto perform service

Plan value: 3Actual value: 5

Increase in theusage of e-service

Plan value: 25Actual value: 19

Price reductionof the service

for citizensPlan value: 20Actual value: 16

Challenges:Information required for the

calculation of KPI may not be readily available (confidential or requiring research)

Results of the project so farDefined set of important

KPIs for eGov evaluation

Customer

Improve quality of e-services

Average response time per service

Service availability

Number of complaints received from citizens about e-service

Detected system errors in e-service execution

Citizen participation in the democracy

processes

Accessibility of e-government services and applications

Multichannel access to e-services

Barrier-free access to desktop, mobile and web applications (for people with disabilities and special needs)

Availability of online public and private information regarding administrative service

Inclusive e-services Unified and user-friendly interfaces for access to e-services

Reduction of time, efforts and price for

usage of administrative

services as well as for search and access to personal and public

data

User satisfaction

Repetitive usage of e-service

Number of clicks to perform service

Increase in the usage of an e-service

Price reduction of the service for citizens

Increase citizen awareness

Availability of advertisements in "key" media

Knowledge of the existence of the service

New service users

Strategic perspective

KPIs

Strategic objectives

Results of the project so far

KPI User satisfaction Type 

Lagging

Category Reduction of time, efforts and price for usage of administrative services as well as for search and access to personal and public data

Calculation Survey using questionnaire with likert scale rating questions. Could be placed online on the portal where the electronic service is offered.

Unit%

Plan value 2011

 

 

Detailed description of the KPI: Leading and lagging indicators

are applied

Further developments of the project

• Identify the most important KPIs from the defined list• Weights must be assigned to the different KPIs to denote their

importance. • The government must accept the KPIs and suggest plan values.

Publishing the Results to Achieve Complete Transparency

Results:Publishing the results

from the project and the invested financial resources for stakeholders to see.

A portal should be developed that will present:•The Bulgarian e-governance strategic objectives.•The KPIs' plan and actual values.•The proposed projects for realization, their contribution to the strategy and a clear ranking of the projects.•For each realized project the project KPIs should be reported on this portal.

ITCC eGov Evaluation Tool • Clear vision• Clear methodology• Transparency