copyright rso ® spa, milano framework contract 30-ce-0121765/00-57 (supply of services for the...

Copyright RSO® SpA, Milano

Framework Contract 30-CE-0121765/00-57(Supply of services for the further development, reinforcement

and promotion of benchlearning)

Benchlearning Final ConferenceMeasuring eGovernment Impact

Giancarlo Senatore

RSO Senior Partner

Ghent, Nov 30th, 2009

Copyright RSO® SpA, Milano 2

THE PROJECT

THE PILOTS

BENCHLEARNING

BENCHLEARNING COMMUNITY

INDEX


The project

Rooted in eGEP’s findings – eGovernment Economics Project, Benchlearning is a mean … to test the comparability of impact indicators, to build measurement awareness and capacities, to share good practices.

On a voluntary and flexible basis, 12 public agencies covering 10 European countries have freely committed themselves to join 3 Pilot eGovernment Benchlearning exercises on a 2-yearly time span.

Through a systematic data gathering, the Agencies will prove whether eGovernment services and applications are finally delivering the expected outcomes.


Expenditure Study

eGOV costs monitoring methodology

Expenditure estimate for EU25

• Total ICT: € 36.5 billion (2004)

• eGOV only: € 11.9 billion (2004)

Measurement Framework

About 90 indicators

Implementation methodology

Measurement Framework

EconomicModel

eGovernment Productivity GDP Growth

Scenarios show that future eGovernment research and programmes (2005-2010) could boost EU25 GDP

by up to 1.54 percent

Back to 2005: the eGovernmentEconomics Project (eGEP)


Net Costs

Set-up

Provision

Maintenance

Efficiency

Democracy

Cashable financial gains

Better organisational and IT architectures

Better empowered employees

Inter-institutional cooperation

Openness and participation

Transparency and accountability

Political Value

Financial & organisational Value

Constituency Value

Effectiveness

Reduced admin. burden

Increased user value & satisfaction

More inclusive public services

The eGEP measurement framework


eGEP

EU25 Benchmarking

Very simple fully comparable

indicators

EU must agree with•Member States indicators•Methodology for new benchmarking

National level monitoring of

eGOV

Less simple indicators,

some comparability

problems

A national level unit can•Impose indicators top down•Build consensus on most comparable ones

Micro-level business case & measurement

A public agency can:• Select any of eGEP indicators• First use them for ex ante

business cases• Then for steady and

continuous measurement• Use eGEP implementation

tools

Sophisticated indicators, no comparability

problems

How could it be used?


Overall scoreTypeInternational (EU25)

Difficulties score•Cooperation•Comparability•Feasibility

Policy system benchmark

•Cooperation•Comparability•Feasibility

Public policy benchmark


Organisational benchmark

4=High; 3=Medium-high; 2=Medium; 1=Low; 0= null


Measurement/Internal benchmark

Member State (holistic)

Member State (within vertical and/ or region)

Level

Individual public agency (voluntary)

444 333 213 102

HIGH

MEDIUM

LOW

MEDIUM-HIGH

Lessons from eGEP:impact measurement difficulties


A bottom-up collaborative benchmarking based on a peer-

to-peer experimental exchange among fairly comparable public

agencies from at least two different EU Member States,

designed as a symmetric learning process, that (…) will

implement and calculate more sophisticated indicators in a

chosen area of impact the ICT enabled services the selected

agencies provide and in the process will build transformative

capacities.

2008-2009: the Benchlearning challenge


Why to benchlearn?

To benchmark only some eGEP indicators:

•The simplest and more comparable.

To boost the public sector’s impact evaluation capabilities:

•Focus on most sophisticated impact indicators;•Measurement capacities are built bottom-up.

To provide the involved agencies with tangible benefits:

•Opportunity to look at processes complexity;•Identify enabling and hindering factors.


Benchlearning is:•Voluntary, bottom up and learning oriented;•Flexible, with no need of uniform rigid indicators.

Gradually scalable from micro to meso and macro:

•Groups of similar organisations;

•Groups of similar verticals / regions;

•Groups of similar countries.

Provides insights and learning on the eGOV value chain:

•Key drivers and success factors;

•Main barriers;

•Organisational processes and input.

Benchlearning = bottom-up benchmarking


Analysis of eService set-up and delivery processes

Elaboration of new impact indicators

Peer-to-peer exchange of experiences

Transfer knowledge on measurement tools

To understand the success factors and barriers behind

processes

To collaboratively test its feasibility and comparability

To build awareness on good practices

To enable PAs measure their own performance

To extrapolate the promising areas where EU can become a global leaderTo extrapolate the promising areas where EU can become a global leader

ACTIVITYACTIVITY AIMAIM

What benchlearning is aiming to achieve


Ag.1Ag.8

Ag.13

vs

TEST AND LEARNING

TEST AND LEARNING

Ag.1 Ag.2 Ag.3

Ag.4 Ag.5 Ag.6

Ag.7 Ag.8 Ag.9

Ag.11 Ag.12 Ag.13

BEST IN CLASS

BEST IN CLASS

Benchlearning vs. benchmarkig


RANKINGRANKINGCAPACITIES AND LESSONS

CAPACITIES AND LESSONS

21

3

Ag.13

Ag.8

Ag.1

vsAg.1

Ag.8

Ag.13

Benchlearning vs. benchmarking: outcomes


First year measurement

Set-up:Letter of intent from the participating agencies;Running of a kick-off meeting with all participating agency.

As is and mainstreaming:Review of existing measurement systems and data;Analysis of organisational strategy and context;Draft report on indicators and preliminary measurement.

First full measurement (or zero measurement):Data gathering instruction to agencies;Remote support to agencies to gather the data;Validation of data and calculation of indicators.

How benchlearning works (1/2)


Set-up (same as Y1);

As is and mainstreaming (same as Y1);

Second full measurement (same as Y1).

Continuous exchange of information (

www.epractice.eu/community/benchlearning);

Inter-agency workshops.

Provision to the agencies of a measurement organisational

model (processes and roles);

Final recommendations and final report.

Second year measurement

Exchange activities

Sustainability actions

How benchlearning works (2/2)

By end of Dec 2009


Public sector information indicator

Efficiency indicator

Simplified version of the eGEP Measurement Framework: eGEP 2.0Simplified version of the eGEP Measurement Framework: eGEP 2.0

EFFICIENCYEFFICIENCY

Standard cost model based indicator

Work in progress on number of data field

indicator

ADMINISTRATIVE BURDEN REDUCTION

ADMINISTRATIVE BURDEN REDUCTION

Plurality of subjective and objective metrics

Work in progress on a combined index

CITIZEN CENTRICITYCITIZEN CENTRICITY

Pilot 1 Pilot 2 Pilot 3

Expected project outcomes: eGEP 2.0


Pilot 1 Efficiency gains

Agenzia del Territorio (Italian National Cadastre) Oficina Virtual del Cadastro (Cadastre Virtual Office of Spain) Lantmäteriet (National Land Survey of Sweden)

Observers: Regional agencies (Emilia-Romagna, Catalonia)

Agenzia del Territorio (Italian National Cadastre) Oficina Virtual del Cadastro (Cadastre Virtual Office of Spain) Lantmäteriet (National Land Survey of Sweden)

Observers: Regional agencies (Emilia-Romagna, Catalonia)

About the Pilot

Agencies involved:

Pilot measurement of “efficiency gains” of cadastral eServices: Development of a set of indicators to measure the efficiency gains and

savings due to the delivery of cadastral eServices; Definition of the indicators in line with the agencies’ requests to assess the

added value of online cadastral information supply in terms of internal and social impact;

Data gathering and analysis (volumes, eService costs, organisation and customer satisfaction programs);

Shared experiences among agencies.


Pilot 2 Administrative burden

About the Pilot

Agencies involved:

Piloting of indicators in the Administrative Burden Reduction Field Proof-of-concept service: Business registration Data sources: Standard Cost Model Focus groups and interviews Methodology:

Specific focus on users (both business and civil servants)Qualitative and quantitative approach to burden measurement

Pilot agencies G2B: Belgium: FPS Economy Slovenia: Ministry of Public Administration Greece: Ministry of Public Administration

Project Observers: Fedict, Dutch Ministry of Interior, Greek Information Society Observatory

Pilot agencies G2B: Belgium: FPS Economy Slovenia: Ministry of Public Administration Greece: Ministry of Public Administration

Project Observers: Fedict, Dutch Ministry of Interior, Greek Information Society Observatory


Pilot 3 User centric impact

DirectGov (British eGovernment portal) Service-public.fr (French eGovernment portal) Mojauprava (Croatian eGovernment portal)

Additional eGovernment portal data collected from Italy and Hungary

DirectGov (British eGovernment portal) Service-public.fr (French eGovernment portal) Mojauprava (Croatian eGovernment portal)

Additional eGovernment portal data collected from Italy and Hungary

About the Pilot

Pilot measurement of “user centric impact” of national government portals: Understand what the EC has already done on measuring e-Government and

specific indicators developed for user-centric impact and benchmarking national portals

Development of a measurement framework that can be used to benchmark the user-centricity of national government portals. The framework measures 5 key aspects: content richness, service sophistication, user choice and control, quality control and design for usability

Shared experiences among agencies.

Agencies involved:


Micro level only, single

public organisations:

•3-4 learning organisations managing homogeneous services

Groups assembled from similar

countries:

•Leverage existing collaboration

networks.

Third party facilitators (EU

contractors, governments…):

• Intense and in-depth work.

Start simple Voluntary participation:

•Participants self-interested in capacity building/learning.

Clear mandate and leadership buy-in:

•Groups to be assembled

not by facilitator.

Multi-stakeholders but

firm governance:

•Exchange and consensus

•But with clear lines of

accountability.

Generate ownership

Benchlearning groups: how to manage them


Having a say in the blog

session

Having a say in the blog

session Suggesting and attending

new events

Suggesting and attending

new events

Recommending documents for the Community

library

Recommending documents for the Community

library

Exchanging ideas on eService

impact measurement and

evaluation

Exchanging ideas on eService

impact measurement and

evaluation

Sharing experience and cases

Sharing experience and cases

Benchlearning community within ePractice.eu


What results will be presented tomorrow

What kind of indicators we selected for the measurement of Efficiency Gains,

Administrative Burden Reduction and User Centricity;

How we managed the comparability issues due to the structural differences of

public services in the different countries;

How we took into account the organisational process and the eService costs for the

service provision in each country;

What have been the results of terms of learning;

How we quantified the benefits from eGovernment;

ICT-enabling of processes leads to significant savings both for businesses and

internally within governments.

The results obtained in the pilots illustrate the business cases for better and more

convenient eGovernment solutions.

…

…

copyright rso ® spa, milano framework contract 30-ce-0121765/00-57 (supply of services for the...

Documents