fourth international conference on online deliberation (od2010) july 1st, 2010

Tools and Technologies in eParticipation: Insights from Project

Evaluation

Melanie Bicking and Maria A. Wimmer University of Koblenz (UKL)

{bicking | wimmer}@uni-koblenz.de

Fourth International Conference on Online Deliberation (OD2010)July 1st, 2010

Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010

Table of Content

Problem scope

eParticipation Projects Evaluation Methodology

Insights from Project Evaluation

Outlook

-2-


Overall scope of eParticipation Project Evaluation

Evaluate eParticipation project results (platforms) in terms of

Processes supportedPolicies supportedTopics discussedTools and technologies used

Assess success and potential impact of project resultsInvestigate dialogue between stakeholders and the public over time and its outcomes

Receive feedback from each interested stakeholder who is interested in using respective eParticipation platforms of the projects

-3-


Table of Content

Problem scope



Outlook

-4-


Assets of eParticipation projects to be assessed

Tools and Technology: ICT deployed in the field of application and used for participation

Processes supported: different stages within the legislation process

Topics tackled: different topics processed in the legislation process

Policies supported:strategic directions and policies addressed in the project

-5-



Assets to be assessed

KeySuccess and Impact Evaluation criteria

Point of departurePlatforms of eParticipation projects

Processes supported

Tools / Technologies

deployed

Topics discussed

Policies supported

Usability Perceived usefulness

Policies met

Interest Perceived Importance

Appropriateness Appeal

Attractiveness

-6-


Assessing the impact

By addressing different target groups via

Internal assessments

External assessments

By assessingThe quality of reaching the target groups

Whether sustainable interest has been reached

-7-


The two parts of evaluation

Internal assessment External assessment

Self-Assess

-ment

Structured: Questionnair

e

End user

Question-

naire

Structured

Experts inthe field

Semi-structured: SWOT-

Analysis

Peer Review

Evaluation reports

ReportsUn- structure

d

-8-


UsabilityAppropriateness

AppealAttractiveness

Platforms of eParticipation projects

Tools / Technologies

deployedProcesses supported

Topics discussed

Interest, Perceived Importance

Policies met

Policies supported

Overall view on evaluation framework

(external)

End-user Questionnaire

Expert Questionnaire

Peer review

Evaluation project reports

Self-assessment Questionnaire

SWOT-analysis

(internal)

Evaluation approach

Assets to be assessed

Key evaluation criteria

Impact assessment features

Addressing target groups through instruments of evaluation

Point of departureWhat to evaluate?

How to evaluate?

Main target of evaluation: impact towards target groups

Instruments of evaluation

Target groups

ImpactReaching the target group Achieving sustainable interest

-9-


Table of Content

Problem scope



Outlook

-10-

Means to establish contacts with target group

Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 -11-

Means applied to establish contacts with the most impact


Degree of innovation of tools and technologies deployed


Usability of tools and technologies


Web Content Accessibility Guidelines (WCAG)


Overview of the eParticipation tools deployed and used by target users


Lessons learnt and best practices

Aspects hindering eParticipation projects to succeed

Lack of support from elected representatives resulting not only but also in wanting acceptance of the citizens

Past oriented thinking and acting, as well as culture of elimination of errors thereby remaining in tried and tested/trusted routines (this is in particularly true for the use of ICT)

Confusing information, lack of transparency and accessibility

Lack of sharing information with other projects to learn from their experiences or to create synergies

Lessons learnt from executing eParticipation projects evaluation

An evaluation of the impact is methodologically difficult

Engaging end users in the assessment of the projects was problematic

Better using mail surveys than online surveys



Table of Content

Problem scope



Outlook

-18-

Toward further analysis

The paper bases on parts of ongoing monitoring and evaluation of eParticipation projects funded under the eParticipation Preparatory Action of the EC

Evaluation passes through an iterative process including internal and external assessments of the projects

projects started and finished or will finish at different points in time

At the moment the 2nd round of expert evaluation is running

Results will be available in a few months includinga comparative analysis of 1st and 2nd round of Expert Evaluation

the projects that started at the beginning of 2009 have now advanced to the point that their pilots are live and evaluation of their pilots will start soon involving Self-Assessment, Expert Evaluation, and Peer Review


Expected outcomes

At the end, a comprehensive and wide-ranging evaluation of the projects funded under the eParticipation Preparatory Action of the EC will be provided

The study provide feedback on the single projects and calls, as well as on the eParticipation Preparatory Action itself

From this, profound insights into eParticipation and the design and conduction of eParticipation projects are expected thereby providing helpful lessons learnt and recommendations for future eParticipation initiatives to the EC and other designated stakeholders.



Thank you for your attention!

Melanie Bicking and Maria A. Wimmer University of Koblenz (UKL){bicking | wimmer}@uni-koblenz.de

-21-

fourth international conference on online deliberation (od2010) july 1st, 2010

Documents

online deliberation

project evaluationoutlook

impact evaluation criteria

evaluation frameworkassets

parts of evaluation

assessedkey success

different topics

different stages