online evaluation for local government information unit
DESCRIPTION
TRANSCRIPT
![Page 1: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/1.jpg)
Introduction
• About me• About Involve• About evaluation...we will look at;
• Why we might evaluate• How we might evaluate• What we might measure
![Page 2: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/2.jpg)
"Not everything that can be counted counts, and not everything that counts can be counted."
Albert Einstein’s deskat Princeton University
![Page 3: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/3.jpg)
“Evaluation of online participation ... is still just an evaluation of
participation!”*Alice Casey’s desk at the Involve offices, London
* Less catchy, but
also true
![Page 4: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/4.jpg)
Back to basics...why evaluate?
• Clarifying your objectives and purpose
•Ensuring goals are achievable and measurable
• Improving ongoing management
• Improved public accountability and value data
• Improving future work – learning!
![Page 5: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/5.jpg)
A choice of purpose• Evidence generatorCollecting the evidence of success
• Learning toolPlanning our future work more effectively
Very important for emerging online methods
![Page 6: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/6.jpg)
Its never too soon...
There’s a role for the evaluator at all stages;
• Scoping• Planning• Delivery• Reporting back
... so, begin with the end in mind.
![Page 7: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/7.jpg)
External or internal evaluation?
Internal • Develops in-house learning• Self-assessment• Less defensive reaction• Reduces risks of
innovation• Lower Costs
External• An outside perspective• Perceived independence• Credibility• Therapeutic• Lets others focus on doing
![Page 8: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/8.jpg)
So, if ;
"Not everything that can be counted counts, and not everything that counts can be counted."
…then how on earth do we know what to count and how to count it?
Evaluation of online participation
![Page 9: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/9.jpg)
We’re still learning...
but here are some ideas...
![Page 10: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/10.jpg)
How do we measure?
Use online and offline methods•Online surveys, timing is important•Pop-ups vs. email•Consider incentivising•Ongoing panel of participants•Telephone.... Skype?•Targeted publications•Peer interviews (include ‘people missed’)•Use focus groups (include ‘people missed’)
![Page 11: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/11.jpg)
Involve your stakeholders...
Target participant groups provide insight
• Ensure they understand why you are evaluating
• Consult them about methodology• Consult them about indicators • Include them on the evaluation team• Invite them to be peer researchers
![Page 12: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/12.jpg)
A step further... Participatory Evaluation
• What role should members of the public and/or service users play in evaluating projects?
• What are the benefits and downsides to having them involved?
• If you’re interested, read more here: Putting People Into Public Services; NationalConsumer Council (2008)
![Page 13: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/13.jpg)
What do we measure?
![Page 14: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/14.jpg)
Some useful definitionsTerm Definition
Outputs Tangible products (reports, meetings etc.)
Outcomes Overall direct results (increased skills, reduced crime etc.)
Impact Longer-term results (may include unintended effects)
Quantitative Objective, measurable, predetermined response options
Qualitative Subjective, interpretative, open ended response options
![Page 15: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/15.jpg)
Web stats:(Tip: Check out GoogleAnalytics)
1.Unique visitors to the site
2. Length of time spent on a page
3. Downloads of certain resources or materials
‘Thin’ engagement
Deeper engagement
![Page 16: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/16.jpg)
Interaction quality: eg. online forums, blogs, wikis.
1.Number of comments2.Number of user-instigated threads or topics3.Length of comments4.Number and length of chains of comments ‘discussions’5. Variety of participants, including representation of decision makers
![Page 17: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/17.jpg)
Motivations and perceptions:Understanding the experience
•Did participants feel they had influence?
•Did participants feel that they understood what they were being asked to contribute to and why?
• How did the moderator’s role seem to affect the engagement exercise?
![Page 18: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/18.jpg)
Outcomes and impact:What actually changed?
•Did participants have any demonstrable influence on the process?
•Did policy or decision making change as a result in the medium - long term?
• How did the working practices of decision makers change if at all, how did the participants attitudes or views change?
![Page 19: Online evaluation for Local Government Information Unit](https://reader033.vdocument.in/reader033/viewer/2022051816/54518f4daf7959b5648b4b9e/html5/thumbnails/19.jpg)
www.involve.org.uk22-25 Finsbury Square, EC2A 1DX
tel: +44 (0) 207 9206470 email: [email protected] twitter: @cased
Creative commons thanks for photos go to: plindberg, memotions