Lessons Learned from Lessons Learned from eXtension CoP Evaluation eXtension CoP Evaluation

EffortsEfforts

Michael Lambur

eXtension Evaluation and Research Leader

Overall Lessons LearnedOverall Lessons Learned

• Posting content to a website does not ensure outcomes/impact (i.e., If we build it they will come)

• CoP’s who have documented outcomes/impact to date have engaged with their clientele—internal and external

• It’s all about good program development using a new medium

CoP Evaluation EffortsCoP Evaluation Efforts

• Entrepreneurs and their Communities– Surveyed CoP members on five key resource

areas: case studies, information briefs, state and local policies, research references, and webinars

– Most planned to use the website for information for public presentations, articles, and other reports

– 86% of the respondents rated webinars as useful or very useful

CoP Evaluation EffortsCoP Evaluation Efforts

• Financial Security for All– Posted a survey link on all their FAQ’s– 27 people responded– 70.4% indicated that the FAQ provided information

that was useful or very useful– 37% indicated that the FAQ helped them to learn

quite a bit more or a lot more new information– 66.7% indicated that that they may put or definitely

plan to put the information to use

CoP Evaluation EffortsCoP Evaluation Efforts

• HorseQuest– Learning Modules

• Reported a 17% increase in knowledge scores pre to post

• 79% indicated a willingness to recommend Learning Modules to others

CoP Evaluation EffortsCoP Evaluation Efforts

• HorseQuest– Webcasts

• 17 webcasts offered attended by 600 people• 58% rated them as very useful• Reported a 15% increase in knowledge scores pre to

post• 76% planned to make at least one change in their

horse management practices• Archived webcasts are being used in local

programming

CoP Evaluation EffortsCoP Evaluation Efforts

• HorseQuest– YouTube

• 111 video clips have been posted to the HorseQuest YouTube channel

• Videos have been viewed 125,277 times• The channel has 310 individual subscribers• 68% of the videos have been rated, with 75%

receiving the top rating of 5 stars• There have been 239 self-referrals from the channel

to the eXtension HorseQuest website, with an average of 6.81 page views per visit

CoP Evaluation EffortsCoP Evaluation Efforts

• Imported Fire Ants– Conducted an eXtension adoption survey at the

Alabama Association of County Agriculture Agents and Specialists

– 74 individuals responded to the survey– 69% indicated that they were familiar with

eXtension; 58% indicated that they had used eXtension resources

– 21 indicated that the Imported Fire Ants resource area was very useful

CoP Evaluation EffortsCoP Evaluation Efforts

• Just in Time Parenting– Parenting Newsletter

• Parents subscribe and receive a monthly newsletter during the first year after their child’s birth

• After they receive their 12th newsletter, parents will be invited to complete an online survey to document how much they have learned, ways they have changed their behavior, and satisfaction with JITP

• Based on pilot study results, 81% (n=738) who opened messages clicked forward to the appropriate newsletter

CoP Evaluation EffortsCoP Evaluation Efforts

• Livestock and Poultry Environmental Learning Center– Newsletter

• Online survey completed by 147 newsletter subscribers working with livestock and poultry producers

• 64% rated webcasts, 52% rated subject matter pages, 46% rated the website, and 46% rated the newsletter as very useful

CoP Evaluation EffortsCoP Evaluation Efforts• Livestock and Poultry Environmental

Learning Center• 62% indicated that the website was better than others

in the quality of information provided• Knowledge gains were documented• 50% of livestock and poultry producers reported that

they had implemented one or more new practices to protect air or water quality

• 70% of county Extension agents used the website• 84% of Extension specialists used the website to

research an answer for a producer or agent

So Where Do We Go From So Where Do We Go From Here?Here?

• Using social media applications to engage clientele (e.g., Twitter, Facebook, YouTube, Blogs, etc.)

Strategy Blocks and Social Media Tools Strategy Blocks and Social Media Tools (Kanter, 2010)(Kanter, 2010)

Evaluating Social Media Evaluating Social Media ApplicationsApplications

• Engagement

• Forrester Research (Villa, 2009):– Involvement: site visitors, time spent, page views,

reach, frequency– Interaction: contributions to blogs, content

creation and uploads– Intimacy: consumer attitudes, perceptions, and

feelings– Influence: likelihood that consumers will

recommend or advocate

Evaluating Social Media Evaluating Social Media ApplicationsApplications

• Papadakis (Kaushik, 2008):– Kind: the degree to which customers are

positively or negatively engaged– Degree: the degree to which customers are

involved—low to high

Evaluating Social Media Evaluating Social Media ApplicationsApplications

• Involvement and interaction/degree can be measured with analytics (quantitative “head” data)

• Intimacy and influence/kind need to come directly from the customer (qualitative “heart” data)

Evaluating Social Media Evaluating Social Media ApplicationsApplications

• Involvement and interaction/degree:– GA: unique visits, frequency of visits, recency

of visit, depth of visit, length of visit, time spent on site

– Twitter: Following, followers, tweets, Tweetrush, Tweetlists, Tweetvolume

– Facebook: Fans, profile data, wall postings, events

– YouTube: Views, ratings, links, audiences

Evaluating Social Media Evaluating Social Media ApplicationsApplications

• Intimacy and influence/kind:– Comment analysis– Content analysis of the “conversation”– Evaluative questions asked in the

conversation– Links to online surveys

ReferencesReferences

• Kanter, Beth. (2010). 9 ways nonprofits can excel using social media. http://www.bethkanter.org/socialmedia-nonprofits-excel/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+bethblog+%28Beth%27s+Blog%29

• Kaushik, A. (2008). Measuring online engagement: What role does web analytics play? http://www.kaushik.net/avinash/2008/01

• Villa, J. (2009). What does engagement really mean? http://www.mediapost.com/publications/index.cfm?fa=Articles.showArticle&art_aid=111974&passFuseAction=PublicationsSearch.showSearchReslts&art_searched=jose%20villa&page_number=11