medford (ma) green awards evaluation plan
DESCRIPTION
This is an evaluation plan for the city of Medford's Green Awards program. The program celebrates and encourages those residents, businesses, and institutions that go above and beyond to advance environmental sustainability in their community. Three Northeastern University graduate students--Zak Patten, Nathan Peyton, and Po-Yu Yuen--formulated this plan in the fall of 2012.TRANSCRIPT
Medford Green Awards Evaluation Plan
Evaluation Team: Zak Patten, Nathan Peyton, Po-Yu Yuen Instructor: Professor Laurie Dopkins Course: Techniques for Program Evaluation Date: December 2, 2012
Prepared by the Valuate Consulting Team
for the Medford Green Awards Sub-Committee
Medford Green Awards Evaluation Plan
i | Patten, Peyton, and Yuen
Acknowledgements
The Valuate Team wishes to express our gratitude to the Medford Energy & Environment Office (E&E), the Energy Committee, and the Chamber of Commerce for their valuable time and contributions to this project. We also wish to thank Professor Laurie Dopkins for her thoughtful review and input to the development of this evaluation plan. Finally, our appreciation and thanks go to our classmates Armin Akhavan and Christina McPike for their contributions to the initial development of this project.
The Valuate Consulting Team Zak Patten Nathan Peyton Po-Yu Yuen
Medford Green Awards Evaluation Plan
ii | Patten, Peyton, and Yuen
Table of Contents
ACKNOWLEDGEMENTS .......................................................................................................................... i
INTRODUCTION ..................................................................................................................................... 1
PROGRAM OVERVIEW .......................................................................................................................... 2
EVALUATION FOCUS ............................................................................................................................. 7
EVALUATION DESIGN ......................................................................................................................... 10
FINDINGS DISSEMINATION ................................................................................................................. 16
RECOMMENDATIONS .......................................................................................................................... 17
RESOURCES ....................................................................................................................................... 20
APPENDIX ............................................................................................................................................. ii
APPENDIX A: APPLICATION .............................................................................................................. A-1
APPENDIX B: EVALUATION CHECKLIST ................................................................................................ B-1
Medford Green Awards Evaluation Plan
1 | Patten, Peyton, and Yuen
Introduction The city of Medford, Massachusetts, launched its Green Awards program in 2012 as a means of publicly recognizing those businesses and individuals actively working to protect the environment. By celebrating its environmental leaders, the city hoped to encourage more residents and companies to invest their resources in creating a more sustainable community.
We—the Northeastern University graduate student consulting team of Zachary Patten, Nathan Peyton, and Po-Yu Yuen—chose Medford's Green Awards program as our evaluand with the goal of conducting a formative evaluation that exemplified our commitment to environmental sustainability. It was crucial for us to find the right balance between the criterion that the evaluand be of an appropriate scale, sufficiently established, and well functioning and of an appropriate scope for us to achieve a meaningful impact given our limited timeframe and prior evaluation experience. Above all, we desire this evaluation plan—and it recommendations—to be useful to our evaluand.
When Medford implements our evaluation plan, specific activities will be studied closely; including the application, the recipient selection process, the award design, and the marketing of the program and the awards ceremony. Evaluating the activities that follow the annual awards ceremony such as winner follow-up and leadership development will be necessary if the program is to flourish in the years to come. Though these are discussed in greater detail below, the three overarching questions that guide the evaluation of Medford's Green Awards program should be mentioned here as well. They are as follows:
Evaluation Questions
1. Will the development and implementation of the Medford Green Awards be efficient and effective, given the city's internal capacity and resources?
2. Will the Medford Green Awards provide recognition of the green activities in the community and in turn educate Medford residents and businesses about sustainable and clean energy opportunities?
3. Is the Medford Green Awards designed to be self-sustaining beyond the city's intervention by building community leadership that supports the promotion of sustainable and clean energy activities among Medford residents and businesses?
Medford Green Awards Evaluation Plan
2 | Patten, Peyton, and Yuen
Program Overview
BACKGROUND In order to strengthen the city’s outreach efforts in promoting sustainability and clean energy to the community, E&E and the Energy Committee created “Go Green Medford,” a comprehensive outreach campaign that recognizes all sustainable programs and clean energy projects within Medford under a single brand. E&E and the Energy Committee have invested substantial resources toward promoting sustainable practices in the community under the Go Green Medford campaign. For example, in 2011, the Energy Committee partnered with Friends of the Medford Public Library to initiate the Kill-A-Watt meter loaning program, which helps residents assess the efficiency of their electric appliances. In 2012, the city received a two-year technical assistance grant from the Metropolitan Area Planning Council (MAPC) to plan for and implement clean energy projects in the municipal, residential, and commercial sectors.
MEDFORD GREEN AWARDS In the spring of 2012, E&E and the Energy Committee launched Medford’s first annual Green Awards program under the Go Green Medford campaign. The Green Awards program is intended to recognize businesses, organizations, and homeowners that are actively implementing energy-efficiency (EE) and conservation measures, as well as environmentally friendly practices that promote sustainability. The “green” measures include installing EE technology, educating the public about EE, producing renewable energy, recycling and composting, and sustainable landscaping. By recognizing these efforts, the Green Awards program highlights the benefits of energy efficiency, and makes leaders out of Medford residents and business owners. It provides the public with a model for energy efficiency, increases transparency, and rewards those who lead by example.
Program Purposes
The Medford Green Awards program is designed to fulfill three key purposes:
To recognize and celebrate local green practices;
To build community leadership and facilitate peer-to-peer education on sustainability and clean energy measures; and
To raise the profile of green energy conservation options & green programs.
Medford Green Awards Evaluation Plan
3 | Patten, Peyton, and Yuen
Logic Model
The following logic model summarizes the connection between the program inputs and activities and their consequential outputs, outcomes, and impacts.
Medford Green Awards Evaluation Plan
4 | Patten, Peyton, and Yuen
Key Stakeholders
The Green Awards program is facilitated by E&E and is co-sponsored by the Medford Energy Committee and Chamber of Commerce. The following section is a summary of the key stakeholders and their specific roles in the program.
Energy and Environment Office
The Energy and Environment Office (E&E) is a municipal department created by the city of Medford to oversee and implement environmentally beneficial and sustainable actions within the community. The Office is comprised of a full-time director of Energy and Environment and a part-time Energy Efficiency Coordinator. Since its establishment in 2003, the Office has been dedicated to leading green practices; facilitating knowledge transfer; and promoting community outreach, specifically in the areas of energy efficiency, air quality, climate change protection, wetlands protection, open space conservation, and stormwater management. To further these goals, E&E has worked collaboratively with local advisory groups; community organizations; universities; and local, regional, and state agencies. E&E is the main contact of the Medford Green Awards and is responsible for administering the program.
The Energy Committee
In 2010, the city revitalized the Medford Energy Committee, a civic advisory group with the objective of identifying new energy goals and actions for the community and its residents. The Energy Committee currently consists of 12 Medford residents. These mayoral appointees provide a wide spectrum of expertise in promoting local clean energy development and strengthening community outreach and education efforts. The Energy Committee works in conjunction with E&E to encourage clean-energy initiatives, improve energy efficiency, increase renewable energy use, and promote sustainability actions in Medford. The committee plays a key role in designing
Medford Harvest Your Energy Festival 2012 (From left: Medford Energy Committee Chair Fred Laskey, Director of Energy and Environment Carey Duques, and Mayor McGylynn)
Medford Green Awards Evaluation Plan
5 | Patten, Peyton, and Yuen
the Green Awards programming and is co-sponsor of the Green Awards with the Medford’s Chamber of Commerce.
The Chamber of Commerce
The Medford Chamber of Commerce was founded in 1926 to promote the interests of the city’s businesses. As such, the chamber has three primary goals: 1) involve the community, 2) advocate for businesses, and 3) provide opportunities. As mentioned above, the chamber also promotes the Green Awards and provides financial support for the program.
The Medford Green Awards Subcommittee
Director of Energy and Environment Carey Duques and four Energy Committee members make up the Review Committee of the Medford Green Awards. The Review Committee is responsible for further program design, informing the community of the program, reviewing the applications to identify businesses and homeowners with outstanding performance in promoting sustainability, and selecting the award recipients. At the end of each fiscal year, after the announcement of the award recipients, the subcommittee will hold an annual debriefing meeting to evaluate the implementation and performance of the program.
Medford Green Award Plaques and Window Decals for Award Recipients
Medford Green Awards Evaluation Plan
6 | Patten, Peyton, and Yuen
How Are Award Recipients Selected?
Recognition granted under the Green Awards program is expressly open to all Medford residents and Medford-based organizations including commercial, nonprofit, and academic, cultural, and healthcare institutions. Any resident or business can self-nominate or nominate another local constituent to be considered for an award. The nomination process begins by completing a detailed nomination form. These forms are openly available online to anyone willing to complete an application—business owners, clients, and residents alike. While paper forms have been made available, an electronic submission is “strongly encouraged.”
To be considered for an award, the nominator must be able to describe at least one green initiative, though it is strongly suggested that the nomination form contain more than one. The described initiative may be of an energy-conservation or environmental nature though both are preferred. The nomination form also contains a listing of sample energy and environmental initiatives favored by the selection body. These include: 1) on-site clean-energy production; 2) energy-efficiency measures and education; 3) comprehensive waste-management solutions that include elements of recycling, composting, reuse, and waste reduction; and 4) sustainable landscaping that captures stormwater.
The process and criteria for selecting Green Award recipients from among the pool of applicants is entirely left to the discretion of the subcommittee. In this first year of selection, businesses and residents were held to divergent standards based in part on size and ability. Smaller tenant businesses were held to less robust standards than larger institutions such as Tufts University and Whole Foods. The subcommittee also treated renter-residents differently than landlords and homeowners. The subcommittee acknowledges that different residents have different capabilities in implementing sustainability and clean-energy projects. Instead of a more stringently defined framework, the subcommittee uses the terms “Excellence,” “more than just recycling,” and “taking many actions” to describe what they are looking for in the awardee selection process. The subcommittee credits this flexibility in the review process for allowing them to recognize the contribution of a wider array of residents. As such, the subcommittee is committed to maintaining this discretion and believes it will continue to serve them well in the future.
Once selected, the award recipients are announced at the annual Harvest Your Energy Festival in October and detailed in a related press release for local media.
Medford Green Awards Evaluation Plan
7 | Patten, Peyton, and Yuen
Evaluation Focus
The Green Awards is a living
program that is intended to
evolve as the city of Medford
moves forward with its
sustainable and clean-energy
efforts. In the short run, it is the
city's priority to evaluate the pilot
implementation of the Green
Awards and consolidate a
standard program model that is
replicable and accountable in the
coming years. In the long run, in
order to achieve the program
objective, E&E and the Energy
Committee are interested in
developing a reliable evaluation
process that monitors performance. The program’s managers insist that any evaluation process
assists in the identification of new opportunities to both promote their interests and capitalize on
the sustainable and clean-energy potential of local residents and businesses.
Our team will provide assistance to the City of Medford to develop an evaluation plan that
comprehensively monitors current performance and at the same time, proceeds forward to
identify new opportunities. We will create an easy-to-follow guide that will be used annually by
E&E and the Energy Committee to assess the program’s implementation process, monitor current
performance, and identify future opportunities. The evaluation will be divided into three primary
recurrent themes: “Process,” “Performance,” and “Opportunities.”
Process
The Process section evaluates the implementation process of the program. The purpose of this
section is to improve the efficiency and effectiveness of the implementation of the Medford
Green Awards in order to maximize the output and outcome, given available resources and
capacity. The evaluation will identify the merits and challenges implementers experience in
executing the program, specifically at four key stages: (1) application design; (2) applicant
validation; (3) applicant review; and (4) data collection and monitoring.
Medford Harvest Your Energy Festival 2012
Medford Green Awards Evaluation Plan
8 | Patten, Peyton, and Yuen
Performance
The Performance section will focus on the immediate outcomes of the program. This section aims
to help the city (1) promote program participation; (2) adjust and consolidate the program model
to best accentuate the Green Awards’ goals and promote local, sustainable, and clean-energy
efforts under the Go Green Medford brand; and (3) increase local awareness on sustainability and
clean energy. This performance evaluation will utilize both quantitative and qualitative data to
assess whether the implementation of the Green Awards meets the short-term program goals.
Opportunities
The Opportunities section helps the city develop the Medford Green Awards into a self-sustaining
program. Specifically, this section will provide guidance to help the city build community
leadership that retains and expands Medford’s sustainability and clean energy efforts beyond the
intervention of E&E and the Energy Committee. The team will design evaluation questions that
encourage E&E and the Energy Committee to examine critically current program requirements
and think innovatively about new award opportunities that can provide a platform for peer-to-peer
education among local residents and businesses.
EVALUATION QUESTIONS To evaluate the efficiency of the Green Awards program, monitor program performance and success, and identify new program opportunities, this evaluation is designed to center on answering the following key questions.
1. Process – Will the development and implementation of the Medford Green Awards be efficient and effective, given the city's internal capacity and resources? Are there areas where efficiencies can be identified?
a. Are the current application instruments supportive of the applicant validation and review process?
b. How does the current application support long-term energy data collection? How does the data inform the city’s decisions on the establishment of clean energy and sustainability goals for the community, and subsequent monitoring of progress toward those goals?
c. How does the award program fit into the greater Go Green Medford brand?
2. Performance – Will the Medford Green Awards provide recognition of green activities in the community and in turn educate Medford’s residents and businesses about sustainable and clean energy opportunities?
Medford Green Awards Evaluation Plan
9 | Patten, Peyton, and Yuen
a. Do the accessibility of the current application and timing of the process encourage participation?
b. Are the current marketing strategies effectively producing applications, increasing program awareness, and capitalizing on all available marketing tools?
c. Are the current award categories and application descriptions sufficient to educate community members about all valued measures that promote the city’s clean-energy and sustainability goals?
3. Opportunities – Are the Medford Green Awards designed to be self-sustaining by building community leadership and peer-to-peer education that promote sustainability and clean-energy activities among Medford residents and businesses?
a. How are winners encouraged, supported, and developed into community leaders in clean energy and sustainability (and non-selectees into applicants)?
b. Do the current prizes (decals and plaques) and tiers of awards encourage re-applications and additional clean-energy and sustainability activities?
c. Do the current activities and follow-up events provide a platform for peer-to-peer knowledge transfers among applicants, local stakeholders, residents and businesses?
Medford Green Awards Evaluation Plan
10 | Patten, Peyton, and Yuen
Evaluation Design
This evaluation is designed around the expertise and capacity of the evaluators to maximize the usability of this tool. Our team’s goal for this project is to create an evaluation plan that provides the necessary guidance and stimulates discussion on how to improve the program each year. It is therefore our priority to design an evaluation that is meaningful and practical to the subcommittee. Our conversation with the Director of E&E revealed that time and staff capacity are the most critical concerns regarding the evaluation process of the program since it is anticipated that the committee will only meet once at the end of the calendar year to evaluate the program. It is thus critical that the evaluation be designed to be easy and quick to implement.
To answer the key questions of this evaluation, a Use Branch design that incorporates both qualitative and quantitative data analysis will be used. This evaluation is a non-experimental formative evaluation designed to assess the early-stage implementation of the Medford Green Awards. No control groups or random assignments are to be used in this evaluation. A mixed-data collection method that utilizes a range of quantitative and qualitative data-collection instruments are intended for collecting information on the implementation process and the intermediate performance indicators of the Green Awards. The following sections will discuss the data-collection strategies and sampling methods in further detail.
DATA-COLLECTION STRATEGIES
Quantitative Data-Collection
While we would like to collect as much quantitative data related to the Green Awards Program and the Medford community’s use of green technologies, we recognize the critical time and financial limitations under which this program is managed. There are several opportunities for
What is a Use Branch Evaluation?
A Use Branch evaluation focuses on the specific questions and the purposes of research. A Use Branch evaluation centers on the desired outcomes of the evaluation and recognizes the influence of the evaluator’s values. Mixed methods of data collection that integrate both quantitative and qualitative data are often used in this branch to support scientific research methods with commonsense and practical thinking.
Medford Green Awards Evaluation Plan
11 | Patten, Peyton, and Yuen
the capture of additional quantitative data both within and outside of the Green Awards program that would require little additional effort on the part of the program manager and would consume no additional financial resources.
Within the Program:
1. The number and types of applications received. Currently, the Green Awards Program Nomination form collects only a limited amount of data from businesses and residents. We suggest expanding the digital application to include additional metrics related to the nominated party. Specifically, the addition of a check-box on the renter or owner status of the business. An additional box allowing the nominator to indicate how she heard about the Green Awards program would allow the project manager to collect indicators of where she is having the best success in promoting the program.
2. The number and type of awards granted. Compiling this data and tracking changes in the type of project receiving awards would allow the program manager to better access the impact of the program as well as the impact of the standards put in place each year by the subcommittee members.
Outside the program:
1. The number and date of home/business energy evaluations done prior to the inception of the program and the energy retrofits suggested in each.
2. The number, date, and type of energy retrofits performed within Medford prior to the inception of the Green Awards program.
3. The number, date, and type of other green projects prior to the inception of the Green Awards.
4. Annual increases and decreases in all of these figures.
The collection and tracking of this dated information from outside the program is modeled on a pretest-posttest experiment. This is designed to give the project manager an understanding of progress toward the program's goals while also allowing the project manager to tailor the program toward effecting specific outcomes. Again, we understand the limited nature of the resources this program is managed under. In order to limit the time and staffing burden the collection of this information would place on the program manager or her team, we suggest the hiring of one or more of the area's many graduate students as unpaid interns.
Qualitative Data Collection
On the qualitative side, we believe three data-collection methods would be of value to the Green Awards and will advise the program manager to engage in all of them.
Medford Green Awards Evaluation Plan
12 | Patten, Peyton, and Yuen
Subcommittee Meeting
The first is the meeting of the subcommittee itself, which was, during the first year of the program, used to discuss and evaluate the applicants and ultimately, to decide which would receive awards. We envision future subcommittee meetings retaining this evaluative function while also serving as an opportunity to engage in a meta-evaluation of the program itself. The sub-committee can, once the awards have been agreed upon, reflect on the applicants as a whole, consider how they have qualitatively changed, whether there is evidence that the program's goals are being realized, and if not, what the obstacles may be to achieving those outcomes.
Award Winner Focus Group
The second of the data-collection methods we suggest is to invite some or all past Green Awards winners to attend the subcommittee meeting. The process can be treated as a focus group that is designed for gaining feedback from program applicants about the process, the value the winners place on being recognized in the ways they have been, and any recommendations winners might have for helping the program to succeed.
Interview
The third qualitative technique we recommend is the use of one-on-one interviews with past winners, which should be an opportunity for the program manager or another E&E representative to gain perspectives that may be lost using the previous two methods. Interviewing winners allows the program to learn what may not be obvious to the manager or sub-committee member. And interviewing individually would allow for more discreet communication, which may encourage certain winners to be more candid than they might otherwise be in a focus group setting such as the first method mentioned above.
We hope that by using a variety of qualitative methods, Green Awards stakeholders will be able to learn more about their program than they would by relying merely on the information provided via application. The hope is that real insight can be gained, thus helping the program sustain itself and continue to improve year by year.
SAMPLE PLAN In the absence of sufficient staff capacities for comprehensive community-engagement efforts, this evaluation will center on collecting data from award applicants, winners, and the subcommittee members. Given the limited staff capacity in Medford and the challenges to implement a communitywide data-collection strategy, the sampling method in this evaluation plan aims to maximize the data inputs while minimizing the additional workloads on the program staff. To do so, this evaluation will not adopt any control groups or unit selection
Medford Green Awards Evaluation Plan
13 | Patten, Peyton, and Yuen
strategies at the current stage of program implementation. In the absence of an effective way to engage all community members, this evaluation will focus on a self-sampling method for unit selections.
The subcommittee will collect information from local residents and businesses at two points during the program implementation as described in the “Data-Collection Strategies” section. First, quantitative data will be collected from residents and businesses at the point of application. Second, previous award winners will be invited to participate in the subcommittee meetings and the individual interviews. The volunteer winners will provide qualitative information on their experience with the program over the whole year. In both cases, residents and businesses are self-selected to provide the data to the city. The evaluation will also be collecting information from the subcommittee members at the annual meeting. It is expected that all members will attend the meeting, and therefore no unit-sampling strategy is needed.
Sampling Methods and Evaluation Validity
Sampling strategies are crucial to obtaining valid and dependable evaluation results. The absence of a control group or the lack of a thoughtful unit-sampling method may expose the evaluation to the following internal validity threats:
Confounding
Observations of the program performance may, instead of being attributed to the evaluated intervention, be attributed to a third or a combination of external variables, such as strengthened MassSave outreach efforts by the utilities and increased community education by local volunteer organizations, such as Green Medford.
History
Events independent to or beyond the program, such as the Go Green Medford campaign, and Medford’s participation in the Local Energy Action Program, may be responsible for measured changes in Medford’s performance in sustainability and clean energy.
Maturation
The subject (residents and businesses) of the program change over the course of the program. As the city’s sustainability and clean-energy efforts continue over years, residents and local businesses may naturally become more aware of green measures, and hence lead to an increase in sustainability and clean-energy practices, beyond the intervention of the Green Awards.
Medford Green Awards Evaluation Plan
14 | Patten, Peyton, and Yuen
Our team understands that the described strategies are not ideal for collecting accurate and unbiased data. The absence of a comparison group could expose this evaluation to validity threats that question the correlation between the program interventions and its performance. However, with the limitations of staff capacity, available resources, and time requirements, we believe that a simple and easy-to-execute evaluation design will be most tailored to the needs of E&E and the Energy Committee. The goal of this evaluation is to provide insight into the Medford Green Awards. At an early stage, an evaluation process is critical to program development insofar as it facilitates the discussion on how to shape the program moving forward, based on short-term observations. It is therefore our team’s priority to ensure that an evaluation process occurs each year. Based on this rationale, we believe that at the pilot stage, it is a reasonable decision to balance the depth and accuracy of the data with the applicability and sustainability of the evaluation process.
FUTURE EVALUATION DESIGN As the program matures, however, it is recommended that the committee should look into other data-collection mechanisms that can more accurately measure the program’s performance in order to sustain and, potentially, expand the Green Awards program, as well as to attract future funding sources. Data and data-collection and unit-sampling strategies the Green Award Sub-committee should consider pursuing in the future include:
Aggregated Utility Data
The city of Medford is currently working with National Grid to establish an agreed-upon system of collecting aggregated energy-consumption data and MassSave utility energy-efficiency program participation data in the residential and commercial sectors by zip code on a regular basis. Once the city is able to obtain residential and commercial energy data, the city can establish baseline residential and commercial energy consumption, MassSave participation, and monitor the community’s energy reduction progress. The Green Awards subcommittee can use annual energy consumption based on MassSave participation as an indicator to measure the program’s performance in advancing Medford’s residential and commercial energy efficiency efforts.
Comparison Groups
In the absence of a possible control group, the city should consider assessing the performance of the Green Awards program among comparison groups that share similar demographic attributes and sustainability and clean-energy potential but have received different treatments. Medford can utilize comparison groups for evaluating the program's performance in two ways.
Medford Green Awards Evaluation Plan
15 | Patten, Peyton, and Yuen
Comparison Groups by Zip Codes
Once the city receives aggregated utility data by zip code on a regular basis, the subcommittee can assess the role of an award recipient in promoting community sustainability and clean-energy efforts by comparing the outcomes of the different levels of treatment in different neighborhoods. The subcommittee can evaluate the correlation between the number of award recipients in a neighborhood and (1) energy consumption reduction and (2) MassSave participation rate. A positive correlation indicates that the Green Awards has the potential to build community leadership that raises awareness and encourages the implementation of energy efficiency measures.
Comparison Groups by Communities
The city can also compare Medford with neighboring communities that are similar in sizes and demographic characteristics to evaluate the effectiveness of the Medford Green Awards Program. The city can identify adjacent communities, such as Malden and Everett, that share similar demographic characteristics and sustainability and clean-energy opportunities but may have different levels of sustainability and clean-energy outreach and education within their communities. The subcommittee can compare and contrast the performance of each community and assess the performance of different green initiative strategies and models. Since the city does not have access to other communities’ energy data, the following indicators are data that can be easily collected through reviewing secondary documents, such as municipal websites and news articles, and can be used for measuring each community’s performance:
Number of community organizations associated with sustainability and clean energy
Number of community-based sustainability and clean-energy programs
Number of sustainability- and clean-energy-themed events per year
Number of awards received for community-sustainability and clean-energy efforts This exercise will not only evaluate the effectiveness of the Medford Green Awards, but will also provide insight into Medford’s performance in community green efforts with respect to the region.
Medford Green Awards Evaluation Plan
16 | Patten, Peyton, and Yuen
Findings Dissemination To improve and celebrate program success, the findings and lessons learned from this evaluation will be disseminated by the Green Awards committee through two platforms.
ANNUAL SUMMARY MEMO After the annual subcommittee meeting, the Green Awards Subcommittee will prepare a summary memo that documents all evaluation findings and recommendations. The summary memo will be submitted to E&E and the Energy Committee for preparing summary reports, broadcast and news media publications, and budgeting purposes. The memo should include data, facts, and quotes that support the assessment of the program. The section should discuss any challenges and merits the Green Awards Subcommittee faces when implementing the program. The findings should also identify any observations and changes in data that demonstrate the performance of the program. In addition to the findings, the memo should present a list of realistic implementation actions E&E and the Energy Committee should consider in order to (1) improve the execution of the program, (2) advance the design of the program to better meet the program objectives and goals, and (3) identify new opportunities to expand the program.
PUBLIC PRESS RELEASE In order to increase the recognition of local sustainability and clean-energy efforts, inform local residents and businesses of the program, and motivate participation, the Green Award Subcommittee should publish a public press release to acknowledge the green practices local residents and businesses have taken and celebrate Medford’s sustainability and clean-energy successes. The public press release should focus on the findings of the performance and opportunities from the evaluation of the program. The findings should include successful strategies that have resulted in positive performance as well as improved outcomes. In order to persistently advance the city’s sustainability and clean-energy initiatives, the press release should also discuss areas of improvement and goals that have not been accomplished to raise awareness of additional opportunities and alternative measures local residents and businesses should consider.
Medford Green Awards Evaluation Plan
17 | Patten, Peyton, and Yuen
Recommendations As the Medford Green Awards program matures, E&E and the Energy Subcommittee should seek out ways to build stronger connections between the program's educational goals and the development of its award recipients into community leaders who are capable of sustaining the program with little additional support from E&E. Even given the limited resources available to the program, we have identified several opportunities that are currently unexplored or underutilized.
Green Once, Green Thrice…
The Medford Green Award is, at present, a single award granted for any action deemed deserving by the energy subcommittee. In order to both better educate future would-be applicants about action options and encourage past winners to do more, the program's manager should consider creating more defined categories and separate the physical prizes to represent these categories (i.e. Water Conservation Award, Solar Power Award, Wind Power Award, Recycling Award, etc.). In doing so, past winners may be inspired to earn additional recognition by adding sustainability features to current projects or starting new ones.
Winners To Leaders
Recipients of the Medford Green Awards are currently being recognized for an “action” in support of the Go Green Medford brand. Going forward, the program should focus on recognizing, encouraging, and developing the commitment of these award recipients. These winners are proud of what they have accomplished and have displayed their personal commitment to a green lifestyle. What else are these award winners eager to do in their community? We have identified three low-cost options that will allow past winners to continue their involvement in the Go Green Medford campaign and share their experiences and lessons learned:
1) Online Forum – Tech-savvy winners or those with less available time may wish to participate in an online forum. The forum would allow past winners and new winners to remain connected. It would also serve as a resource for those interested in starting green projects of their own.
2) Neighborhood Tour Leaders – This option comes with the added benefit of building a sense of community and neighborhood pride while cultivating a locally based leadership team. Tour leaders would have the opportunity to educate others and
Medford Green Awards Evaluation Plan
18 | Patten, Peyton, and Yuen
present their experiences face-to-face in a comfortable setting. This option, more than any other, lends itself to friendly competition between neighbors and neighborhoods.
3) Events and Venues – Community-based organizations with regular meetings represent opportunities for Green Awards winners to personally introduce the Medford Green Awards Program to already active community members and share experiences while recruiting future participants. A quick search of likely venues includes: The West Medford Community Center, Monthly Community Policing Meeting, Medford’s Annual Housing Meeting, Medford Public Libraries Events, The Zonta Club, and Medford Public Schools’ parent-oriented events.
Non-Selectees to Winners
Applications that are not selected to receive an award for their green project should be informed of what actions they could take to improve their application in order to receive an award in the next year. In order to conserve staff time, this determination should be built into the review and awards-notification processes. Non-selectees who are interested in improving their application could be invited to participate in any one, or all, of the educational events hosted by award recipients. In any event, as the Medford Green Awards program becomes more selective and the awards criteria becomes more narrowly tailored, non-selectees should be encouraged to improve and to reapply: They may indeed become a significant source for future successful applications.
Program Evolution
In its inaugural year, in addition to setting up the program, the Green Awards program focused on identifying the right “green” projects and actions to be singled out. The program’s managers have defined success in terms of the number of applications received, the number of awards granted and the level of attention the program has garnered from the press. Going forward, as the program moves from being a successful annual event into a self-sustaining campaign, the metrics used to define success must also evolve in tandem.
Medford Green Awards Evaluation Plan
19 | Patten, Peyton, and Yuen
Selectee
Non-
Selectee
Applications
Events&Venues
Winnerspresent
projects&share
experience
HarvestFestival:EventandPressReleaseutilizedtopromotefollow-upevents
Selecteeinformed/InvitationtoHarvestFest
requestforcontinuingcommitment
NeighborhoodTourLeadersWinnersgive
toursofnearbyprojects,answer
questionsandshare
experience
OnlineForum
Winnersanswer
questions,share
experience,andmoderate
discussions
ReviewProcess
Non-Selectee
invitedtoeducational
events&encouragedtoimprove
1) Attendeesencouragedtoapply,re-apply,and/ornominateothers.
2) Informationonfreeenergyevaluations,taxcredits,financialsavingandotherformsofaid
distributed.3) Eacheventtypeshouldpromotetheother
Communicatespecific
measurestobetakenforselectionin
nextyear
Encouragedto
re-apply
Medford Green Awards Evaluation Plan
20 | Patten, Peyton, and Yuen
Resources This evaluation plan builds upon the following resources.
Energy & Environment Office. (n.d.). Retrieved October 14, 2012, from City of Medford: http://www.medford.org/Pages/MedfordMA_Energy/index
Green Awards. (n.d.). Retrieved October 14, 2012, from City of Medford: http://www.medford.org/Pages/MedfordMA_Energy/GreenAwards/index
Home. (n.d.). Retrieved October 14, 2012, from Medford Chamber of Commerce: http://www.medfordchamberma.com/
Medford. (2012). Green Business Award Nomination . Medford, MA, USA.
Medford Energy Committee. (n.d.). Retrieved October 14, 2012, from Medford Energy Committee: http://medfordenergy.org/
Mertens, D. M. (2012). Program Evaluation Theory and Practice: A Comprehensive Guide. New York: Guilford Press.
Medford Green Awards Evaluation Plan
ii | Patten, Peyton, and Yuen
Appendix
Medford Green Awards Evaluation Plan
A-1 | Patten, Peyton, and Yuen
Appendix A: Application
GREEN RESIDENTIAL AWARD NOMINATION
Name of Nominee: Address:
Type of Housing: Single family
Condo
Townhouse
Contact Person:
Property Ownership: Own
Rent
Phone Number:
Has the property received an energy assessment from a utility or a third party?
Yes
No
Would you be interested in sharing your energy assessment with the Energy Committee?
Yes
No
Would you be interested in sharing your experience in sustainability and clean energy with the community at our events?
Yes
No
The Medford Green Residential Awards are available to residents that demonstrate energy and environmental initiatives in areas such as energy efficiency, energy conservation, water conservation, preservation of resources, education to fellow residents and/or tenants on environmental and/or energy topics, wastewater management, promotion of alternative forms of transportation and public transit, on-site renewable energy production, purchase of renewable energy, and sustainable food production.
Please note this award is open to all residents including renters and homeowners. You are encouraged to nominate yourself as well as other residents.
The nominator shall describe a minimum of one energy or environmental initiative implemented at the home. Ideal candidates will have multiple energy and environmental initiatives. We encourage electronic submittals of Green Award Applications.
Nominations must be received by (Time and Date).
Examples of Energy Initiatives we are interested in hearing about include, but are not limited to the following:
Renewable Energy production on-site, include a description of the system such as solar PV, wind, solar thermal, geothermal, etc.
Installation of energy efficiency measures, maintenance of the measures, and education and outreach
Green operations with a description of green products installed and how they are maintained, if and how the information is shared with others
Description of a comprehensive waste management system including recycling program, composting, reuse, reduction plan. - Sustainable landscaping and rainwater capture for landscaping, i.e., green roof, rain gardens, etc.
Medford Green Awards Evaluation Plan
B-3 | Patten, Peyton, and Yuen
Please describe activities that have been implemented at the property to conserve energy and protect
the environment (500 words or less recommended).
What type of sustainability and clean energy opportunities would you like to learn more about in the future?
Energy efficiency improvement projects
Financing and funding opportunities for energy efficiency retrofits
PV Solar panel installation
Financing and funding opportunities for solar projects
Energy efficiency improvement opportunities for fuel oil users
Waste management projects
Green operations
Energy efficient and alternative-fuel vehicles
Local food production
Medford Green Awards Evaluation Plan
A-3 | Patten, Peyton, and Yuen
GREEN BUSINESS AWARD NOMINATION
Business Name: Address:
Size of Business: Contact Person:
Number of Employees: Phone Number:
Property Ownership: Own
Rent
E-mail:
Has the business received an energy assessment from a utility or a third party? Yes
No
Would you be interested in sharing your energy assessment with the Energy Committee? Yes
No
Has the property monitored its energy consumption using ENERGY STAR® Portfolio Manager?
Yes
No
Would you be interested in authorizing the City of Medford to access your ENERGY STAR® Portfolio Manager facilities profile?
Yes
No
Would you be interested in sharing your experience in sustainability and clean energy with the community at our events?
Yes
No
The Medford Green Residential Awards are available to residents that demonstrate energy and environmental initiatives in areas such as energy efficiency, energy conservation, water conservation, preservation of resources, education to fellow residents and/or tenants on environmental and/or energy topics, wastewater management, promotion of alternative forms of transportation and public transit, on-site renewable energy production, purchase of renewable energy, and sustainable food production.
Please note this award is open to all residents including renters and homeowners. You are encouraged to nominate yourself as well as other residents.
The nominator shall describe a minimum of one energy or environmental initiative implemented at the home. Ideal candidates will have multiple energy and environmental initiatives. We encourage electronic submittals of Green Award Applications.
Nominations must be received by (Time and Date).
Examples of Energy Initiatives we are interested in hearing about include, but are not limited to the following:
Renewable Energy production on-site, include a description of the system such as solar PV, wind, solar thermal, geothermal, etc.
Installation of energy efficiency measures, maintenance of the measures, and education and outreach.
Green products or services with an explanation of how they are produced, marketed, sold to the public - Green operations, description of green products installed and how they are maintained, if and how employees are trained demonstrating green behaviors.
Description of a comprehensive waste management system including recycling program, composting, reuse, reduction plan.
Sustainable landscaping and rainwater capture for landscaping, i.e., green roof, rain gardens, etc.
Medford Green Awards Evaluation Plan
A-4 | Patten, Peyton, and Yuen
Please describe activities that have been implemented at the business to conserve energy and protect
the environment (500 words or less recommended).
Would you be interested in becoming a partner of the U.S. Department of Energy’s Better Buildings Challenge initiative? For more information about the initiative, please visit: http://www4.eere.energy.gov/challenge/
Yes
No
What type of sustainability and clean energy opportunities would you like to learn more about in the future?
Energy efficiency improvement projects
Financing and funding opportunities for energy efficiency retrofits
PV Solar panel installation
Financing and funding opportunities for solar projects
Energy efficiency improvement opportunities for fuel oil users
Waste management projects
Green operations
Energy efficient and alternative-fuel vehicles
Local food production
Medford Green Awards Evaluation Plan
B-1 | Patten, Peyton, and Yuen
Appendix B: Evaluation
Checklist The Green Awards Subcommittee should identify whether the following indicators have been fulfilled using the data collected within and outside of the program, as well as through discussions at the meetings and one-on-one interviews with Award recipients.
1. Process – Will the development and implementation of the Medford Green Awards
be efficient and effective, given the City's internal capacity and resources? Are there areas
where efficiencies can be identified?
Question Indicator
Are the current application instruments supportive
of the applicant validation and review process? It takes <5 hour of staff time to execute the
application.
It takes <5 hour of staff time to collect and organize the application data.
The current application and application tool are designed to reduce the time spent on
application validation and review process.
The current application review process is not time-consuming to the Subcommittee members.
How does the current application support long-term energy data collection? How does the data inform the city’s decisions on the establishment of community clean-energy and sustainability goals and the monitoring of progress toward those
goals?
The application is producing meaningful data for tracking sustainability and clean-energy
efforts in the community.
The application is producing meaningful data for understanding local interests on sustainability and clean energy in the
community.
How does the award program fit into the greater
Go Green Medford brand? The Green Awards program is being promoted
under the Go Green Medford campaign.
Medford Green Awards Evaluation Plan
B-3 | Patten, Peyton, and Yuen
2. Performance – Will the Medford Green Awards provide recognition of green activities in the
community and in turn educate Medford’s residents and businesses about sustainable and
clean-energy opportunities?
Question Indicator
Do the accessibility of the current application and
the timing of the process encourage participation? There is an increase in the number of
applications.
There is an increase in the diversity of applicants.
Applicants feel that there is sufficient instruction for completing the application.
Applicants feel that it is not time-consuming to complete the application.
Are the current marketing strategies effectively producing applications, increasing program awareness, and capitalizing on all available
marketing tools?
More than two forms of media tools are used
for promoting the Green Awards.
Applicants learn about the program through a diverse set of media and marketing tools.
The number of applications has increased.
Are the current award categories and application descriptions sufficient to educate community members about all valued measures that promote
the city’s clean-energy and sustainability goals?
Current award categories reflect the city’s sustainability and clean-energy goals.
There is an increase in the diversity of sustainability and clean-energy measures among the applications.
There is an increase in participation in MassSave in the residential sector.
There is an increase in participation in MassSave in the commercial sector.
Medford Green Awards Evaluation Plan
B-3 | Patten, Peyton, and Yuen
3. Opportunities – Are the Medford Green Awards designed to be self-sustaining by building
community leadership and peer-to-peer education that promotes sustainability and clean-
energy activities among Medford residents and businesses?
Question Indicator
How are winners encouraged, supported, and developed into community leaders in clean energy and sustainability (and non-selectees into
applicants)?
There is an increase in re-applications by previous award recipients.
Award recipients feel that the city has provided guidance to help them share their sustainability and clean-energy experience with local
residents and businesses.
Non-selectees feel that the city has provided sufficient support and education to help them
become stronger applicants.
Do the current prizes (decals and plaques) and tiers of awards encourage re-applications and additional clean-energy and sustainability activities?
There is an increase in re-applications by previous award recipients.
There is an increase in the diversity of sustainability and clean-energy measures
among the applications.
Re-applications show sufficient changes in sustainability and clean-energy measures than
previous applications.
Do the current activities and follow-up events provide a platform for peer-to-peer knowledge transfers among applicants, local stakeholders,
residents, and businesses?
Award winners are invited to participate and share their experience in local, sustainable and
clean-energy events.
Award winners feel that they are provided with the opportunities to showcase their successful
stories.
There is an increase in community events such as presentations and workshops, that help local residents and businesses access sustainability and clean-energy opportunities.
Local residents and businesses feel that they are more aware of sustainability and clean-
energy opportunities.