executive summary - ilsagfiles.orgilsagfiles.org/sag_files/evaluation_documents/draft... · web...

Evaluation of Illinois Energy Now Building Operator Certification Program

June 2015 through May 2016

Prepared for:Illinois Department of Commerce & Economic Opportunity

Prepared by:

ADM Associates, Inc.3239 Ramos Circle

Sacramento, CA 95827916.363.8383

Draft Report: March 2017

Contact:

Donald Dohrmann, Ph.D., [email protected]

Prepared by:

Brian [email protected]

James Gowen, [email protected]

mailto:[email protected]



Table of ContentsExecutive Summary...................................................................................................................ES-1

1.......................Introduction..........................................................................................................1-1

2.......................Savings Calculation Methodology........................................................................2-1

3.......................Estimation of Net Savings....................................................................................3-1

4.......................Process Evaluation................................................................................................4-1

5.......................Conclusions and Recommendations.....................................................................5-1

Appendix A: Questionnaire for Participant Survey.....................................................................A-1

Appendix B: Participant Survey Responses................................................................................B-1

Appendix C: Supervisor Survey Instrument................................................................................C-1

List of FiguresFigure 4-1 Process Evaluation Overview.....................................................................................4-2

Figure 4-2 Participant Reported Current Job Titles.....................................................................4-4

Figure 4-3 How Participants Learned about the BOC Tuition Rebate.........................................4-6

Figure 4-4 Information Sources Typically Used by Participants.................................................4-7

Figure 4-5 Participant Motivations to Enroll in BOC Course......................................................4-8

Figure 4-6 Participant Implementations Following BOC Training............................................4-10

Figure 4-7 Procedural Energy Efficiency Activities Completed by Participants.......................4-12

List of TablesTable ES-1 Sources Referenced for Savings Calculations........................................................ES-1

Table ES-2 Net Savings by Measure for Participant Sample....................................................ES-2

Table ES-3 Summary of Net kWh Savings for BOC Program.................................................ES-3

Table ES-4 Summary of Net kW Savings for BOC Program...................................................ES-3

Table ES-5 Summary of Net Savings from EPY8/GPY5 Projects............................................ES-3

Table 2-1 Sources Referenced for Savings Calculations.............................................................2-1

Table 3-1 Reported Projects by Measure Type and Influence Level...........................................3-4

Table 3-2 Distribution of Net-to-Gross Responses for Cited Projects.........................................3-5

Table 3-3 Net Savings by Measure for Participant Sample.........................................................3-6

Table 3-4 Distribution of Electric Utilities among BOC Participants..........................................3-7

Table 3-5 Summary of Net kWh Savings for BOC Program.......................................................3-7

Table 3-6 Summary of Net kW Savings for BOC Program.........................................................3-7

Table 3-7 Summary of Net Savings from EPY8/GPY5 Projects.................................................3-8

Table 4-1 Respondent Facility Types...........................................................................................4-4

Table 4-2 Existing Energy Efficiency Policies and Procedures...................................................4-5

Table 4-3 Planned Implementations of Energy Saving Projects................................................4-11

Table 4-4 Barriers to Applying BOC Knowledge......................................................................4-12

Table 4-5 Participant Satisfaction Ratings by Program Element...............................................4-13

Executive Summary

This report presents the results of the impact and process evaluations of the Building Operator Certification Program (BOC), which is administered by the Midwest Energy Efficiency Alliance (MEEA) under a license provided by the Northwest Energy Efficiency Council (NEEC), and which receives program support and tuition rebate funding from the Illinois Department of Commerce & Economic Opportunity (hereinafter referred to as the “Department of Commerce”). This report presents the results the evaluation of program activity occurring during the period June 2015 through May 2016, defined as electric program year eight and natural gas program year five (EPY8/GPY5).

The main features of the evaluation approach are as follows:

Data used to perform the savings evaluation were collected through review of program materials and surveys and follow-up conversations with BOC participants.

An approach based on review of the Illinois Statewide Technical Reference Manual (TRM), savings databases, and work papers was used to quantify energy savings associated with energy efficiency projects implemented by BOC participants as a result of program participation.

Program-attributable, net energy savings were distinguished from energy impacts that are not attributable to the program by using survey-based analysis methods applied to data collected through a survey of a sample of BOC participants.

For the process evaluation, information about program performance, changes to program design, perspectives on program benefits, and program satisfaction levels were obtained through interviews with MEEA staff, surveys with a sample of BOC participants, and surveys of supervisors of BOC participants.

The savings impact estimation process included a review of energy efficiency measure information obtained through the participant survey effort as well as follow-up interviews with the appropriate participant and facility management staff members. As shown in Table ES-1, the evaluators referred to the Illinois TRM in order to estimate savings for each eligible measure type.

Table ES-1 Sources Referenced for Savings Calculations

Measure Category Energy Savings SourcesEnergy Efficient Lighting Illinois Statewide TRMEnergy Efficient Motors Illinois Statewide TRM

Table ES-2 presents the net savings associated with sampled participants for each measure and that achieved net savings within the sampled participant group.

Executive Summary ES-1

Energy Efficiency Program: Building Operator Certification Final Evaluation Report

Table ES-2 Net Savings by Measure for Participant Sample

Measure CategoryTotal Sampled Net Savings

kWh kW ThermsEnergy Efficient Lighting 7,213 0.00 -Lighting Controls 717 0.35 -Total 7,931 0.35 0.00

The total savings shown above were extrapolated to the population of BOC participants who completed the MEEA BOC Program training during EPY8/GPY5. The evaluators were able to complete surveys of 13 out of 53 EPY8/GPY5 BOC participants. Based on participant survey responses, four of the 13 respondents reported completing energy saving projects that potentially generate net savings attributable to the BOC Program. Of the four sampled BOC participants who were associated with potential program-attributable net energy savings, evaluators were able to contact and verify energy savings associated with three BOC participants. The remaining participant could not be reached during the follow-up effort. This participant was removed from the sample and treated as part of the population of non-sampled BOC participants who were not reached during the participant survey effort.

Additionally, during follow-up discussions between participants and ADM engineering staff, the evaluators determined that the projects reported by one participant had not yet been implemented; therefore, net energy savings were calculated for a total of two participants. The energy savings value associated with these two participants was then divided by the total number of survey respondents (12 respondents, excluding the surveyed participant for which ADM obtained insufficient information regarding participant-reported projects) in order to determine the average savings per sampled participant. Once this energy savings value was determined, the energy savings were extrapolated to the program participant population.1

Energy savings were extrapolated in a manner accounting for the distribution of utility service providers within the participant population. Table ES-3 presents the net kWh savings by utility for the Building Operator Certification Program during EPY8/GPY5. It should be noted that because some participants were serviced by non-EEPS electric utilities such as municipal utilities, any electric energy savings generated by these participants are not attributed to the EEPS-funded BOC Program.

1 The sampled savings were extrapolated to a population of 53 total participants.



Table ES-3 Summary of Net kWh Savings for BOC Program

Electric Utility Realized Net kWh Savings

Ameren 15,797

ComEd 15,797

Total 31,593

Table ES-4 presents the program’s EPY8/GPY5 net kW savings by utility.

Table ES-4 Summary of Net kW Savings for BOC Program

Utility Realized Net kW Savings

Ameren 0.70

ComEd 0.70

Total 1.40

No natural gas energy savings were identified as evaluable and countable as net, program attributable savings during the participant survey effort or subsequent engineering follow-up calls.

The total net energy savings of the Building Operator Certification Program during EPY8/GPY5 are summarized in Table ES-5. During this period, net energy savings attributed to the program totaled 31,593 kWh and 1.40 kW. These values do not include savings generated through non-EEPS utilities, which totaled 3,434 kWh and 0.15 kW.

Table ES-5 Summary of Net Savings from EPY8/GPY5 Projects

Savings LevelTotal Net Savings

kWh kW Therms

Per Participant 661 0.03 -

Extrapolated to EPY8/GPY5 Participants 31,593 1.40 -

The following section presents a summary of key findings from the process and impact evaluations of the Building Operator Certification (BOC) Program. These conclusions and recommendations are based on a combination of research activities including participant surveys, interviews with program staff, and reviews of program tracking data, documentation, and prior evaluation reports.

The following is a summary of key conclusions from the evaluation of BOC Program EPY8/GPY5 activity:



Few of the measure identified in the survey generated program-attributable energy savings. As with prior program years, the savings estimation procedure determined that, although participants reported implementing a wide range of projects after participation in the BOC training, the total net energy savings impacts resulting from these projects were lower than may be expected based on the number of measures identified by participants. Primary contributors to the limited net energy savingsinclude:

• Of the 28 measures reported by BOC participants for EPY8/GPY5, approximately 32% met the program attribution criteria specified by the Illinois TRM. This suggests that participants had plans to implement many of the measures prior to attending the BOC training, or that, for many measures, the BOC training was not highly influential to the decisions to implement energy efficiency projects. This does not suggest that the BOC program had no effect on the 68% of projects that did not meet the net attribution criteria; information gained through BOC courses may have improved participants’ adherence to best practices or improved the overall planning and quality assurance process, and participants reported that they found the program to be very valuable. However, the evaluation results suggest that for a majority of reported projects, the associated energy savings would have been achieved in the absence of the training program.

• Of the 28 measures reported by BOC participants for EPY8/GPY5, 28% were associated with other incentive programs, according to survey responses. While the BOC training was likely influential for many of these projects, the incentivized energy savings are claimable by other EEPS-funded program administered by the Department of Commerce and investor-owned utilities and thus cannot also be attributed to the BOC Program. This rate of external incentive receipt is lower than the prior program year rate of 56% of reported projects, but remains a factor in limiting energy savings attributable to BOC.

Participant and supervisor satisfaction is consistently high. As was found during prior program years, BOC graduates indicated a high level of satisfaction with all elements of their program experience and did not indicate any systematic or major issues with program structure, management, or operation. Respondents provided extremely high satisfaction ratings for all listed elements of their BOC Program experience, and only one respondent reported being at all dissatisfied with any element (the length of time to receive the rebate). Overall, the participant survey results suggest that program delivery has been very effective, and that there are few potential areas for improvement from the student perspective.

Statewide budget delay limited program operations: The number of program participants associated with Department of Commerce tuition rebates decreased from 117 individuals in EPY7/GPY4 to 53 individuals in EPY8/GPY5. As discussed by program staff, this was primarily because of operational challenges faced by MEEA throughout EPY8/GPY5 due to a delay in the approval of the Illinois statebudget. As BOC provides tuition rebates that are funded by the Department of Commerce through the Illinois energy efficiency budget, this budget issue was associated with uncertainty among program staff and participants regarding



when, and to what extent, it would be possible to provide these incentives. MEEA continued to operate BOC training courses and informed participants that they would receive the tuition reimbursement as soon as funding became available. As budget delays continued, the BOC Program continued to operate throughout EPY8/GPY5 but held fewer training courses and limited program operations to core activities. The veteran component and certification maintenance support services provided by the program were not conducted during EPY8/GPY5 due to these budget-related resource issues. A temporary state budget was ultimately passed after the end of EPY8/GPY5.

MEEA continued active public relations and marketing efforts. Despite the uncertainty resulting from the statewide budgetary issues, MEEA continued to conduct marketing and recruitment activities during EPY8/GPY5. MEEA’s marketing strategy consists of a variety of outreach methods, including attending events and working through community colleges and other organizations to promote the program. According to MEEA, these outreach efforts help the public to understand that the program is still available, and that program operations will likely recover and increase now that a temporary budget has been passed. Although EPY8/GPY5 experienced a significant decrease in program participants, MEEA staff explained that they expect this to be a temporary issue and that interest in the program is still high.

Overall, the evaluators found that while there were significant operational challenges during EPY8/GPY5, the Building Operator Certification Program has continued to deliver a valuable service and is well suited to providing up-to-date and actionable information to building managers. Based on information gathered through the staff interview and participant and supervisor survey efforts, the evaluators provide the following recommendations for consideration moving forward:

Track and highlight potential natural gas projects: The EPY8/GPY5 evaluation did not identify any natural gas energy savings attributable to the BOC Program. While the sample sizes achieved for participant surveys may be a contributing factor, the EPY7/GPY4 evaluation also showed a lack of net attributable natural gas energy savings, and a high majority of reported projects are associated with electricity usage only. Although the end uses and building operations topics addressed by the BOC Program are fairly comprehensive and include subjects related to natural gas usage, it is difficult to quantify program benefits beyond what is identified through participant-reported data. In order to assist in further highlighting the value of education provided through the BOC Program, the evaluators recommend that MEEA take note of any natural gas projects mentioned by current or past graduates, and track these projects so that they can be assessed and potentially attributed to the program.

Prepare students for possible EM&V outreach: During the participant survey effort, several respondents expressed concern regarding the purpose of the survey, and at least one participant was hesitant to provide project-related information to the evaluators due to concerns that the information would be used inappropriately. While the evaluators were able to explain the context of the survey and the reason for the evaluation, it appeared that



participants may not have been aware that they would be contacted for this purpose. In order to assist in collecting feedback and project-related data from program participants, it may be useful to ensure that students are aware of the presence and purpose of EM&V, and that they may be contacted during the months following their graduation from the BOC Program.


1. Introduction

This report presents the results of the impact and process evaluation of the Building Operator Certification Program offered by the Department of Commerce. This report presents results of activity during the period June 2015 through May 2016.

1.1 Description of Program

The Building Operator Certification Program (BOC Program) is a nationally recognized competency based training and education program for building operators. The Department of Commerce provides funds for program administration, instructor fees and travel, training coordination fees and travel, marketing and outreach, and tuition rebates for program graduates. The program is administered in partnership with the Midwestern Energy Efficiency Alliance (MEEA), which administers a regional program in eight states through a license from the BOC copyright holder, the Northwest Energy Efficiency Council (NEEC).

The Department of Commerce and MEEA launched the BOC Program in Illinois in 2003. EPY8/GPY5 marks the second year of the current three-year program cycle.

1.1.1 Program Administration

MEEA is responsible for managing the grant from the Department of Commerce, marketing the program, and facilitating the course. Once NEEC approves the application and the certification is official, MEEA will provide the rebate for the course.

The majority of the course materials provided by NEEC are related to technical foundations. MEEA works with instructors to create course content specific to the region, e.g. weather impacts and utility program specifics. Some instructors are involved with the advisory committee that determines the strategic direction of the program including the certification standards, course content, and future program scope. Eligibility requirements for BOC instructors include:

Instructors must have teaching experience and technical expertise in the course topic area for which they apply. NEEC evaluates applications for both instruction and industry experience.

3+ years of experience providing instruction to working professionals in the field(s) of commercial building energy management, facility management, building engineering, operations and maintenance, or a closely related field.

2+ years of employment in the field or industry related to the training topic(s) for which the applicant is seeking qualification (e.g., HVAC systems, electrical systems, indoor air quality, etc.)

Bachelor’s Degree. Work experience may be substituted.

The program is publicized through trade publications, industry associations, and industry groups such as ASHRAE and the State Board of Education.

Introduction 1


1.2 Impact Evaluation Approach

The overall objective of the impact evaluation of the BOC Program was to estimate the electric and natural gas energy savings that resulted from participation in the program. The impact evaluation excludes energy savings achieved through projects for which the operator received an incentive through another Department of Commerce or EEPS program.

The M&V approach includes the following main features:

Surveys administered to EPY8/GPY5 BOC Program participants;2

Telephone interviews to identify participants who implemented energy efficiency measures for which no EEPS-funded incentive was received;

Telephone verification of claimed energy efficiency measures at sampled sites; and

Extrapolation of energy savings of sampled participants to account for the population of participants.

1.2.1 Data Collection Procedures

Participants in the BOC Program for EPY8/GPY5 were contacted by telephone or email to ascertain what energy efficiency measures they had implemented since attending the training program. Participants were also asked questions to determine the probability that they would have implemented the measures without the training and questions related to process evaluation.

Although ADM attempted to contact all EPY8/GPY5 participants by telephone and email, some participants did not respond to the survey requests. Out of the 53 participants who completed the BOC Program training during the program year, 13 responded to the initial participant survey.

Follow-up telephone interviews and email communications were conducted for those participants who stated they implemented energy efficiency measures as a result of the training, and did not receive an incentive for these measures from another Department of Commerce program or EEPS utility rebate.

1.2.2 Data Collection and Estimation of Sample Site Gross Savings

During the follow-up telephone interviews and email communications, savings analysis staff accomplished two tasks:

First, the implementation status of all measures referred to by interviewed participants was verified. ADM evaluation staff members verified that the energy efficiency measures were installed and functioning properly.

2 ADM attempted to contact all EPY8/GPY5 participants for the purposes of telephone or online surveying. A total of 13 participants ultimately responded to the survey requests.

Introduction 2


Second, ADM staff members collected information regarding any details necessary for savings calculation. Data were collected based on the measure input requirements of the savings estimation methodology being referenced for the particular measure.

1.3 Process Evaluation Approach

This section presents the key tasks that were included in the process evaluation for the program year. The EPY8/GPY5 evaluation included a limited process evaluation focused on identifying any significant changes to program design, course curriculum, and delivery, and on tracking the status of evaluation conclusions and recommendations that were identified in previous evaluation years. In order to accomplish this process evaluation, the evaluators conducted a review of current program documentation including course assessments, held in-depth interviews with MEEA staff, administered participant satisfaction and course feedback surveys, and administered surveys to supervisors of participants who had completed the BOC training during EPY8/GPY5.

1.3.1 Review Program Documentation

At the start of the process evaluation effort, the evaluators reviewed documentation and data for the BOC Program. This involved working with MEEA staff to identify and obtain relevant documents for review.

As with prior years, the evaluators reviewed participant tracking records. These data were used for several purposes.

Preliminary analysis of the characteristics of the participant populations, to be used for planning purposes and provide an increased understanding of program participation.

Extracting information about participant facility types and the types of businesses represented by program participants.

Quantifying the total number of EPY8/GPY5 BOC Program participants for the purposes of savings extrapolation.

Other reviewed documentation included certification maintenance activities completed by past participants, updated course curriculum summaries, and internal course assessment forms that were filled out by EPY8/GPY5 participants upon course completion.

1.3.2 Conduct Program Staff Interviews

The evaluators interviewed MEEA program management staff in order to gain insight into changes to program structure or operation, to identify current program issues and trends, and to determine the status of issues identified during prior evaluations.

For EPY8/GPY5, topics addressed by the in-depth interview included:

Organizational changes to the program since EPY7/GPY4;

Introduction 3


Topic related to program resources and challenges associated with statewide budget issues;

Marketing activity and strategy for the current program year;

Current strengths and weaknesses of the program;

Areas where the program has been changed or strengthened; and

Anticipated changes to the program.

1.3.3 Conduct Participant Surveys

The evaluators collected data from BOC Program participants to support the process evaluation. As with prior evaluations, the goal of these surveys was to obtain a detailed understanding of the participant perspective of the BOC Program, the process involved in participants’ making the decision to attend training, participants’ perceptions of the process, the effect of the training programs on participants’ knowledge and behavior, and the benefits the participants perceive. In total, 13 of the 53 BOC participants responded to the participant survey. This response rate was achieved as a result of three rounds of email invitations, two rounds of telephone calls, and a newsletter reminder sent out by MEEA.

The content of the survey was similar to that of prior program years and focused on the following issues:

Motivations for participating in the program;

Factors that influenced the participant to enroll in the program;

Perceived benefits from completing the training courses;

Satisfaction with the program;

Suggestions for program improvement;

Whether the participant has engaged in energy efficient practices since participating in the program;

Whether the participant made additional energy efficient purchases since participating in the program; and

Firmographics and occupation details.

The results from the participant survey are used to inform both the process and impact components of the evaluation. The evaluators used information provided by participants to identify potential energy saving projects and follow-up with facilities as needed in order to collect necessary project details. Additionally, the participant survey provided insight into the participant perspective, allowing the evaluators to identify trends in program performance and any issues regarding program structure, operation, and delivery that may require attention.

Introduction 4


1.3.4 Conduct Supervisor Surveys

ADM administered an online survey to supervisors of employees who attended the BOC training during EPY8/GPY5. The purpose of the survey was to assess the value of the training to the organization, identify any impacts on employees’ job behaviors and performance that the supervisor may have observed, barriers to completing efficiency improvements, and barriers to participation in the program. ADM received the contact information for 45 supervisors, of whom three responded to the survey. This response rate was achieved as a result of two rounds of email invitations.

1.4 Organization of Report

This report on the impact and process evaluation of the Building Operator Certification Program for the period June 2015 through May 2016 is organized as follows:

Chapter 2 presents and discusses the methods used for estimating savings for measures installed under the program.

Chapter 3 presents and discusses the methods used for and results obtained from estimating net savings the program.

Chapter 4 presents and discusses the results obtained from the process evaluation of the program.

Chapter 5 presents evaluation conclusions and recommendations for the program.

Appendix A provides a copy of the questionnaire used for the participant survey.

Appendix B presents tabulated results from the participant survey.

Appendix C provides a copy of the questionnaire used for the supervisor survey.

Introduction 5

2. Savings Calculation Methodology

This chapter addresses the estimation of kWh and peak kW reductions resulting from measures implemented in facilities of participants that obtained tuition rebates from the Department of Commerce for participating in the Building Operator Certification Program in electric program year eight and natural gas program year five (EPY8/GPY5) during the period of June 2015 through May 2016. Section 2.1 through Section 2.3 describe the steps taken to identify energy saving projects, select the appropriate data reference sources, and calculate the resulting energy savings. Chapter 3 describes the net savings estimation methodology and presents the total EPY8/GPY5 net savings for the program.

2.1 Review of Participant Survey Responses

The participant survey administered to BOC training participants served as the initial source for data regarding projects implemented during EPY8/GPY5. Participants provided information related to measures installed and equipment changes implemented after participating in the training program. Participants provided available inputs such as measure type, facility square footage, and other details. The evaluators reviewed these results and identified all projects that would potentially generate savings for EPY8/GPY5 of the program.

2.2 Selection of Data Sources for Savings Calculation

Upon completion of the data collection process, the evaluators performed a desk review of the available data and determined the optimal savings calculation methodology. The evaluators referred to several sources in order to estimate savings for each measure type. This process included referring to the Illinois TRM for deemed savings values and stipulated savings calculations, as shown in Table 2-6 below.

Table 2-6 Sources Referenced for Savings Calculations

Measure Category Energy Savings SourcesEnergy Efficient Lighting Illinois Statewide TRMEnergy Efficient Motors Illinois Statewide TRM

2.3 Savings Methodologies by Measure

The following section lists each measure type, along with the formula or deemed savings determination used during the impact evaluation.

Savings Calculation Methodology 1


2.3.1 LED Energy Savings

The energy savings associated with LEDs were quantified using the deemed calculations shown in the Illinois Statewide TRM. The calculations are as follows:

ΔkWh = (Wattsbase– Wattsee)* HOURS * WHFe * ISR

Where,

Wattsbase = input wattage of the existing or baseline system.

Wattsee = actual wattage of LED purchased/installed.

Hours = total operating hours of the lighting.

WHFe = waste heat factor for energy to account for cooling energy savings from efficient lighting

ISR = in Service Rate (assumed to be 100%)

Summer Coincident Peak Demand Savings

ΔkW =((Wattsbase-WattsEE)/1000) * WHFd*CF

Where,

WHFd = Waste Heat Factor for Demand to account for cooling savings from efficient lighting in cooled buildings.

CF = Summer Peak Coincidence Factor

The facility qualifying for net attributable savings for LED fixtures in EPY8/GPY5 had installed 12 LED lamps in an exterior location. The facility had annual operating hours of 4,903, and a baseline wattage of 160 for the existing lamps.

2.3.2 Energy Efficient Motors

The energy savings associated with energy efficient motors were quantified using the deemed calculations shown in the Illinois Statewide TRM. The calculations are as follows:

kWhbase = (0.746 × 𝐻𝑃 × 𝐿𝐹/𝜂𝑚𝑜 ) × 𝑅𝐻𝑅𝑆𝐵𝑎𝑠𝑒 × ∑ (%𝐹𝐹 × 𝑃𝐿𝑅𝐵𝑎𝑠𝑒)

kWhretrofit = (0.746 × 𝐻𝑃 × 𝐿𝐹/𝜂𝑚𝑜 ) × 𝑅𝐻𝑅𝑆𝐵𝑎𝑠𝑒 × ∑ (%𝐹𝐹 × 𝑃𝐿𝑅Retrofit)∆kWhfan = kWhBase – kWhRetrofit



∆kWhtotal = ∆kWhfan × (1 + IEenergy)

Where:

kWhbase = Baseline annual energy consumption (kWh/yr)

kWhretrofit = Retrofit annual energy consumption (kWh/yr)

∆kWhfan = Fan-only annual energy savings

∆kWhtotal = Total project annual energy savings

0.746 = Conversion factor for HP to kWh

HP = Nominal horsepower of controlled motor

LF = Load Factor; Motor Load at Fan Design CFM (Default = 65%)𝜂𝑚𝑜𝑡𝑜𝑟 = Installed nominal/nameplate motor efficiency

RHRSbase = Annual operating hours for fan motor based on building type

%FF = Percentage of run-time spent within a given flow fraction range

PLRbase = Part load ratio for a given flow fraction range based on the baseline flow control type

PLRretrofit = Part load ratio for a given flow fraction range based on the retrofit flow control type

IEenergy = HVAC interactive effects factor for energy (default 15.7%)

Summer Coincident Peak Demand Savings

kWbase = (0.746 × 𝐻𝑃 × 𝐿𝐹/𝜂𝑚𝑜 ) × 𝑅𝐻𝑅𝑆𝐵𝑎𝑠𝑒,FFpeak kWretrofit = (0.746 × 𝐻𝑃 × 𝐿𝐹/𝜂𝑚𝑜 ) × 𝑅𝐻𝑅𝑆𝐵𝑎𝑠𝑒,FFpeak∆kWfan = kWBase – kWRetrofit

∆kWtotal = ∆kWfan × (1 + IEdemand)

Where:

kWbase = Baseline summer coincident peak demand (kW)

kWretrofit = Retrofit summer coincident peak demand (kW)

∆kWfan = Fan-only summer coincident peak demand impact

∆kWtotal = Total project summer coincident peak demand impact

PLRbase = The part load ratio for the average flow fraction between the peak daytime hours during the weekday peak time period based on



the baseline flow control type (default average flow fraction during peak period = 90%)

PLRretrofit = The part load ratio for the average flow fraction between the peak daytime hours during the weekday peak time period based on the retrofit flow control type (default average flow fraction during peak period = 90%)

IEdemand = HVAC interactive effects factor for summer coincident peak demand (default 15.7%)

The facility qualifying for net attributable savings for motors improvements in EPY8/GPY5 had installed three motors as an exhaust-return end use. The motors are 7.5 horsepower and are controlled by variable-speed drives (VSDs).


3. Estimation of Net Savings

This chapter reports the results from estimating the net impacts of the Building Operator Certification (BOC) Program during EPY8/GPY5, where net savings represents the savings achieved by program participants that can be attributed to the effects of the program

As the savings calculation methodology was based on responses received from the participant survey and required follow-up calls with participants who reported implementing measures, the evaluators determined program attribution levels prior to contacting participants for follow-up data collection. This allowed the evaluators to contact only those participants who indicated that they had implemented a project, and whose were determined to be at least partially attributable to the program. As the savings calculation methodology did not involve following up with participants who were identified as full free riders, the evaluation focused exclusively on net savings rather than estimating net and gross savings.

3.1 Procedures Used To Estimate Net Savings

For the BOC Program, the evaluators assessed the program attribution of each measure by assessing whether the Building Operator Certification training influenced the implementation of the measure.

Net energy savings analysis for training programs may typically involve determining whether or not a participant had plans and intentions to attend the training independent of program support such as tuition rebates. However, for the purposes of the BOC evaluation, it was determined that the Department of Commerce provides multiple forms of financial and non-financial support that are instrumental to the operation of the BOC program.

Thus, even if a participant states that he or she would have attended the training without receiving the Department of Commerce tuition rebate, it is not possible to determine whether the Department of Commerce was indirectly influential in the participants’ decision making. For example, MEEA staff has stated that some BOC training courses would not have taken place, or that they would have had to limit enrollment, if the Department of Commerce had not provided financial and non-financial support to the program structure.

The evaluators determined that while the Department of Commerce tuition rebate is likely an important factor in participant decision-making, its importance to participants would not be considered for the purposes of the net savings analysis. This determination has been implemented for all evaluation years including EPY8/GPY5.

Thus, savings from the action of a participant are attributable to the program as long as the participant would not have taken the same energy saving action without attending the BOC training. In order to assess this factor, “Building Operator Certification training influence on

Estimation of Net Savings 1


project implementation”, participant survey respondents were asked the following two questions for each reported measure:

Net Savings Question 1: “ How important was the information and/or assistance you received through the Building Operator Certification Program in your decision to implement [measure], using a scale of 0 to 10, where 0 is not at all important and 10 is extremely important?”

Net Savings Question 2: ”If you had not received the information and/or assistance through the Building Operator Certification Program, how likely is it that your organization would still have implemented [measure], using a 0 to 10 scale, where 0 means you definitely would not have implemented this measure and 10 means you definitely would have implemented this measure?

The response to the first question cited above is Measure Attribution Score 1, and the response to the second question cited above is Measure Attribution Score 2. Savings associated with reported measures were considered to be attributable to the program if the “Attribution Score” is greater than 7.0. The “Attribution Score” is defined as follows:

Attribution Score = (Measure Attribution Score 1 + (10 – Measure Attribution Score 2))/2

If this condition is met, the evaluators determine that the specific measures referenced in the question are attributable to the program; otherwise, the evaluator determines that the specific measures referenced in the questions are not attributable to the program. The attribution criteria represent a threshold approach, in which energy impacts associated with measures implemented by program participants are either 100% program-attributable or 0% program-attributable.

This is consistent with the Training and Technical Assistance Protocol section, “Approach for Identifying and Quantifying Program-Attributable Savings”, as presented in Volume 4 of the Illinois Technical Reference Manual (TRM) Version 5.0, “Cross-Cutting Measures and Attachments”.

In addition to the criteria outlined above, the evaluators referenced available data including consistency check data and information gathered during follow-up telephone calls to perform documented modifications to individual attribution determinations.

To prevent double counting savings across programs, participants were asked if they received an incentive for the energy saving project that was implemented. If they did, these savings are not attributed to the BOC program.

The data used to determine net savings were collected through a participant survey of 13 program participants for projects completed during or after participant attendance of BOC training courses in EPY8/GPY5.



3.2 Results of Net Savings Estimation

The procedures described in the preceding section were used to estimate net savings for the Building Operator Certification (BOC) Program during EPY8/GPY5.

Four of the 13 surveyed participants indicated that they implemented at least one project due to their experience in the BOC Program and had not applied for or received an incentive through a utility or Department of Commerce program (e.g. a project that met the above program attribution criteria). Savings were calculated only for projects that met the net savings criteria described in Section 3.1.

The following table presents the number of reported projects by measure type or maintenance category. The first column displays the total number of measures reported by survey respondents, regardless of incentive receipt or program influence. The second column displays the number of reported measures for which survey respondents reported that they had not applied for or received an external incentive. Follow-up telephone interviews were conducted to determine the savings for projects identified in the “Reported Measures Without Incentive that were BOC Training Influenced” column of the table. The right-most column identifies the number of measures that were verified as evaluable and that generated savings attributable to the BOC Program.



Table 3-7 Reported Projects by Measure Type and Influence Level

Measure/Maintenance Type

Number of Projects

Total Reported Measures

Reported Measures

without External Incentive

Reported Measures Without

Incentive that were BOC Training

Influenced

Verified and Evaluable

Net Savings Measures

Lighting Controls 3 1 - -Lighting 7 5 1 1Motors 2 1 1 1VSD 1 - - -EMS 2 - 1* -Economizer 2 2 1* -Heating System 2 2 1 -Air Conditioning 3 3 1* -Compressed Air 1 1 1* -Other Improvements 2 2 - -Electric Panel Maintenance 1 1 - -Ventilation Maintenance 2 2 - -Total 28 20 6 2

*Measure not implemented as of 2/28/17, based on engineering follow-up call

The evaluators attempted to conduct follow-up verification and data collection telephone calls with each participant to ensure that measures cited during the survey effort were accurately recorded and were associated with BOC Program influences.

The above values are based on responses gathered through the participant survey effort, and do not necessarily reflect the number of projects that achieved savings through the verification and measurement effort. Specifically, a total of four measures were screened out as ineligible for net savings during the engineering follow-up communications. All of these measures were reported by a single customer, who clarified that the energy management system, economizer, air conditioning system, and heating system projects had not yet been implemented in their facility and that they are currently collecting baseline data in order to determine the viability of such projects. These four measures were removed from the list of eligible projects. Additionally, one respondent who reported implementing heating system improvements did not respond to engineering follow-up efforts and therefore insufficient data were collected to calculate savings for this measure. This respondent was removed from the survey sample prior to extrapolating savings to the BOC participant population.

Table 3-8 displays the distribution of respondent scores for the two survey questions that are used to structure the net-to-gross determination for reported measures. As per the net savings methodology outlined in Section 3.1, measures must receive an overall score of seven or greater



in order to be categorized as eligible for net savings through this training program, where the overall score is calculated as the average of the Importance of Program score and the inverse of the Likelihood of Implementation without Program score. The values shown below represent all reported measures in the survey, regardless of whether a separate utility or Department of Commerce incentive was received for the project.

Table 3-8 Distribution of Net-to-Gross Responses for Cited Projects

Score

Distribution of Respondent Scores

Importance of Program

Likelihood of Implementation

without Program

10 1 39 2 48 13 17 3 16 1 15 4 34 0 33 0 32 0 31 0 00 3 5

Average 6.6 4.8

3.2.1Discussion of Net-to-Gross Findings

Of the 28 measures that were identified by survey respondents, 19 did not pass the net attribution criteria. This represents a net attribution rate of approximately 32% for identified projects, and suggests that many participants already had plans to implement measures prior to their participation in the training, or that the decision to implement the energy saving measures was made by an individual other than the person who had attended BOC training. The percentage of projects associated with a utility or Department of Commerce incentive was approximately 29%, which is a decrease from EPY7/GPY4.3 This decrease may be due to the limited availability of energy efficiency program incentives during EPY8/GPY5; statewide budget issues were a limiting factor across the portfolio of available EEPS programs. The measures that did receive incentives may have been influenced by the BOC training, but incentivized savings are claimable by the Department of Commerce and the utilities and cannot also be attributed to the BOC Program.

3 During the EPY7/GPY4 evaluation, respondents reported applying for and/or receiving incentives for 56% of reported measures.



3.3 Net Savings Summary

Table 3-9 presents the sampled net savings, by measure, for each measure category that achieved net savings within the sampled participant group. The sampled projects that were found to be eligible for net savings were an efficient motors project and an efficient lighting project. Few maintenance improvements were cited by participant survey respondents, and none of these maintenance projects met the net attribution criteria specified in Section 3.1.

No natural gas savings were identified as evaluable and eligible for net savings during the participant survey effort or subsequent engineering follow-up calls.

Table 3-9 Net Savings by Measure for Participant Sample

Measure CategoryTotal Sampled Net Savings

kWh kW ThermsEnergy Efficient Lighting 7,213 0.00 -Lighting Controls 717 0.35 -Total 7,931 0.35 0.00

The total savings shown above were extrapolated to represent the population of BOC participants who completed the MEEA BOC Program training during EPY8/GPY5. The evaluators were able to conduct participant surveys with 13 of the 53 BOC participants who completed the course during EPY8/GPY5. Based on participant survey responses, four of the 40 respondents reported energy saving projects that potentially qualified for net savings attributable to the BOC Program. Of the four sampled BOC participants who were associated with potential net savings through the program, evaluators were able to contact and verify savings for three facilities. The remaining participants could not be reached during the follow-up effort. This participant was removed from the sample and treated as part of the remaining population of BOC participants who were not reached during the survey effort.

Additionally, during engineering follow-up discussions the evaluators determined that the projects reported by one facility had not yet been implemented; therefore net savings were calculated for a total of two facilities.

This resulted in a total savings value representing the two BOC participants who had implemented projects attributable to the program. This savings value was then divided by the total number of survey respondents (12 respondents, after subtracting out the participants with insufficient information) in order to determine the average savings per sampled participant. Once this savings value was determined, the savings were extrapolated to the program participant population.4

4 The sampled savings were extrapolated to a population of 53 total participants.



Savings were extrapolated based on the distribution of utility service providers within the participant population. Table 3-10 presents the percentage of BOC participants serviced by each electric utility during EPY8/GPY5. These proportions were applied to the net savings value in order to develop savings by utility.

Table 3-10 Distribution of Electric Utilities among BOC Participants

Utility Percentage of Total Participants

Ameren 45%ComEd 45%Other 10%

Total 100%

Table 3-11 presents the net kWh savings by utility for the Building Operator Certification Program during EPY8/GPY5. It should be noted that because some participants were serviced by non-EEPS electric utilities such as municipal utilities, electric savings generated through these participants were not claimable by the BOC Program investor utilities.

Table 3-11 Summary of Net kWh Savings for BOC Program

Electric Utility Realized Net kWh Savings

Ameren 15,797

ComEd 15,797

Total 31,593

Table 3-12 presents the net kW savings by utility for the Building Operator Certification Program during EPY8/GPY5.

Table 3-12 Summary of Net kW Savings for BOC Program

Utility Realized Net kW Savings

Ameren 0.70

ComEd 0.70

Total 1.40

The total net energy savings of the Building Operator Certification Program during EPY8/GPY5 are summarized in the following table. During this period, net energy savings attributed to the program totaled 31,593 kWh and 1.40 kW. These values do not include savings generated through non-EEPS utilities, which totaled 3,434 kWh and 0.15 kW.



Table 3-13 Summary of Net Savings from EPY8/GPY5 Projects

Savings LevelTotal Net Savings

kWh kW Therms

Per Participant 661 0.03 -

Extrapolated to EPY8/GPY5 Participants 31,593 1.40 -

The electricity savings for EPY8/GPY5 are significantly lower than the savings that were attributed to EPY7/GPY4 (approximately 1,752,400 kWh). As with EPY7/GPY4, no natural gas energy savings were identified for EPY8/GPY5. Although the participant population in EPY8/GPY5 was smaller than previous years (there were 117 participants in EPY7/GPY4), this does not explain the magnitude of the decrease in savings; per-participant savings in EPY7/GPY4 were approximately 18,005 kWh, while per-participant savings in EPY8/GPY5 are approximately 661 kWh.

As noted in prior evaluations of this program, one contributing factor to the variability in attributable savings across program years is likely the relatively small sample sizes available for evaluation purposes. With a sample size of 13, as was obtained in EPY8/GPY5, the presence of a single project with high savings would significantly affect total program savings. However the extrapolated savings for EPY8/GPY5 are based on two relatively small projects that in total consisted of replacing three motors and 12 exterior LED lamps. As this evaluation did not identify any eligible large industrial projects, such as the compressed air project that contributed to the majority of savings in EPY7/GPY4, observable program impacts are fairly limited for EPY8/GPY5. However these results are not necessarily predictive of future performance; as stated in prior evaluation reports, the range of possible projects implemented by BOC participants is very wide in terms of scope, cost, and end use likely resulting in high savings variability across program years.


4. Process Evaluation

This chapter discusses results of the Building Operator Certification Program process evaluation for electric program year eight and natural gas program year five (EPY8/GPY5).

The purpose of the process evaluation is to assess the program from a structural, operational, and managerial perspective in order to identify program strengths, weaknesses, and opportunities. This evaluation is based on surveys with BOC participants, supervisors of participants, MEEA staff feedback, and analysis of program data and documentation.

As the BOC Program has now been evaluated for multiple consecutive years, the evaluators conducted a limited process evaluation focused on identifying any significant changes to program design, course curriculum, and delivery, and on tracking the status of evaluation conclusions and recommendations that were identified in previous evaluation years.

This chapter begins with a summary and discussion of the results from the EPY8/GPY5 BOC participant survey. This is followed by a discussion of the outcomes of the MEEA staff interview. The chapter concludes by highlighting key findings and program recommendations resulting from the process evaluation.

4.1 Evaluation Objectives

The purpose of the process evaluation is to examine program operations and results throughout the program operating year, and to identify potential program improvements that may prospectively increase program efficiency or effectiveness in terms of participation and satisfaction levels.

This process evaluation was designed to document the operations and delivery of the Building Operator Certification Program during electric program year seven and natural gas program year four (EPY8/GPY5). Figure 4-1 provides an overview of the evaluation process, including the research activities performed.

Process Evaluation 1


Figure 4-1 Process Evaluation Overview

Key research questions to be addressed by this evaluation of EPY8/GPY5 activity include:

Is the Building Operator Certification Program using its available resources in a way that sufficiently supports program operation, growth, and performance?

Is the Building Operator Certification Program effectively engaging participants and meeting their energy efficiency and educational needs?

Did the Building Operator Certification Program reduce barriers to increased energy efficiency project implementation?

Did the Building Operator Certification Program respond to previous recommendations obtained through prior evaluation efforts?

During the evaluation, data and information from several sources were analyzed to achieve the stated research objectives. Participant perspectives on the BOC training program were collected using a survey tool conducted over the phone and online. Participants’ supervisors were surveyed in order to gain the supervisors’ perspectives on program benefits and how their employees have applied the knowledge they gained in the BOC Program to their workplace. Staff perspectives on the internal organization and operational efficiency of program delivery were examined through an interview with MEEA program management staff, and review of program documentation (e.g. participant tracking data).


Research FindingsParticipant PerspectiveSupervisor Perspective

Program Operations Perspective

Research ActivitiesParticipant Surveys

Program Structure ReviewMEEA Staff Interview

Participant Supervisor Surveys

Program BackgroundParticipation Data Program Documentation Prior Evaluations


4.2 Summary of Primary Data Collection

Participant surveys: Participant surveys serve as the foundation for understanding the participant perspective. The participant surveys provide participant feedback and insight regarding participant experiences with the Building Operator Certification Program. Respondents report on their satisfaction with the program, detail their motivations and the factors affecting their decision making process, and provide recommendations related to improving the program. For EPY8/GPY5 of the Building Operator Certification Program evaluation, 13 program participants responded to the participant survey. This response rate was achieved as a result of three rounds of email invitations, two rounds of telephone calls, and a newsletter reminder sent out by MEEA.

Supervisor surveys: ADM administered an online survey to supervisors of employees who attended the BOC training during EPY8/GPY5. The purpose of the survey was to assess the value of the training to the organization, identify any impacts on employees’ job behaviors and performance that the supervisor may have observed, barriers to completing efficiency improvements, and barriers to participation in the program. ADM received the contact information for 45 supervisors, of whom three responded to the survey. This response rate was achieved as a result of two rounds of email invitations.

Interviews and discussions with MEEA staff: In-depth interviews and ongoing discussions with MEEA staff throughout the evaluation provided insight into various aspects of the program and its organization. Specifically, the interview focused on the status of BOC offerings during EPY8/GPY5 in the context of statewide budget issues that affected a variety of EEPS offerings.

4.3 Participant Outcomes

A telephone survey was conducted to collect data about participant decision-making, preferences, and opinions of the Building Operator Certification (BOC) Program. In electric program year eight and natural gas program year five (EPY8/GPY5), 53 course participants successfully completed the training and received the associated certification. In total, 13 participants fully responded to the process evaluation components of the telephone survey.

The EPY8/GPY5 survey instrument was nearly identical to the instrument used for EPY7/GPY4 in terms of program areas discussed and types of information gathered. This section presents comparisons between participant responses in EPY8/GPY5 and prior years when appropriate.

4.3.1 Participant Characteristics

When asked to categorize their facility types, respondents provided a wide variety of responses as displayed in Table 4-14. The most common facility types were academic facilities and office buildings, but religious facilities, healthcare facilities, and manufacturing facilities were also among the responses provided.



Table 4-14 Respondent Facility Types

What type of facility is it? (Do not read list)

ResponsePercent of

Respondents (N = 13)

College/University 15%Office - High Rise 15%Elementary 8%Healthcare Clinic 8%Lodging Hotel/Motel 8%Manufacturing Facility 8%Office - Mid Rise 8%Religious Facility 8%Other (please specify) 8%Don't know 15%

Survey respondents were asked a series of questions related to employment including job titles, and length of employment in their current role. As shown in Figure 4-2, 23% of respondents operations or facilities operations manager job titles while 23% of respondents indicated that they are engineers.

Operations/Facilities operations manager

23%

Engineer23%

Other manager, team leader, supervisor15%

Facility tech or assistant15%

Engineering manager8%

Maintenance manager8%

Other maintenance staff8%

What is your current job title? (N = 13)

Figure 4-2 Participant Reported Current Job Titles

When asked how long they had worked in their current role, employed respondents provided a wide range of responses, ranging from 2 to 36 years. The average was approximately 11.5 years.



As with prior evaluations, this suggests that participants are highly experienced in their fields, and are likely very familiar with their facilities’ equipment and processes.

Respondents were also asked about the number of building operator staff in their facilities. On average, respondents reported that their facilities had 7.3 such staff members, a decrease from the average 9.8 reported in EPY7/GPY4. When asked how many of these staff members had completed either Level 1 or both Level 1 and Level 2 of BOC training, respondents reported that an average of 2.6 staff members had accomplished this. Three respondents indicated that they were the only individual in their facility who had completed BOC training.

4.3.2 Existing Energy Efficiency Policies or Procedures

In order to gauge participants’ prior and current organizational structures with regard to energy efficiency, survey respondents were asked about energy efficiency policies or procedures that may be in place at their facilities. As shown in Table 4-15, the majority of respondents reported having a staff member responsible for energy efficiency. Overall, more than three-quarters of respondents indicated that their organization has at least one of the listed policies or procedures in place. The respondent selecting Other explained that their organization also has an engineering company to review energy efficiency initiatives and provide feedback about the organization’s energy opportunities.

Table 4-15 Existing Energy Efficiency Policies and Procedures

Which of the following policies or procedures does your organization have in place regarding energy efficiency improvements?

ResponsePercent of

Respondents(N=13)

A staff member responsible for energy and energy efficiency 54%

An energy management plan 38%

Policies that incorporate energy efficiency in operations and procurement 38%

Active training of staff 38%

Other (please specify) 8%

Don't know 23%

Five respondents provided information about their facilities’ energy management plan goals. The majority of these respondents did not have specific energy usage targets in mind but provided general information about their organization’s approach to energy efficiency when considering specific projects or end uses. Specific commentary related to energy management plans includes:



Right now we are doing a consolidation of the facility. The goal is to save half of the energy budget.

We use 2008 as a baseline and we have a goal each year to save more money [than] that year and the years in between.

We work towards implementing anything that will show or offer improvement in energy savings.

4.3.3 Program Awareness and Information Channels

As with prior years, BOC participants were asked a series of questions designed to offer insight into general program and rebate awareness and to gauge participant interaction with various marketing and information channels.

Figure 4-3 displays participant responses regarding how they learned about the BOC tuition rebate. The percentages shown are the percentages of respondents, and respondents were able to select multiple responses. The most commonly cited channel for learning about the tuition rebate was through a BOC program representative. Respondents also learned of the tuition rebate from friends or colleagues, equipment vendors, and MEEA.

A Midwestern Energy Efficiency Alliance (MEEA) representative

Equipment vendor or building contractor

From a utility representative

Friend or colleague

Don't know

Other (please explain)

From a BOC program representative

0% 5% 10% 15% 20% 25% 30% 35%

8%

8%

15%

15%

15%

23%

31%

Figure 4-3 How Participants Learned about the BOC Tuition Rebate

Several additional response options were provided for this survey question, although some options were not chosen by any respondents. The methods of learning about the BOC tuition rebate that were not cited by any respondents include:

A Department of Commerce representative


Percentage of Respondents (N = 13)


The Department of Commerce website;

Brochures or advertisements;

Trade journals or magazines;

Attended a conference workshop or seminar;

Past experience with the program;

An energy service company; and

An Energy Resource Center (ERC) representative.

Participants were asked to name sources their organizations typically rely on for information regarding energy efficiency (including energy efficient practices, equipment, materials, and design features). The following figure displays the distribution of results, where respondents were able to provide multiple responses.

The most commonly cited source of information was equipment vendors or building contractors. Overall, respondents reported relying on a wide range of sources for energy usage and energy efficiency information. In addition to the listed information sources, respondents selecting the Other response cited sources such as the Institute for Environmental Sustainability, consultants, and email brochures from industry sources.

The Midwestern Energy Efficiency Alliance (MEEA)

Brochures or advertisements

Don't know

Trade associations or business groups you belong to

Friends and colleagues

Utility representatives

Trade journals or magazines

Architects, engineers or energy consultants

Other (please describe)

Equipment vendors or building contractors

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

8%

8%

8%

15%

15%

23%

23%

31%

38%

46%


Figure 4-4 Information Sources Typically Used by Participants



4.3.4 Factors Affecting Participation

Participants cited several main factors when asked why they participated in the courses, as shown in Figure 4-5. Respondents were able to select more than one reason for participating in the program, and the majority of respondents provided at least two responses.

As with EPY7/GPY4, the two most common reasons participants cited for participating in the training course were to learn about energy efficiency or to learn new skills.

To improve my chances of getting a new job

Required by company/organization

Because of the tuition rebate

To earn continuing education credits

Other (please specify)

To advance in my current job

To learn about energy efficiency

To learn new job skills

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

8%

15%

15%

31%

31%

46%

46%

77%


Figure 4-5 Participant Motivations to Enroll in BOC Course

4.3.5 Participant Actions Following BOC Training

As with prior evaluation years, the respondents were asked if any energy efficiency improvements had been made to their facilities since they attended the BOC course. These responses were used to inform the savings impact analysis. This individual question relates only to the timing of projects, and does not yet take into account free ridership levels or whether the participant received a separate incentive for the energy efficiency improvements. Thus, respondents provided information about any energy efficiency improvement since the program, even if the BOC Program did not influence the implementation.

Respondents were asked about a wide range of measures and maintenance activities that may have generated electric or natural gas energy savings. The equipment and other measures addressed by this portion of the survey include:

Lighting;

Lighting controls;

Air conditioning;



Economizer;

Heating system;

Cooling system;

Motors;

Energy Management System (EMS); and

Variable Speed Drive (VSD).

The maintenance activities addressed by this portion of the survey include:

Electric panel maintenance;

Heating system maintenance;

Cooling system maintenance;

Ventilation maintenance;

Compressed air maintenance; and

Motor maintenance.

Additionally, respondents were given the opportunity to provide details about any equipment implementations or maintenance activities that do not fall under these listed categories.

4.3.5.1. Energy Efficient Equipment Implementation

Approximately 70% of respondents (9 of 13) indicated that they had purchased and installed new equipment since participating in the BOC courses. Figure 4-6 displays the types of projects that were cited by these respondents. The distribution of equipment types is fairly similar to that of the past three program years, with lighting being the most commonly reported measure. The next most common energy efficiency measure was air conditioning improvements, followed by heating system improvements. Few respondents reported implementing compressed air or variable speed drive improvements.

It should be noted that the information presented below presents all measures reported by BOC participant survey respondents, regardless of whether they were influenced by the BOC training or the associated tuition rebate. The savings impact chapter of this report (Chapter 3) presents net savings for the BOC Program, taking into account BOC training influence, tuition rebate influence on attendance, and whether the participant received a separate incentive for implementing their energy efficiency project(s).



VSDs

Compressed air projects

Don't know

Energy efficient motors

Energy management system project

Economizer

Other improvements

Lighting Controls

Heating system improvements

Air conditioning improvements

None

Energy efficient lighting

0% 10% 20% 30% 40% 50% 60%

8%

8%

8%

15%

15%

15%

15%

23%

23%

23%

23%

54%


Figure 4-6 Participant Implementations Following BOC Training

4.3.5.2. Maintenance Improvements and Changes

Respondents were asked if they had implemented one or more maintenance improvements at their facility since participating in the BOC training. None of the participants indicated that they had made any changes to cooling system, heating system, motor, or compressed air maintenance. Two participants indicated that they had made a change to their ventilation maintenance procedures; both of these participants stated that they now conduct ventilation maintenance more frequently than they did prior to the training. Additionally, one respondent indicated that they conduct more frequent electrical panel maintenance than they did prior to the training. The presence of reported maintenance changes decreased significantly in EPY8/GPY5 survey results as compared to EPY7/GPY4. None of the respondents reported having made a methodology change to their maintenance practices, which has not been the case in any of the past three prior program evaluations.

4.3.6 Plans to Implement Additional Projects

Although the participant survey was administered several months after participants completed their final course and obtained their certification, the time needed to identify, plan, and implement energy efficiency projects can range from several weeks to a year or more. Thus, it is likely that individuals who completed the training during EPY8/GPY5 will implement energy saving projects that cannot be captured during the EPY8/GPY5 evaluation. In order to identify these projects, survey respondents were asked to identify any projects that are currently planned but have not yet been implemented.



As shown in Table 4-16, the most commonly identified projects were energy efficient lighting, heating system improvements, and air conditioning improvements. When asked whether they had initiated the plans for these projects, four respondents stated that they had initiated the plans while six respondents indicated that someone else had initiated the plans.

It should be noted that the likelihood of these projects being implemented and the timing of the implementation is unclear. Additionally, many of these projects may qualify for Department of Commerce grants or EEPS incentives and the level of influence of the BOC on the implementation of these prospective projects has not been fully established.

Table 4-16 Planned Implementations of Energy Saving Projects

Does your facility currently have plans to implement

any of the following types of energy efficiency

projects?

ResponsePercent of

Respondents (N=13)

Energy efficient lighting 23%Heating system improvements 23%Air conditioning improvements 23%Lighting controls 15%Energy efficient motors 15%Energy management systems 15%VSDs 8%Economizer on air handler 8%None 23%Don't know 38%

4.3.7 Other Energy Efficiency Activities

Respondents were also asked about other activities related to energy efficiency that may have occurred at their facilities. These activities included implementing an energy budget, recording energy use, and setting and achieving energy savings goals. Participants provided information about which of these had occurred prior to participating in the BOC course, and which had occurred only after participating in the BOC course. Figure 4-7 displays the results. For the energy efficiency activities listed, respondents most commonly reported that they had already implemented these activities prior to the training.



Implemented an energy budget

Recorded energy use over time

Set energy savings goals

Achieved energy savings goals

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

46%

46%

31%

23%

23%

8%

8%

Only did after the program Had already done before the program


Figure 4-7 Procedural Energy Efficiency Activities Completed by Participants

4.3.8 Barriers to Implementation

In addition to asking participants whether they had implemented equipment or maintenance improvements since attending the BOC training, survey respondents were asked whether they had encountered any barriers to applying their BOC training in their workplace. Four of the thirteen respondents indicated that they have encountered barriers to energy efficiency implementation in their workplace. Two of these respondents indicated that insufficient budget as a primary barrier, while none of the respondents cited lack of supervisor support as a barrier.

Table 4-17 Barriers to Applying BOC Knowledge

What barriers have you encountered? (Do not read list, but use as possible prompts)

ResponsePercent of

Respondents (N = 4)

Lack of supervisor support 0%Insufficient budget 50%Organization/company not committed to energy efficiency improvements

25%

Not enough staff resources to plan efficiency projects 25%

Other (please specify) 25%Don't know 0%

*Since respondents were able to select more than one response, the sum of the percentages in the table above can exceed 100%.



4.3.9 Participant Satisfaction with the Program

Respondents were asked about their levels of satisfaction with selected aspects of the course, aspects of the financial incentive, and their overall program experience. Responses were provided on a scale of very dissatisfied to very satisfied. Table 4-18 shows participant satisfaction by each selected program element.

Satisfaction ratings were very high for each listed program element, and all respondents reported being satisfied or very satisfied with their overall BOC Program experience. None of the respondents reported being dissatisfied with the BOC course instructors, course schedule, tuition application process, or tuition rebate amount. One respondent reported being dissatisfied with the time elapsed to receive the tuition rebate. Although participants have reported high satisfaction levels during each evaluation year, the EPY8/GPY5 satisfaction ratings are higher than previous program years.

Table 4-18 Participant Satisfaction Ratings by Program Element

Element of Program Experience

Satisfaction Rating (N = 13)

Very Satisfied

Satisfied

Neither Satisfied nor Dissatisfied

Dissatisfied Very Dissatisfied

Don't know

Course instructors 62% 31% 8% - - -Overall experience with BOC Program 54% 38% 8% - - -Tuition rebate amount 54% 15% 15% - - 15%Tuition rebate application process 46% 23% 8% - - 15%Time elapsed to receive tuition rebate 23% 31% - 8% - 31%Course schedule 77% 23% - - - -

4.3.10 Usefulness of Particular BOC Courses

Participants were then asked whether they found any of the courses they attended through the BOC to be particularly useful. All 13 survey respondents reported that they had found at least one of the courses to be very useful. These respondents then provided open-ended commentary discussing the topics and courses that they found particularly useful; the evaluators categorized these comments into topic categories.

Specific courses or subject matter cited as particularly useful by survey respondents include:

Lighting (3 respondents)

HVAC (2 respondents)

Indoor air quality (1 respondent)

Benchmarking (1 respondent)

Building automation systems (1 respondent)



Several respondents did not identify specific topics but provided open-ended commentary indicating that they found several or all of the BOC courses very useful:

Most of them were useful in dealing with utilities and facilities as well as being more efficient overall.

All of them, I use them on a daily basis as I am managing the utility and energy usage for two separate complexes we provide services for.

The courses helped me to understand what decisions and actions upper management makes and now I can understand why they do what they do.

Respondents were then asked whether they thought that any particular BOC course was not useful, but none of the survey respondents indicated that any courses were not useful. This is an improvement over the prior program year, when 15% of respondents reported that at least one course had not been very useful.

4.3.11 Participant Recommendations and Overall Impressions

As with prior program years, participants reported a high level of satisfaction with the program, and emphasized the valuable information they received and how they are able to apply it in the workplace. Although participants appear to perceive significant benefits from participating in BOC courses, the number of evaluable energy savings projects identified in this survey is lower than prior years. This is partially due to a smaller sample size, but overall participants cited fewer projects per facility than in the past. This may be related to multiple issues, including decreased availability of external EEPS incentives during EPY8/GPY5.

Participants also reported fewer maintenance activities than in prior years, and none of the respondents stated that they had made a methodology change to their maintenance procedures as a result of the training. As BOC training continues to include a substantial focus on operations and maintenance best practices, it is unclear why there has been a per-participant decrease in the number of reported maintenance changes. However several participants explained that they are not the main decision maker for energy efficiency implementation in their facility, so while they may be able to contribute to improved maintenance practices, they may not have the ability to immediately apply everything they have learned through the courses.

BOC participants have continued to rely on a wide range of information sources to learn about energy efficiency, which emphasizes the importance of MEEA’s active marketing and outreach strategy. The survey feedback suggests that program representatives and word-of-mouth marketing are likely effective methods of communicating with building operators and their organizations. Past evaluations have also found that BOC participants rely on utility messaging and vendors and contractors for learning about energy efficiency opportunities.

In terms of barriers to energy efficiency implementation, participants most commonly reported insufficient funding as a barrier. However, the majority of participants stated that they had not



encountered any significant barriers in applying the knowledge they had gained through BOC training.

When asked to provide additional open-ended commentary about their experience with BOC, the majority of survey respondents used the opportunity to reiterate their high satisfaction and praise the program’s quality and value. The only recommendations implied by participants were to have longer class times to allow students to more effectively absorb the provided information, and to ensure that instructors are organized and follow the provided course booklets. Each of these was mentioned by one respondent.

Respondents provided extremely high satisfaction ratings for all listed elements of their BOC Program experience, and only one respondent reported being at all dissatisfied with any element (the length of time to receive the rebate). Overall, the participant survey results suggest that program delivery has been very effective, and that there are few potential areas for improvement from the student perspective.

4.3.12 Consistency with Internal Course Surveys

MEEA provided ADM with exports of in-class surveys that were administered to BOC students during EPY8/GPY5. These surveys were designed to gather feedback from students regarding the content and structure of BOC courses, as well as to gauge overall satisfaction with the program. ADM reviewed the results of these in-class surveys and found that the results were consistent with feedback gathered through ADM’s surveying efforts. Students reported high levels of satisfaction with each aspect of the program including course organization, audio/visual materials, handouts, and instructor-led class exercises. Nearly all respondents reported that they would recommend the program to others and that they felt they had received a good value for the time and money spent for the course. Additionally, nearly all respondents indicated that the courses were useful and comprehensive, and that they had introduced a variety of new information.

4.4 Supervisor Outcomes

ADM administered an internet survey to supervisors of employees who attended the BOC training. The purpose of the survey was to assess the value of the training the organization, any impacts on employees’ job behaviors and performance that the supervisor may have observed, barriers to completing efficiency improvements, and barriers to participation in the program. A similar supervisor survey was administered during the EPY7/GPY4 BOC program evaluation. The current survey effort serves to assess participant supervisors’ current opinions and perceived program benefits and to identify any program issues or opportunities relevant to the supervisor perspective that may have emerged since the EPY7/GPY4 evaluation.

ADM received the contact information for 45 supervisors, of whom three responded to the survey. The respondent supervisors represented a total of six BOC graduates from EPY8/GPY5;



two of the supervisors represented two graduates each, while two of the supervisors represented a single graduate.

Due to the limited sample frame and low number of respondents, the information presented in this section is not intended to represent the full population of participant supervisors. Supervisor feedback is presented in order to provide general insight into the supervisor perspective for EPY8/GPY5.

4.4.1 Overall Usefulness of the Program

Supervisors of Building Operator Certification Program graduates were asked whether the courses had been useful in increasing their employees’ skill level and knowledge in various aspects of their jobs. Specifically, the survey asked how useful the courses had been in helping the employee identify energy efficiency improvements, monitor facility energy use, improve maintenance practices, and identify ways to improve occupant comfort. All of the supervisors reported that the courses had been very useful in each of these categories. As with prior program years, these results suggest that supervisors have seen improvement in several aspects of employee performance since the BOC courses were completed.

Two of the three supervisors indicated that the BOC Program had also been useful in helping their employees perform more effectively in other areas of their jobs. When asked to elaborate on these other areas of improvement, these supervisors provided the following responses:

This program let them develop a greater understanding of the importance of energy efficiencies and how they relate to tenant comfort.

[The program assisted with] regulation compliance.

When asked whether their employees had used or applied any of the concepts or methods taught in the BOC courses, all of the supervisors confirmed that employees had done this. This is consistent with prior years and suggests that BOC graduates are actively promoting energy efficiency and building management best practices in the workplace and that the program has resulted in observable improvements in their quality of work.

4.4.2 Equipment Changes Implemented or Recommended Since Graduation

Supervisors were then asked to specify the equipment changes that their employees had either implemented or recommended since they completed the BOC training courses. These changes were separated into several categories, including:

Lighting controls;

Energy efficient lighting;

Variable speed drives or variable frequency drives;

Energy saving improvements to compressed air systems;



Energy management systems;

Energy saving improvements to heating systems;

Energy saving improvements to cooling systems;

Economizers; and

Water heating efficiency improvements.

For each of the above categories, supervisors were asked to indicate whether their employees had either implemented a change or recommended the implementation of a change within that specific system or equipment type. All of the supervisors reported that their employee had both recommended and undertaken at least one project type. The three supervisors identified a total of 10 projects that their employees had undertaken since participating in the BOC Program, and another 13 projects that their employees had recommended since participating in the program.

The most commonly cited projects were energy efficient lighting, with all three of the supervisors reporting that their employees had undertaken lighting projects. The most recommended but not implemented project s were economizer, variable speed drive, and energy management system projects, which had been recommended by four employees but not undertaken by any employees.

Although some of these projects have not yet been implemented and may not be approved by the participants’ organizations, employees of these supervisors appear to be following BOC guidelines by identifying a variety of energy efficiency and comfort improvements and notifying their supervisors of these opportunities.

4.4.3Maintenance Changes Implemented or Recommended Since Graduation

Supervisors were then asked to specify the maintenance changes that their employees had either implemented or recommended since they completed the BOC training courses. One of the three supervisor respondents reported that their BOC-graduate employees had implemented a maintenance improvement since completing the BOC Program. When asked to elaborate on these maintenance changes, this supervisor explained that their employees had implemented the recirculation of condensate to improve cooling efficiencies, monitored the operation of air dampers, and had generally ensured that equipment is operating at peak efficiencies since completing BOC training.

4.4.4Barriers to Energy Efficiency Implementation

In order to gauge the overall ability of organizations to reduce their energy usage, supervisors were asked whether they face any barriers to the implementation of energy efficiency improvements. Two of the three supervisors identified barriers; one supervisor stated that they face a lack of staffing resources for energy efficiency projects, and the other supervisor stated that they face a lack of financial resources for energy efficiency projects. None of the supervisors identified a lack of commitment to energy efficiency, lack of knowledge regarding energy



efficiency, or limited staffing resources as barriers to energy efficiency implementation in their facility.

4.4.5Organizational Importance of Building Operator Certification

The Building Operator Certification Program is now established as a widely recognized training and education program that encourages best practices and contributes to efficient and mindful facility operation. The effects of the training program are not only limited to the specific projects and maintenance improvements conducted by BOC graduates, and include qualitative benefits to employees and their organizations. The supervisor survey included several questions to address this idea, focusing on the overall value and importance of the certification.

BOC participant supervisors were asked whether their employees had added value to their organization since completing the training, with responses separated into the following categories:

Saving energy at your facility;

Saving money;

Helping to improve occupant comfort;

Advising in decisions about equipment operation or replacement;

Having more productive interactions with contractors; and

Undertaking, recommending, or influencing any energy-efficiency projects.

All three supervisors indicated that their employee had added value in all of the above categories, with the exception of one supervisor who did not know whether their employee had helped to improve occupant comfort or had become more productive during interactions with contractors.

In order to gauge to what extent BOC graduates share their training knowledge and educate colleagues about what they have learned, supervisors were asked about their employees’ activities since returning from the BOC courses. When asked whether their employees had shared what they had learned with other employees, all three supervisors reported that this had occurred.

Supervisor respondents were then asked how important to the hiring decision it would be for a potential employee of their organization to have the Building Operator Certification. All three of the supervisors reported that it would be either important or very important to the hiring decision. Supervisors were then asked how important having the certification is for current employees to receive promotions or advancements, and all three of the supervisors stated that it is an important or very important factor.



4.4.6Considerations for Enrolling Employees in BOC

In order to gauge how supervisors determine who to send to BOC training, respondents were asked what factors they consider when deciding whether to enroll employees in the program. These considerations were divided into a list of categories including time and staff availability, training location and costs, employee professional development, legal requirements, benefits to the organization, and the employee’s personal interest.

Two of the three supervisors indicated that location of the training and employee professional development were considerations when deciding whether to enroll an employee in the training. Training costs and employee personal interest were each cited by one of the supervisors as important considerations.

As a follow-up question, supervisors were then asked whether their employees would have been sent to the Building Operator Certification Program if the tuition rebate had not been available. Two of the three supervisors stated that their employees probably would have attended the training without the rebate, and the remaining supervisor stated that their employee probably would not have attended. As with the prior program year, this suggests that the perceived benefits of Building Operator Certification were numerous or valuable enough to justify spending the full tuition cost.

4.4.7Future Energy Efficiency Activity

In order to gauge whether the BOC has had a significant effect on organizations’ overall decision making and planning, supervisors were asked to speculate about their future involvement in energy efficiency. First, supervisors were asked whether the BOC training has increased the likelihood that their organization will participate in energy efficiency programs such as incentive programs. Two of the three supervisors stated that the program has increased this likelihood, and were asked to elaborate on their responses. The one supervisor who provided an open-ended comment stated:

Due to our success, others are interested in making [their properties] more efficient and I use this program as a reason we have succeeded.

As a general follow-up question, supervisors were asked whether the employee training for Building Operator Certification has increased the likelihood that the organization will make investments in energy efficiency. All three of the supervisors reported that the program has increased the likelihood of these investments.

4.4.8Future Enrollment and Program Referrals

Finally, supervisors were asked about future plans to recommend the BOC Program or to enroll additional employees in the program. Two of the three supervisors reported that they would recommend the BOC Program to their colleagues, either within or outside of their organizations (the third supervisor stated that they didn’t know whether they would recommend the program).



One of the three supervisors stated that they expect to enroll additional staff in the program during future years.

4.4.9Overall Supervisor Impressions

The EPY8/GPY5 supervisor survey responses suggest that supervisors of BOC Program graduates associate Building Operator Certification with a high level of value to their organizations. All three supervisor respondents indicated that the program courses had resulted in positive effects related to their employees’ work performance, and all supervisors indicated that their employees had recommended and undertaken energy saving actions as a result of their participation in the training program. As with prior program years, both the participant survey and supervisor survey suggest that supervisors are supportive of their employees’ application of BOC-related knowledge in the workplace.

4.5 Program Operations Perspective

This section summarizes core Midwest Energy Efficiency Alliance (MEEA) staff interview findings. In order to gather information regarding the operational efficiency and program delivery process for the Building Operator Certification Program, the evaluators conducted interviews and ongoing discussions with MEEA program management staff. Discussion topics were designed to provide insight into any changes to the design, structure, and operation of the BOC Program since EPY7/GPY4, and to identify current program issues and trends. One key focus of the EPY8/GPY5 staff interview effort was to gain insight into how the statewide budgeting issues had affected the design and deliver of the BOC Program.

4.5.1 Summary of Interview Findings

Statewide budget delay limited program operations: MEEA staff explained that the primary challenges for BOC during EPY8/GPY5 were related to the lack of a statewide budget for Illinois. The structure of energy efficiency funding in Illinois stipulates that 75% of the available EEPS budget is provided to utilities, while 25% is provided to the Department of Commerce. The Department of Commerce uses these funds to implement and support a variety of energy efficiency programs and services, including providing tuition rebates to BOC graduates. With the statewide budget delay, MEEA was unable to provide these tuition rebates to participants until after a temporary budget was passed, which occurred after the end of EPY8/GPY5. Throughout the program year, MEEA continued to operate BOC training courses and conduct recruitment efforts, but as the budget delays continued MEEA became cautious about expending resources. MEEA staff reported that class sizes were smaller in general during the program year, and that one course had to be cancelled due to low enrollment. In total, 53 participants completed the BOC program during EPY8/GPY5.

Delay in tuition reimbursement: Despite the budgetary issues, MEEA continued to operate BOC training courses and informed participants that they would receive the tuition reimbursement as soon as funding became available. Tuition reimbursements were



promised to students attending each of the courses offered during EPY8/GPY5 with the exception of the first course that was offered in Springfield, IL. Students attending this course received a tuition grant from Lincoln Land Community College, and therefore do not qualify for a reimbursement. Upon receipt of funding through the temporary budget, MEEA began to issue the tuition rebates to participants; at the time of the interview, approximately 20 participants had received their tuition rebate and MEEA was in the process of issuing the remaining rebates.

Program limited to core offering: MEEA staff reported that due to the issues discussed above, BOC program operations during EPY8/GPY5 were for the most part limited to core components including recruitment efforts and standard course offerings. The veteran component and certification maintenance support services provided by the program in prior years were not conducted during EPY8/GPY5. MEEA noted that although certification maintenance was not sponsored during the year, the fee for graduates is $65 and it is likely that this is not a significant financial barrier to the maintenance of the BOC certificate for most participants. The program operated a total of five courses during the year, three of which were Level I courses, and two of which were Level II courses.

Continued marketing efforts: MEEA continued to conduct marketing and recruitment activities during EPY8/GPY5. MEEA’s marketing strategy consists of a variety of outreach methods, including attending events and working through community colleges and other organizations to promote the program. MEEA noted that one aspect of program promotion is maintaining the public’s perception of BOC, and that the marketing and recruitment efforts helped to reassure prospective participants that the program was still available and would continue to be offered. According to MEEA staff, interest in the program remains high despite the challenges and decreased activity associated with EPY8/GPY5.

Program recovery: During the fall of 2016 MEEA learned that EEPS funding would be provided through a temporary budget. Upon receiving confirmation of this, MEEA increased its program recruitment and planning efforts in preparation for a more comprehensive and active program offering. MEEA staff noted that they expect program activity to recover and increase moving forward, and that the limitations introduced by the statewide budget delays appear to have been temporary. Specifically, MEEA staff noted that class sizes appear to be increasing and that they expect more courses to be offered during EPY9/GPY6.

Project tracking: MEEA staff stated that the program continues to make efforts to identify and track energy efficiency projects that are planned or implemented by BOC graduates. For example, participants complete course surveys that ask them to provide information regarding any upcoming projects in their facilities, and MEEA staff occasionally learn about projects through communications with past program graduates.

Addition of multifamily component: Regarding potential program improvements, MEEA staff reported that they have been assessing the extent to which BOC can provide additional support to multifamily building operators. MEEA has been working to identify



the unique building management characteristics associated with a continually operational residential facility, and plans to incorporate a multifamily component into BOC during the spring of 2017. This will be the first BOC course that targets multifamily building operators, and MEEA staff hopes that this will lead to a new major market segment of participants in future program years.


5. Conclusions and Recommendations

The following section presents a summary of key findings from the process and impact evaluations of the Building Operator Certification Program during electric program year eight and natural gas program year five (EPY8/GPY5). These conclusions and recommendations are based on a combination of research activities including participant surveys, supervisor surveys, interviews with program staff, and reviews of program tracking data, documentation, and prior evaluation reports.

Few of the measure identified in the survey qualified for net savings. As with prior program years, the savings estimation procedure determined that although participants reported implementing a wide range of projects after their participation in the BOC training, the total net savings impacts resulting from these projects were lower than may be expected based on the number of measures identified. Primary contributors to the limited net savings impacts include:

• Of the 28 measures reported by BOC participants in EPY8/GPY5, approximately 32% met the net attribution criteria specified by the Illinois TRM. This suggests that participants had plans to implement many of the measures prior to attending the BOC training, or that the BOC participants were not influential decision makers in the facilities’ decisions to implement energy efficiency projects. This does not suggest that the BOC program had no effect on the 68% of projects that did not meet the net attribution criteria; information gained through BOC courses may have improved participants’ adherence to best practices or improved the overall planning and quality assurance process, and participants reported that they found the program to be very valuable. However the evaluation results suggest that for a majority of reported projects, the associated energy savings would have been achieved in the absence of the training program.

• Of the 28 measures reported by BOC participants in EPY8/GPY5, 28% were associated with other incentive programs according to survey responses. While the BOC training was likely influential for many of these projects, the incentivized savings are claimable by the Department of Commerce and the utilities and cannot also be attributed to the BOC Program. This rate of external incentive receipt is lower than the prior program year rate of 56% of reported projects, but remains a factor in limiting energy savings attributable to BOC.

Participant and supervisor satisfaction is consistently high. As was found during prior program years, BOC graduates indicated a high level of satisfaction with all elements of their program experience and did not indicate any systematic or major issues with program structure, management, or operation. Respondents provided extremely high satisfaction ratings for all listed elements of their BOC Program experience, and only one respondent reported being at all dissatisfied with any element (the length of time to receive the rebate). Overall, the participant survey results suggest that program delivery has been very effective, and that there are few potential areas for improvement from the student perspective.

Conclusions and Recommendations 1


Statewide budget delay limited program operations: The number of program participants associated with Department of Commerce tuition rebates decreased from 117 individuals in EPY7/GPY4 to 53 individuals in EPY8/GPY5. As discussed by program staff, this was primarily because of operational challenges faced by MEEA throughout EPY8/GPY5 due to a delay in the approval of the Illinois statewide budget. As BOC provides tuition rebates that are funded by the Department of Commerce through the Illinois energy efficiency budget, this budget issue created uncertainty among program staff and participants regarding when, and to what extent, it would be possible to provide these incentives. MEEA continued to operate BOC training courses and informed participants that they would receive the tuition reimbursement as soon as funding became available. As budget delays continued, the BOC Program continued to operate throughout EPY8/GPY5 but held fewer training courses and limited program operations to core activities. The veteran component and certification maintenance support services provided by the program were not conducted during EPY8/GPY5 due to these budget-related resource issues. A temporary budget was ultimately passed, but this occurred after the end of EPY8/GPY5.

MEEA continued active public relations and marketing efforts. Despite the uncertainty resulting from the statewide budgetary issues, MEEA continued to conduct marketing and recruitment activities during EPY8/GPY5. MEEA’s marketing strategy consists of a variety of outreach methods, including attending events and working through community colleges and other organizations to promote the program. According to MEEA, these outreach efforts help the public to understand that the program is still available, and that program operations will likely recover and increase now that a temporary budget has been passed. Although EPY8/GPY5 experienced a significant decrease in program participants, MEEA staff explained that they expect this to be a temporary issue and that interest in the program is still high.

Overall, the evaluators found that while there were significant operational challenges during EPY8/GPY5, the Building Operator Certification Program has continued to deliver a valuable service and is well suited to providing up-to-date and actionable information to building managers. Although this did not translate to a high level of quantifiable net savings, feedback from participants and supervisors clearly demonstrates the organizational benefits that this training and certification can bring to a facility. Based on information gathered through the staff interview and participant and supervisor survey efforts, the evaluators provide the following recommendations for consideration moving forward:

Track and highlight potential gas projects: The EPY8/GPY5 evaluation did not identify any natural gas energy savings attributable to the BOC Program. While the sample sizes achieved for participant surveys across evaluation years may be a contributing factor, the EPY7/GPY4 evaluation also showed a lack of net attributable natural gas energy savings, and a high majority of reported projects are associated with electricity usage only. Although the end uses and building operations topics addressed by the BOC Program are fairly comprehensive and include subjects related to natural gas usage, it is difficult to quantify program benefits beyond what is identified through participant-reported data. In order to



assist in further highlighting the value of education provided through the BOC Program, the evaluators recommend that MEEA take note of any natural gas projects mentioned by current or past graduates, and track these projects so that they can be assessed and potentially attributed to the program.

Prepare students for possible EM&V outreach: During the participant survey effort, several respondents expressed concern regarding the purpose of the survey, and at least one participant was hesitant to provide project-related information to the evaluators due to concerns that the information would be used inappropriately. While the evaluators were able to explain the context of the survey and the reason for the evaluation, it appeared that participants may not have been aware that they would be contacted for this purpose. In order to assist in collecting feedback and project-related data from program participants, it may be useful to ensure that students are aware of the presence and purpose of EM&V, and that they may be contacted during the months following their graduation from the BOC Program.


Appendix A: Questionnaire for Participant Survey

Hello may I speak with [participant name]? My name is ___________and I am calling on behalf of the Midwestern Energy Efficiency Alliance and the Illinois Department of Commerce and Economic Opportunity (DCEO).

According to our records you completed building operator certificate training and received a tuition rebate.

Is that correct?( ) Yes( ) No (Thank and terminate)( ) Don’t know (Thank and terminate)

I would like to speak with you about your experience with that course. The survey should take about 20 minutes. Is this a good time to talk?

[If no, reschedule] [If refused, skip to end of survey and hit submit]

1. What are the sources your organization relies on for information about energy efficient practices, equipment, materials and design features? (Do not read list. Select all that apply.)

( ) DCEO representatives( ) The DCEO website( ) Utility representatives( ) The Midwestern Energy Efficiency Alliance (MEEA)( ) Brochures or advertisements( ) Trade associations or business groups you belong to( ) Trade journals or magazines( ) Friends and colleagues( ) The Smart Energy Design Assistance Center (SEDAC)( ) The Energy Resource Center (ERC)( ) Architects, engineers or energy consultants( ) Equipment vendors or building contractors( ) Other (please describe)( ) Don’t know

2. How did you learn about the Department of Commerce tuition rebate for the BOC training? (Do not read list. Select all that apply.)

( ) From a BOC program representative( ) A Midwestern Energy Efficiency Alliance (MEEA) representative( ) A DCEO representative mentioned it( ) The DCEO website( ) From a utility representative( ) Brochures or advertisements

Appendix A A-1


( ) Trade association or business group you belong to( ) Trade journal or magazine( ) Friend or colleague( ) From a representative of Smart Energy Design Assistance Center (SEDAC)( ) From a representative of the Energy Resource Center (ERC)( ) An architect, engineer or energy consultant( ) Equipment vendor or building contractor( ) Attended a conference workshop or seminar ( ) Past experience with the program ( ) An energy service company( ) Other (please describe)( ) Don’t know

3. When you learned about the tuition rebate available for the BOC courses, did you already know about the BOC training?

( ) Yes ( ) No( ) Don't know

4. Which of the following policies or procedures does your organization have in place regarding energy efficiency improvements at this facility? (Select all that apply)

( ) An energy management plan (If checked, go to 4A)( ) A staff member responsible for energy and energy efficiency( ) Policies that incorporate energy efficiency in operations and procurement( ) Active training of staff( ) Other (please specify)( ) Don’t know

4A. Does your energy management plan include goals for energy savings?( ) Yes (If checked, go to 4B)( ) No( ) Don't know

4B. Could you describe the goals specified in your energy management plan?

5. Were any of the courses you took through the BOC program particularly useful? ( ) Yes (If marked, go to 5A)( ) No( ) Don’t know

5A. Which ones and what made them useful?

6. Were there any courses that you found to not be very useful? ( ) Yes (If marked, go to 6A)( ) No( ) Don’t know

Appendix A A-2


6A. Which ones and what made them not very useful?

7. Why did you attend the BOC training? (Do not read list. Select all that apply.) (Use as prompts if necessary )

( ) Required by company/organization( ) To learn new job skills( ) To advance in my current job( ) To improve my chances of getting a new job( ) To earn continuing education credits( ) To learn about energy efficiency( ) Because of the tuition rebate( ) Other( ) Don’t know

8. Have you encountered any barriers to applying what you learned about energy efficiency improvements during the BOC training?( ) Yes (If checked, go to 8A)( ) No( ) Don’t know

8a. What barriers have you encountered? (Do not read list, but use as possible prompts)( ) Lack of supervisor support( ) Insufficient budget( ) Organization/company not committed to energy efficiency improvements( ) Not enough staff resources to plan efficiency projects( ) Other( ) Don’t know

9. What is the approximate square footage of your building or buildings?

10. What percentage of that space are you responsible for?

11. How many hours per week is your site open for business?

12. What type of facility is it? ( Do not read list)( ) College/University( ) Elementary( ) Grocery( ) Healthcare Clinic( ) Heavy Industry( ) High School/Middle School( ) Hospital( ) Hotel/Motel( ) Light Industry( ) Lodging Hotel/Motel( ) Manufacturing Facility

Appendix A A-3


( ) Medical( ) Office - High Rise( ) Office - Low Rise( ) Office - Mid Rise( ) Religious Facility( ) Restaurant( ) Retail - Department Store( ) Retail - Strip Mall( ) Retail/Service( ) School (K-12)( ) Warehouse( ) Other( ) Don’t know

13. Since participating in the BOC program have you implemented any of the following types of energy efficiency projects? (Ask follow up energy impact assessment questions for any project types indicated)

( ) Lighting Controls ( ) Energy efficient lighting( ) NEMA premium energy efficient motors( ) VSDs( ) Compressed air projects( ) Energy management systems( ) Heating system improvements( ) Air conditioning improvements( ) Economizer on an air handler( ) Water heating efficiency improvements( ) Other improvements( ) None- ( ) Don’t know

14. Is there somebody we can contact about the measures that may have been installed after attending the BOC course? Please provide a name, phone number, and email address.

15. At how many facilities did you implement any of the previously listed projects?( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7( ) 8( ) 9( ) 10 or more( ) Don’t know

Appendix A A-4


16. Is there somebody we can contact about the measures that may have been installed after attending the BOC course? Please provide a name, phone number, and email address.

Energy Impact Follow Up Questions

Lighting ControlsNet-to-GrossLC1A. The next few questions relate to the lighting controls you implemented. Were lighting

controls specifically recommended to you by a BOC course instructor or thorough BOC course materials?

( ) Yes( ) No( ) Don’t know

LC1B. How important was your experience in the Building Operator Certification Program in your decision to implement these lighting controls, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?

( ) 0 – Not at all important( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7( ) 8( ) 9( ) 10 – Extremely important

LC1C. Why do you give it this rating? (Please explain)

LC1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented these lighting controls, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?

( ) 0 – Definitely would not have implemented( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7( ) 8( ) 9( ) 10 – Definitely would have implemented

Appendix A A-5


[Display LC1E if LC1B response = 0,1,2,3 AND LC1D response = 0,1,2,3 OR LC1B response = 8,9,10 AND LC1D response = 8,9,10]

LC1E. You scored the importance of your program experience to your decision to implement the lighting controls with [answer from LC1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from LC1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationLC2. Had you implemented a similar project prior to attending the BOC training?

( ) Yes ( ) No( ) Don’t know

LC3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


LC4. What type of new lighting controls did you implement? ( ) Occupancy sensors (if checked, go to LC4A, LC4B, LC4C)( ) Day lighting controls (if checked, go to LC4D, LC4E, LC4F)( ) Don’t know

LC4A. How many fixtures are controlled by the occupancy sensors, what type of fixture are they, and what is the wattage of those fixtures?

Type of fixture Number of fixtures Wattage of fixtures12345678910

LC4B. How many hours per day did the lights controlled by the occupancy sensors operate before the controls were installed?

LC4C. Did the hours of operation for the lights change on weekends or holidays? If so, what were the operational hours during weekends or holidays?

Appendix A A-6


LC4D. How many fixtures are controlled by the daylighting sensors, what type of fixture are they, and what is the wattage of those fixtures?

Type of fixture Number of fixtures Wattage of fixtures12345678910

LC4E. How many hours per day did the lights controlled by the daylighting controls operate before the controls were installed?

LC4F. Did the hours of operation for the lights change on weekends or holidays? If so, what were the operational hours during weekends or holidays?

LC5 . What was the total estimated project cost for the lighting controls you installed? Please be as specific as possible.

Energy efficient lightingNet-to-GrossEEL1A. The next few questions relate to the energy efficient lighting you implemented. Was

energy efficient lighting specifically recommended to you by a BOC course instructor or thorough BOC course materials?


EEL1B. How important was your experience in the Building Operator Certification Program in your decision to implement this energy efficient lighting, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


Appendix A A-7


EEL1C. Why do you give it this rating? (Please explain)

EEL1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this energy efficient lighting, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


[Display EEL1E if EEL1B response = 0,1,2,3 AND EEL1D response = 0,1,2,3 OR EEL1B response = 8,9,10 AND EEL1D response = 8,9,10]

EEL1E. You scored the importance of your program experience to your decision to implement the energy efficient lighting with [answer from EEL1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from EEL1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationEEL2. How many lighting projects did you complete?

( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7( ) 8( ) 9( ) 10 or more ( ) Don’t know

EEL3. Had you implemented a similar project prior to attending the BOC training?( ) Yes ( ) No( ) Don’t know

Appendix A A-8


EEL4. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


EEL5. For the fixtures that were replaced in the (number of project) project, please indicate the type of fixture, number of fixtures, and wattage of those fixtures. (Repeat question for each facility which lighting projects were completed.)

Fixture Type Fixture Count Fixture WattageOld fixtureNew fixture

EEL6. How many hours per day are the lights operational?

EEL7.What was the total estimated project cost for the energy efficient lighting you installed?

Energy efficient motorsNet-to-GrossEEM1A. The next few questions relate to the energy efficient motors you implemented. Were

energy efficient motors specifically recommended to you by a BOC course instructor or through BOC course materials?


EEM1B. How important was your experience in the Building Operator Certification Program in your decision to implement this energy efficient motors project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


EEM1C. Why do you give it this rating? (Please explain)

EEM1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this energy efficient motors project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have

Appendix A A-9


implemented this measure and 10 means you definitely WOULD have implemented this measure?


[Display EEM1E if EEM1B response = 0,1,2,3 AND EEM1D response = 0,1,2,3 OR EEM1B response = 8,9,10 AND EEM1D response = 8,9,10]

EEM1E. You scored the importance of your program experience to your decision to implement the energy efficient motors with [answer from EEM1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from EEM1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationEEM2. Had you implemented a similar project prior to attending the BOC training?


EEM3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


EEM4. Were these motors installed for HVAC end-uses or for industrial end-uses? (Select all that apply)

( ) HVAC end-use (if selected, go to EEM4A,)( ) Industrial end-use (if selected, go to EEM4B)( ) Don’t know

EEM4A. Thinking about one of the motors you installed, please provide the motor application (hot water pump, chilled water pump, supply fan, return fan, or cooling tower fan), efficiency of the motor, horsepower of the motor, and whether or not VSD’s control the motor. Additionally, please state how many motors you installed that have these same specifications. (After respondent provides this information, ask whether they installed any

Appendix A A-10


additional motors with different specifications. Then, place the additional data in a separate motor group and repeat the data collection procedure until all motors or sets of motors have been described.)

(If the respondent is unable to provide the information on the project specifics, ask if the interview could be rescheduled at a time when the respondent could provide the information, or if there is someone else to speak to who was knowledgeable about the projects.)

Efficiency of Motors

Motor application

Horsepower of motors

VSD’s (“y” for yes / “n” for no)

Number installed

(Motor Group 1)(Motor Group 2)(Motor Group 3)(Motor Group 4)(Motor Group 5)

EEM4B. Thinking about one of the motors you installed for industrial purposes, please provide the motor application (hot water pump, chilled water pump, supply fan, return fan, or cooling tower fan), efficiency of the motor, horsepower of the motor, and whether or not VSD’s control the motor. Additionally, please state how many motors you installed that have these same specifications. (After respondent provides this information, ask whether they installed any additional motors with different specifications. Then, place the additional data in a separate motor group and repeat the data collection procedure until all motors or sets of motors have been described.)

(If the respondent is unable to provide the information on the project specifics, ask if the interview could be rescheduled at a time when the respondent could provide the information, or if there is someone else to speak to who was knowledgeable about the projects.)

Efficiency of Motors

Number installed

Motor application

Horsepower of motors

Hours per day of operation

VSD’s (“y” for yes / “n” for no)

(Motor Group 1)(Motor Group 2)(Motor Group 3)(Motor Group 4)(Motor Group 5)

EEM5. What was the total estimated project cost for the efficient motors you installed? Please be as specific as possible.

Appendix A A-11


EEM6. Who can we contact about the technical specifics of the energy efficient motors you installed? Please be as specific as possible.

VSDsNet-to-GrossVSD1A. The next few questions relate to the VSD project you implemented. Were VSDs

specifically recommended to you by a BOC course instructor or through BOC course materials?


VSD1B. How important was your experience in the Building Operator Certification Program in your decision to implement this VSD project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


VSD1C. Why do you give it this rating? (Please explain)

VSD1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this VSD project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


Appendix A A-12


[Display VSD1E if VSD1B response = 0,1,2,3 AND VSD1D response = 0,1,2,3 OR VSD1B response = 8,9,10 AND VSD1D response = 8,9,10]

VSD1E. You scored the importance of your program experience to your decision to implement the VSD project with [answer from VSD1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from VSD1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationVSD2. Had you implemented a similar project prior to attending the BOC training?


VSD3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


VSD4. Were the VSDs installed on existing motors part of an HVAC system?( ) Yes (if selected, go to VSD4A)( ) No (if selected, go to VSD4B)( ) Some were part of an HVAC system, some were not (if selected, go to VSD4A and

VSD4B)( ) Don’t know

VSD4A. For each of the VSDs used in a HVAC system, please provide the number of VSDs installed and the horsepower of the motors controlled.

Motor Application Number of VSDs Installed

Horsepower of Motors Controlled by VSDs

Hot Water PumpChilled Water PumpSupply Fan: Constant VolumeSupply Fan: Air Foil/inlet Guide VanesSupply Fan: Forward Curved Fan, with discharge dampersSupply Fan: Forward Curved Inlet Guide VanesCooling Tower FanCustom Process

VSD4B. For the existing motors not used in a HVAC system, what is the total number of motors and total motor horsepower controlled by the VSDs?

Number of motors:Individual motor horsepower:

Appendix A A-13


Operation hours:Motor efficiency:

VSD5. What was the total estimated project cost for the VSD’s you installed? Please be as specific as possible.

VSD6. Who can we contact about the technical specifics of the VSD installation if needed? Please provide a name, phone number, and email address.

Compressed air projectsNet-to-GrossCA1A. The next few questions relate to the compressed air project you implemented. Were

compressed air projects specifically recommended to you by a BOC course instructor or through BOC course materials?


CA1B. How important was your experience in the Building Operator Certification Program in your decision to implement this compressed air project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


CA1C. Why do you give it this rating? (Please explain)

CA1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this compressed air project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?

( ) 0 – Definitely would not have implemented( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7

Appendix A A-14


( ) 8( ) 9( ) 10 – Definitely would have implemented

[Display CA1E if CA1B response = 0,1,2,3 AND CA1D response = 0,1,2,3 OR CA1B response = 8,9,10 AND CA1D response = 8,9,10]

CA1E. You scored the importance of your program experience to your decision to implement the compressed air project with [answer from CA1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from CA1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationCA2. Had you implemented a similar project prior to attending the BOC training?


CA3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


CA4. What is the horsepower of each air compressor in the system?

Horsepower VSD? (yes or no)Compressor 1Compressor 2Compressor 3Compressor 4Compressor 5

CA5. What kind of compressed air project did you implement? (Do not read list.) (Select all that apply.) (For each response selected, follow up with CA6.)

( ) New high efficiency single-speed compressor ( ) New high efficiency variable-speed compressor ( ) New efficient refrigerated air dryer ( ) New efficient desiccant air dryer ( ) Improved staging controls ( ) Other (Please specify type of compressed air equipment and quantity of units)( ) Don’t know

CA6. For the new high efficiency (type of air compressor) compressor, what is the total number of compressors and horsepower of each new compressor? (Repeated for each compressed

Appendix A A-15


air project selected in CA5.)

Number of new compressors:Horsepower for each new compressor:

CA7. What type of other air compressor project did you implement? Please describe the equipment and quantity of units.

CA8. Who can we contact about the technical specifics of the compressed air project(s)? Please provide name, phone number, and email address. Not in survey

Energy management systemsNet to GrossEMS1A. The next few questions relate to the energy management system project you

implemented. Were energy management system projects specifically recommended to you by a BOC course instructor or through BOC course materials?


EMS1B. How important was your experience in the Building Operator Certification Program in your decision to implement this energy management system project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


EMS1C. Why do you give it this rating? (Please explain)

EMS1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this energy management system project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?

( ) 0 – Definitely would not have implemented( ) 1( ) 2( ) 3( ) 4

Appendix A A-16


( ) 5( ) 6( ) 7( ) 8( ) 9( ) 10 – Definitely would have implemented

[Display EMS1E if EMS1B response = 0,1,2,3 AND EMS1D response = 0,1,2,3 OR EMS1B response = 8,9,10 AND EMS1D response = 8,9,10]

EMS1E. You scored the importance of your program experience to your decision to implement the energy management system project with [answer from EMS1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from EMS1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationEMS2. Had you implemented a similar project prior to attending the BOC training?


EMS3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


EMS4. What is the square footage of the area that the Energy Management System controls?

EMS5. Did you install a new energy management system after the BOC training?( ) Yes( ) No( ) Don’t know

EMS6. Did you make changes to an existing energy management system after the BOC training?( ) Yes( ) No( ) Don’t know

EMS7. Please describe the function of the Energy Management System? (Do not read list. Select all that apply.)

( ) On and off schedule( ) Does everything( ) Cooling plant optimization( ) Cooling distribution optimization

Appendix A A-17


( ) Outdoor air ventilation (economizer)( ) Outdoor air ventilation (demand controlled ventilation with CO sensor)( ) Air distribution optimization( ) Heating plant and distribution optimization( ) Other (Please specify all other)( ) Don’t know

EMS8. What was the total estimated project cost for the energy management system you installed? Please be as specific as possible.

EMS9. Who can we contact about the technical specifics of the energy management system project(s)? Please provide name, phone number, and email address.

Heating system improvementsNet to GrossHS1A. The next few questions relate to the heating system project you implemented. Were

heating system projects specifically recommended to you by a BOC course instructor or through BOC course materials?


HS1B. How important was your experience in the Building Operator Certification Program in your decision to implement this heating system project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


HS1C. Why do you give it this rating? (Please explain)

HS1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this heating system project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?

( ) 0 – Definitely would not have implemented( ) 1( ) 2

Appendix A A-18


( ) 3( ) 4( ) 5( ) 6( ) 7( ) 8( ) 9( ) 10 – Definitely would have implemented

[Display HS1E if HS1B response = 0,1,2,3 AND HS1D response = 0,1,2,3 OR HS1B response = 8,9,10 AND HS1D response = 8,9,10]

HS1E. You scored the importance of your program experience to your decision to implement the heating system project with [answer from HS1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from HS1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationHS2. Had you implemented a similar project prior to attending the BOC training?


HS3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


HS4. What is the primary heating system type for the system you made improvements to? (Do not read list) (Select all that apply)

( ) Hot air furnace( ) Wall or floorboard radiator (steam, hot water or electric resistance)( ) Steam, hot water or electric resistance coils in ventilation system( ) Space heaters( ) Heat pump (air source)( ) Heat pump (ground source)( ) Heat pump (water loop)( ) Electric boiler( ) Gas boiler( ) Other (Please specify)( ) Don’t know

Appendix A A-19


HS5. What type of heating system improvements that produce energy savings did you implement? (Do not read list) (Select all that apply) (For each response selected, follow up with HS8.)

( ) Installed a heat recovery system ( ) Installed a furnace( ) Installed a high efficiency boiler ( ) Installed a high efficiency low turn-down burner ( ) Installed oxygen trim control ( ) Other (Please describe the type and quantity of equipment installed) ( ) Don’t know

HS6. What is the primary fuel source for heating? (Do not read list)( ) Electric( ) Gas( ) Oil( ) Purchased steam( ) Other (Please specify) ( ) Don’t know

HS7. What kind of heating system efficiency improvements did you make? Please include as many details about capacity, efficiency, and quantity as possible.

HS8. What is the capacity and efficiency of the (improvement type) installed? (Repeated for each project selected in HS5.)

Capacity (BTU):Efficiency level (AFUE):

HS9. What was the total estimated project cost for the energy efficient heating system you installed? Please be as specific as possible.

HS10. Who can we contact about the technical specifics of the heating system project(s)? Please provide name, phone number, and email address.

Air conditioning improvementsNet to GrossAC1A. The next few questions relate to the air conditioning project you implemented. Were air

conditioning projects specifically recommended to you by a BOC course instructor or through BOC course materials?


AC1B. How important was your experience in the Building Operator Certification Program in your decision to implement this air conditioning project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?

( ) 0 – Not at all important

Appendix A A-20


( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7( ) 8( ) 9( ) 10 – Extremely important

AC1C. Why do you give it this rating? (Please explain)

AC1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this air conditioning project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


[Display AC1E if AC1B response = 0,1,2,3 AND AC1D response = 0,1,2,3 OR AC1B response = 8,9,10 AND AC1D response = 8,9,10]

AC1E. You scored the importance of your program experience to your decision to implement the air conditioning project with [answer from AC1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from AC1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationAC2. Had you implemented a similar project prior to attending the BOC training?


AC3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?

Appendix A A-21



AC4. What is the primary cooling system type for the system you made improvements to? (Do not read list) (Select all that apply)

( ) Chiller – air-cooled ( ) Chiller – water or evaporatively cooled ( ) Evaporative cooler ( ) Fans( ) Direct expansion – air-cooled packaged or split system cooling or heat pump( ) Geothermal heat pump( ) Window or thermal units (PTAC/PTHP)( ) Other (Please specify)( ) Don’t know

AC5. What type of air conditioning improvements that produced energy savings did you implement? (Do not read list.) (Select all that apply)

( ) Installed new high-efficiency chiller(s) (Go to AC7)( ) Installed new terminal unit(s) (Go to AC8)( ) Installed heat pump(s) (Go to AC9)( ) Installed package unit(s) (Go to AC10)( ) Installed split system(s) (Go to AC11)( ) Other (Please describe the type and quantity of equipment installed)( ) Don’t know

AC6. What kind of air conditioning improvements did you make that were not listed above?

AC7. What is the capacity and efficiency level of the chiller(s) you installed?

Capacity Efficiency levelImprovement type 1Improvement type 2Improvement type 3

AC8. What is the capacity and efficiency level of the terminal unit(s) you installed?

Number of units Capacity (tons)

Efficiency level (EER)

Efficiency level (SEER)

Unit(s)

AC9. What type of heat pump did you install? (Do not read list) (Use as possible prompts)( ) Air cooled heat pump( ) Water source heat pump( ) Ground source heat pump( ) Water cooled heat pump( ) Don’t know

Appendix A A-22


AC9A. What is the capacity and efficiency level of the chiller(s) you installed?

Number of heat pumps

Efficiency level (EER or SEER or HSPF or COP)

Capacity of: Less than 1 1/2 tonsCapacity of: 1 1/2 tons to 2 1/2 tonsCapacity of: More than 2 1/2 tons to 5 tonsCapacity of: More than 5 tons to 11 tonsCapacity of: More than 11 tons to less than 20 tonsCapacity of: More than 20 tons

AC10. What is the capacity and efficiency level of the package unit(s) you installed?

Number of units Efficiency level (EER or SEER)


AC11. What is the capacity and efficiency level of the split system you installed?

Number of units Efficiency level (EER or SEER)


AC12. What was the total estimated project cost for the air conditioning improvements you installed? Please be as specific as possible.

AC13. Who can we contact about the technical specifics of the air-conditioning system project(s)? Please provide name, phone number, and email address

Economizers on Air HandlersNet to Gross

Appendix A A-23


E1A. The next few questions relate to the economizer project you implemented. Were economizer projects specifically recommended to you by a BOC course instructor or through BOC course materials?


E1B. How important was your experience in the Building Operator Certification Program in your decision to implement this economizer project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


E1C. Why do you give it this rating? (Please explain)

E1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this economizer project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


[Display E1E if E1B response = 0,1,2,3 AND E1D response = 0,1,2,3 OR E1B response = 8,9,10 AND E1D response = 8,9,10]

E1E. You scored the importance of your program experience to your decision to implement the economizer project with [answer from E1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program

Appendix A A-24


with [answer from E1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationE2. Had you implemented a similar project prior to attending the BOC training?


E3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


E4. For the installed economizer, what is the capacity of the cooling system (in tons)?

E5. What is the total estimated project cost for the economizer you installed?

E6. Who can we contact about the technical specifics of the economizer project(s)? Please provide name, phone number, and email address

Appendix A A-25


Water heating efficiency improvementsNet to GrossWH1A. The next few questions relate to the water heating project you implemented. Were water

heating projects specifically recommended to you by a BOC course instructor or through BOC course materials?


WH1B. How important was your experience in the Building Operator Certification Program in your decision to implement this water heating project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


WH1C. Why do you give it this rating? (Please explain)

WH1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this water heating project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


[Display WH1E if WH1B response = 0,1,2,3 AND WH1D response = 0,1,2,3 OR WH1B response = 8,9,10 AND WH1D response = 8,9,10]

Appendix A A-26


WH1E. You scored the importance of your program experience to your decision to implement the water heating project with [answer from WH1B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from WH1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationWH2. Had you implemented a similar project prior to attending the BOC training?


WH3. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


WH4. What type of water heating improvements that produced energy savings did you implement? (Do not read list. Select all that apply)

( ) Installed timeclock to turn off circulation pump after hours (Go to WH5 and WH6)( ) Installed heat recovery system (Go to WH8)( ) Installed a more efficient hot water heater (Go to WH7)( ) Insulated pipes(s) (How thick was the insulation and how many linear feet were

installed?)( ) Installed low-flow faucets, pre-rinse spray valves, or low-flow showerheads (Go to

WH10)( ) Other (Go to WH8)( ) Don’t know

WH5. What kind of water heating system is controlled by the timeclock?( ) Boiler( ) Hot water heater( ) Don’t know

WH6. What is the capacity and efficiency level of the boiler that the timeclock is installed on?

Capacity (BTU) Efficiency level (AFUE %)Boiler

WH7. What is the capacity, number, and efficiency level of the more efficient hot water heater or boiler?

Appendix A A-27


Capacity Type New efficiency Old efficiency QuantityHeater/BoilerHeater/BoilerHeater/BoilerHeater/BoilerHeater/Boiler

WH8. Please describe the water heating improvements that produced energy savings including the type of equipment and quantity.

WH9. What was the total estimated project cost for the water heating improvements you installed? Please be as specific as possible.

WH10. Please indicate the quantity of low-flow faucets, pre-rinse spray valves, and low-flow showerheads you installed:

Measure Type Quantity Installed

Low-flow faucets

Low-flow showerheads

Pre-rinse spray valves

Other improvementsO1. You mentioned that you implemented some other energy efficiency projects. Can you

describe what these projects were?

Net to GrossO2A. The next few questions relate to the other project you implemented. Was this other project

specifically recommended to you by a BOC course instructor or through BOC course materials?


O2B. How important was your experience in the Building Operator Certification Program in your decision to implement this other project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?

( ) 0 – Not at all important( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7

Appendix A A-28


( ) 8( ) 9( ) 10 – Extremely important

O2C. Why do you give it this rating? (Please explain)

O2D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this other project, using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


[Display O2E if O2B response = 0,1,2,3 AND O2D response = 0,1,2,3 OR O2B response = 8,9,10 AND O2D response = 8,9,10]

O2E. You scored the importance of your program experience to your decision to implement the other project with [answer from O2B] out of 10 possible points. You also scored the likelihood of implementing this measure if you had not participated in the program with [answer from O2D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this measure?

Measure InformationO3. Had you implemented a similar project(s) prior to attending the BOC training?


O4. Have you received or applied for a financial incentive from a utility or the Illinois DCEO for this project?


O5. Did these projects produce electricity, or natural gas savings or both?( ) Electricity( ) Natural gas

Appendix A A-29


( ) Both( ) Don’t know

O6. What was the total estimated project cost for the other energy efficiency improvements you installed? Please be as specific as possible.

O7. Who can we contact about the technical specifics of these other project(s)? Please provide name, phone number, and email address

Maintenance Energy Impacts Assessment

17. Now I’d like to ask you about changes in maintenance activities you may have implemented at your facility since completing the BOC training. For each of the following activities, please indicate if you have performed them differently or more frequently or both since participating in the BOC training.

Differently More Frequently Both No

ChanceDon’t Know

Maintenance on the cooling system equipment?

( ) ( ) ( ) ( ) ( )

Maintenance on the heating equipment? ( ) ( ) ( ) ( ) ( )

Motor maintenance, including belt alignment and tension?

( ) ( ) ( ) ( ) ( )

Maintenance on compressed air system? ( ) ( ) ( ) ( ) ( )

Electrical panel maintenance? ( ) ( ) ( ) ( ) ( )

Ventilation maintenance? ( ) ( ) ( ) ( ) ( )

Other energy savings maintenance? ( ) ( ) ( ) ( ) ( )

[If maintenance is performed differently, more frequently, or both, for any category go to M1 and ask selected Maintenance Improvement Questions]

18. At how many facilities did you make these changes to your maintenance practices?( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7

Appendix A A-30


( ) 8( ) 9( ) 10 or more( ) Don’t know

Follow Questions for Maintenance ImprovementsCooling system

Net to GrossCSM1A. The next few questions relate to the cooling system maintenance you implemented.

Was cooling system maintenance specifically recommended to you by a BOC course instructor or through BOC course materials?


CSM1B. How important was your experience in the Building Operator Certification Program in your decision to implement this cooling system maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


CSM1C. Why do you give it this rating? (Please explain)

CSM1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this cooling system maintenance using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this maintenance and 10 means you definitely WOULD have implemented this maintenance?

( ) 0 – Definitely would not have implemented( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7( ) 8

Appendix A A-31


( ) 9( ) 10 – Definitely would have implemented

[Display CSM1E if CSM1B response = 0,1,2,3 AND CSM1D response = 0,1,2,3 OR CSM1B response = 8,9,10 AND CSM1D response = 8,9,10]

CSM1E. You scored the importance of your program experience to your decision to implement the cooling system maintenance with [answer from CSM1B] out of 10 possible points. You also scored the likelihood of implementing this maintenance if you had not participated in the program with [answer from CSM1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this maintenance?

Maintenance InformationCSM2. Please tell me which of the following changes you’ve made to your cooling system

maintenance practices? ( ) Changes to cooling tower service (please include total tons)( ) Changes to chiller bundle cleaning (please include chiller tons)( ) Changes to condenser cleaning (please include cooling tons)( ) Changes to refrigerant charge adjustment (please include system tons) ( ) Other changes ( ) Don't know

CSM3. Who can we contact about the technical specifics of the cooling system maintenance practices if needed? Please provide a name, phone number, and email address.

Heating equipmentNet to GrossHE1A. The next few questions relate to the heating system maintenance you implemented. Was

heating equipment maintenance specifically recommended to you by a BOC course instructor or through BOC course materials?


HE1B. How important was your experience in the Building Operator Certification Program in your decision to implement this heating system maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?

( ) 0 – Not at all important( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7

Appendix A A-32


( ) 8( ) 9( ) 10 – Extremely important

HE1C. Why do you give it this rating? (Please explain)

HE1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this heating system maintenance using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this maintenance and 10 means you definitely WOULD have implemented this maintenance?


[Display HE1E if HE1B response = 0,1,2,3 AND HE1D response = 0,1,2,3 OR HE1B response = 8,9,10 AND HE1D response = 8,9,10]

HE1E. You scored the importance of your program experience to your decision to implement the heating system maintenance with [answer from HE1B] out of 10 possible points. You also scored the likelihood of implementing this maintenance if you had not participated in the program with [answer from HE1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this maintenance?

Maintenance InformationHE2. Please tell me which of the following changes you've made to your heating

equipment maintenance practices. (Select all that apply)( ) Heat exchanger cleaning (Please provide capacity in BTU’s)* no box to provide

info( ) Blowdown frequency (Please provide boiler capacity in BTU’s and number of

traps) )* no box to provide info

( ) Steam trap (Please provide number of traps and whether they were cleaned, repaired, or replaced))

( ) Other ( ) Don't know

Appendix A A-33


HE3. Please ask for additional information for each change indicated above, such as how frequently the maintenance was performed before and after the course. Not in

online survey

HE4. Who can we contact about the technical specifics of the heating system maintenance practices if needed? Please provide a name, phone number, and email address.

Air Compressor MaintenanceNet to GrossAC1A. The next few questions relate to the air compressor maintenance you implemented. Was

air compressor maintenance specifically recommended to you by a BOC course instructor or through BOC course materials?


AC1B. How important was your experience in the Building Operator Certification Program in your decision to implement this air compressor maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


AC1C. Why do you give it this rating? (Please explain)

AC1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this air compressor maintenance using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this maintenance and 10 means you definitely WOULD have implemented this maintenance?

( ) 0 – Definitely would not have implemented( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7

Appendix A A-34


( ) 8( ) 9( ) 10 – Definitely would have implemented

[Display AC1E if AC1B response = 0,1,2,3 AND AC1D response = 0,1,2,3 OR AC1B response = 8,9,10 AND AC1D response = 8,9,10]

AC1E. You scored the importance of your program experience to your decision to implement the air compressor maintenance with [answer from AC1B] out of 10 possible points. You also scored the likelihood of implementing this maintenance if you had not participated in the program with [answer from AC1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this maintenance?

Maintenance InformationAC2. Please tell me all the changes you have made to your air compressor equipment

maintenance. ( ) Audible leak detection( ) Ultra-sonic leak detection( ) Pressure optimization( ) End-use isolation( ) Filter changes( ) System diagnostics( ) Other ( ) Don't know

AC3. What is the total horsepower of the air compressor(s)?

AC4. What is the average CFM (Cubic Feet Per Minute) of the air compressor(s)? AC5. Who can we contact about the technical specifics of the air compressor maintenance

changes if needed? Please provide a name, phone number, and email address.

Ventilation MaintenanceNet to Gross

VM1A. The next few questions relate to the ventilation maintenance you implemented. Was ventilation maintenance specifically recommended to you by a BOC course instructor or through BOC course materials?


VM1B. How important was your experience in the Building Operator Certification Program in your decision to implement this ventilation maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?

( ) 0 – Not at all important( ) 1

Appendix A A-35


( ) 2( ) 3( ) 4( ) 5( ) 6( ) 7( ) 8( ) 9( ) 10 – Extremely important

VM1C. Why do you give it this rating? (Please explain)

VM1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this ventilation maintenance using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this maintenance and 10 means you definitely WOULD have implemented this maintenance?


[Display VM1E if VM1B response = 0,1,2,3 AND VM1D response = 0,1,2,3 OR VM1B response = 8,9,10 AND VM1D response = 8,9,10]

VM1E. You scored the importance of your program experience to your decision to implement the ventilation maintenance with [answer from VM1B] out of 10 possible points. You also scored the likelihood of implementing this maintenance if you had not participated in the program with [answer from VM1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this maintenance?

Maintenance InformationVM2. Please tell me all the changes you have made to your ventilation maintenance. (For

each change mentioned, ask how frequently they do this). ( ) Economizer optimization/repair( ) Sensor Calibration( ) Setpoint optimization( ) Balancing( ) Filter changes( ) System diagnostics

Appendix A A-36


( ) Sealed leaks / replaced door gaskets( ) Other ( ) Don't know

VM3. What is the total horsepower of the serviced fans?

VM4. Who can we contact about the technical specifics of the ventilation maintenance practices if needed? Please provide a name, phone number, and email address.

Other MaintenanceOM1A. The next few questions relate to the other maintenance you implemented. Was this other

maintenance specifically recommended to you by a BOC course instructor or through BOC course materials?


OM1B. How important was your experience in the Building Operator Certification Program in your decision to implement this other maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


OM1C. Why do you give it this rating? (Please explain)

OM1D. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this other maintenance using a scale of 0 to 10 where 0 means you definitely WOULD NOT have implemented this maintenance and 10 means you definitely WOULD have implemented this maintenance?

( ) 0 – Definitely would not have implemented( ) 1( ) 2( ) 3( ) 4( ) 5( ) 6

Appendix A A-37


( ) 7( ) 8( ) 9( ) 10 – Definitely would have implemented

[Display OM1E if OM1B response = 0,1,2,3 AND OM1D response = 0,1,2,3 OR OM1B response = 8,9,10 AND OM1D response = 8,9,10]

OM1E. You scored the importance of your program experience to your decision to implement the other maintenance with [answer from OM1B] out of 10 possible points. You also scored the likelihood of implementing this maintenance if you had not participated in the program with [answer from OM1D] out of 10 possible points. Can you please explain the role the program had in your decision to implement this maintenance?

Maintenance InformationOM2. Please describe the other maintenance changes that you have made since attending

the BOC training. [If needed, prompt with please describe the change in practice and how frequently it is performed]

OM3. Who can we contact about the technical specifics of the other maintenance practices if needed? Please provide a name, phone number, and email address.

Now I would like to ask a few general questions about your experience with the program.

19. Do you think that there are certain barriers that may make it difficult for potential program participants to attend or complete the BOC training? What are they? (Don’t read list. Select all that apply.)

( ) Time( ) Cost( ) Not aware of it( ) Supervisor approval( ) No barriers( ) Don’t know

20. Please indicate your level of satisfaction with the following elements of the BOC training.

Element of Experience Very Dissatisfied Dissatisfied Neither Satisfied

nor Dissatisfied Satisfied Very Satisfied

Don’t know

Course schedule ( ) ( ) ( ) ( ) ( ) ( )

Course instructors ( ) ( ) ( ) ( ) ( ) ( )

Tuition rebate application process

( ) ( ) ( ) ( ) ( ) ( )

Tuition rebate amount ( ) ( ) ( ) ( ) ( ) ( )

Time elapsed to receive tuition rebate

( ) ( ) ( ) ( ) ( ) ( )

Appendix A A-38


Overall experience with the BOC Program

( ) ( ) ( ) ( ) ( ) ( )

20A. Please describe the ways in which you were not satisfied with the BOC training or the tuition rebate.

21. For each of the following activities, please indicate if you had already completed them prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed them:

ActivityCompleted Prior to Training

Completed Before and After Training

Only Completed After Training

Not Yet Completed Them

Don’t know

Implemented an energy budget ( ) ( ) ( ) ( ) ( )

Recorded energy use over time ( ) ( ) ( ) ( ) ( )

Set energy savings goals ( ) ( ) ( ) ( ) ( )Achieved energy savings goals ( ) ( ) ( ) ( ) ( )

22. Have you participated in any other DCEO energy efficiency programs?( ) Yes (if checked, go to 22A)( ) No( ) Don’t know

22A. What other DCEO energy efficiency programs did you participate in?

23B. How important was the BOC course in your decision to participate in these other DCEO programs? (Read list)

( ) Very important( ) Somewhat important( ) Neutral( ) Somewhat unimportant( ) Not important at all( ) Don’t know/Not applicable

23. What is your current job title? (Do not read list)( ) Operations/Facilities operations manager( ) Maintenance manager ( ) HVAC supervisor or technician( ) Engineering manager( ) Facilities manager( ) Engineer( ) Maintenance manager( ) General contractor( ) Building management specialist

Appendix A A-39


( ) Other engineering position( ) Other manager, team leader, supervisor

24. How many years have you worked in this role?

25. How many building operator staff is there at your current location?

26. How many of these staff has completed the BOC training (either Level 1 or Level 1&2)?

27. Does your facility currently have plans to implement any of the following types of energy efficiency projects? (Select all that apply)

( ) Lighting Controls ( ) Energy efficient lighting( ) NEMA premium energy efficient motors( ) VSDs( ) Compressed air projects( ) Energy management systems( ) Heating system improvements( ) Air conditioning improvements( ) Economizer on an air handler( ) Water heating efficiency improvements( ) Other improvements( ) None( ) Don’t know

[Ask Q28 if any projects indicated in Q28]28. In what month and year do you expect the aforementioned projects to be implemented?

29. Did you initiate the plans for these upcoming projects, or did someone else in your organization?

( ) I initiated the plans [If selected, ask Q30]( ) Someone else initiated the plans ( ) Don’t know

30. Did you initiate the plans for these upcoming projects due to information you gained through Building Operator Certification training?

( ) Yes( ) No ( ) Don’t know

31. Do you have any other comments or feedback that you would like to provide regarding the Building Operator Certification Program? (Please explain)

Thank you for taking this survey of participants in the building operator certification program.Your response is very important to us.

Appendix A A-40

Appendix B: Participant Survey ResponsesAs part of the evaluation effort, a telephone survey was administered to Building Operator Certification training participants who completed the MEEA program. This survey provided the information used in Chapter 3 to estimate free ridership and potential savings for projects in the BOC Program. Additionally the survey gathered information pertaining to the program process evaluation.

Each participant was interviewed using the survey instrument provided in Appendix A. During the interview, a participant was asked questions about (1) his or her general decision making regarding purchasing and installing energy efficient equipment, (2) his or her knowledge of and satisfaction with the BOC Program, and (3) the influence that the BOC Program had on his or her decision to install energy efficiency measures (e.g., lighting measures, HVAC measures, maintenance and operation improvements).

The following tabulations summarize participant survey responses. Three columns of data are presented. The first column presents the number of survey respondents (n) associated with each response. The second column presents the percentage of survey respondents associated with each response.

Appendix B B-1


What are the sources your organization relies on for information about energy efficient practices, equipment, materials and design features? (Do not read list. Select all that apply.)

Response (n=13)Percent of

Respondents*

DCEO representatives 0 0%The DCEO website 0 0%Utility representatives 3 23%The Midwestern Energy Efficiency Alliance (MEEA) 1 8%

Brochures or advertisements 1 8%Trade associations or business groups you belong to 2 15%

Trade journals or magazines 3 23%Friends and colleagues 2 15%The Smart Energy Design Assistance Center (SEDAC) 0 0%

The Energy Resource Center (ERC) 0 0%Architects, engineers or energy consultants 4 31%Equipment vendors or building contractors 6 46%Other (please describe) 5 38%Don't know 1 8%


How did you learn about the DCEO tuition rebate for the BOC training?


Respondents*

From a BOC program representative 4 31%A Midwestern Energy Efficiency Alliance (MEEA) representative 1 8%

A DCEO representative mentioned it 0 0%The DCEO website 0 0%From a utility representative 2 15%Brochures or advertisements 0 0%Trade association or business group you belong to 0 0%

Trade journal or magazine 0 0%Friend or colleague 2 15%From a representative of Smart Energy Design Assistance Center (SEDAC) 0 0%

From a representative of the Energy Resource Center (ERC) 0 0%

An architect, engineer or energy consultant 0 0%Equipment vendor or building contractor 1 8%Attended a conference workshop or seminar 0 0%Past experience with the program 0 0%An energy service company 0 0%Other (please explain) 3 23%Don't know 2 15%

Appendix B B-2



When you learned about the tuition rebate available for the BOC courses, did you already know about the BOC training?

Response (n=11) Percent of Respondents

Yes 8 73%No 3 27%Don't know 0 0%

Which of the following policies or procedures does your organization have in place regarding energy efficiency improvements?


Respondents*

An energy management plan 5 36%A staff member responsible for energy and energy efficiency 7 50%

Policies that incorporate energy efficiency in operations and procurement 5 36%

Active training of staff 5 36%Other (please specify) 1 7%Don't know 3 21%


Does your energy management plan include goals for energy savings?



Were any of the courses you took through the BOC program particularly useful?


Yes 12 100%No 0 0%Don't know 0 0%

Were there any courses that you found to not be very useful?


Yes 0 0%No 12 100%Don't know 0 0%

Appendix B B-3


Why did you attend the BOC training? (Do not read list. Select all that apply.) (Use as prompts if necessary )


Respondents*

Required by company/organization 2 15%To learn new job skills 10 77%To advance in my current job 6 46%To improve my chances of getting a new job 1 8%To earn continuing education credits 4 31%To learn about energy efficiency 6 46%Because of the tuition rebate 2 15%Other (please specify) 4 31%Don't know 0 0%


Have you encountered any barriers to applying what you learned about energy efficiency improvements during the BOC training?



What barriers have you encountered? (Do not read list, but use as possible prompts)


Lack of supervisor support 0 0%Insufficient budget 2 50%Organization/company not committed to energy efficiency improvements 1 25%

Not enough staff resources to plan efficiency projects 1 25%

Other (please specify) 1 25%Don't know 0 0%


What is the approximate square footage of your building or buildings?

Response (n=8)Average square footage

Average square footage 673,875

What percentage of that space are you responsible for?

Response(n=10) Percent

responsiblePercent responsible 100.0

How many hours per week is your site open for business?

Response (n=11) Hours per week

Hours per week 127.7

Appendix B B-4




College/University 2 15%

Elementary 1 8%

Grocery 0 0%Healthcare Clinic 1 8%Heavy Industry 0 0%High School/Middle School 0 0%Hospital 0 0%Hotel/Motel 0 0%Light Industry 0 0%Lodging Hotel/Motel 1 8%Manufacturing Facility 1 8%Medical 0 0%Office - High Rise 2 15%Office - Low Rise 0 0%Office - Mid Rise| 1 8%Religious Facility 1 8%Restaurant 0 0%Retail - Department Store 0 0%Retail - Strip Mall 0 0%Retail/Service 0 0%School (K-12) 0 0%Warehouse 0 0%Other (please specify) 1 8%Don't know 2 15%

Since participating in the BOC program have you implemented any of the following types of energy efficiency projects? (Read list)


Respondents*

Lighting Controls 3 23%Energy efficient lighting 7 54%Energy efficient motors 2 15%VSDs 1 8%Compressed air projects 1 8%Energy management system project 2 15%Heating system improvements 3 23%Air conditioning improvements 3 23%Economizer 2 15%Water heating efficiency improvements 0 0%Other improvements 2 15%None 3 23%Don't know 1 8%


Appendix B B-5


At how many facilities did you implement any of the previously listed projects?


1 6 67%2 1 11%3 2 22%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%

10 or more 0 0%Don't know 0 0%

LC1A. The next few questions relate to the lighting controls you implemented. Were lighting controls specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 1 33%8 2 67%9 0 0%10 0 0%

Don't know 0 0%

Appendix B B-6


LC1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented these lighting controls, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 1 33%1 0 0%2 0 0%3 0 0%4 1 33%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 1 33%

Don't know 0 0%

LC2. Had you implemented a similar project prior to attending the BOC training?






LC4. What type of new lighting controls did you implement? (Do not read list)


Respondents*

Occupancy sensors 3 100%Daylighting 1 33%Don't know 0 0%


EEL1A. The next few questions relate to the energy efficient lighting you implemented. Was energy efficient lighting specifically recommended to you by a BOC course instructor or through BOC course materials?



Appendix B B-7




0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 1 14%6 0 0%7 2 29%8 2 29%9 0 0%10 0 0%

Don't know 0 0%

EEL1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this energy efficient lighting, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 1 14%1 0 0%2 0 0%3 0 0%4 1 14%5 1 14%6 0 0%7 1 14%8 0 0%9 1 14%10 1 14%

Don't know 1 14%

EEL2. How many lighting projects did you complete?


1 5 71%2 1 14%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%


EEL3. Had you implemented a similar project prior to attending Response (n=7) Percent of

Respondents

Appendix B B-8


the BOC training?




Yes 2 29%No 3 43%Don't know 2 29%

EEM1A. The next few questions relate to the energy efficient motors projects you implemented. Were energy efficient motors specifically recommended to you by a BOC course instructor or through BOC course materials?



EEM1B. How important was your experience in the Building Operator Certification Program in your decision to implement these energy efficient motors, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 2 100%9 0 0%10 0 0%

Don't know 0 0%

EEM1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented these energy efficient motors, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 1 50%3 0 0%4 0 0%5 0 0%6 1 50%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

EEM2. Had you implemented a similar project prior to attending the BOC training?


Yes 1 50%

Appendix B B-9


No 1 50%Don't know 0 0%




EEM4. Were these motors installed for HVAC end-uses or for industrial end-uses? (select all that apply)


Respondents*

HVAC end-use 1 50%Industrial end-use 1 50%Don't know 0 0%


VSD1A. The next few questions relate to the VSDs you implemented. Were VSDs specifically recommended to you by a BOC course instructor or through BOC course materials?



VSD1B. How important was your experience in the Building Operator Certification Program in your decision to implement these VSDs, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 1 100%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

VSD1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented these VSDs,


0 0 0%1 0 0%2 0 0%

Appendix B B-10


using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?

3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 1 100%9 0 0%10 0 0%

Don't know 0 0%

VSD2. Had you implemented a similar project prior to attending the BOC training?






VSD4. Were the VSDs installed on existing motors part of an HVAC system?



CA1A. The next few questions relate to the compressed air project you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%

7 0 0%

8 1 100%9 0 0%10 0 0%

Appendix B B-11


Don't know 0 0%

CA1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this compressed air project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 1 100%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

CA2. Had you implemented a similar project prior to attending the BOC training?






EMS1A. The next few questions relate to the EMS you implemented. Were EMSs specifically recommended to you by a BOC course instructor or through BOC course materials?



EMS1B. How important was your experience in the Building Operator Certification Program in your decision to implement the EMS, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 1 50%

Appendix B B-12


6 0 0%7 0 0%8 1 50%9 0 0%10 0 0%

Don't know 0 0%

EMS1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented the EMS, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 1 50%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 1 50%10 0 0%

Don't know 0 0%

EMS2. Had you implemented a similar project prior to attending the BOC training?






EMS5. Did you install a new energy management system after the BOC training?



EMS6. Did you make changes to an existing energy management system after the BOC training?



HS1A. The next few questions relate to the heating system project you implemented. Were heating system projects specifically


Yes 2 100%

Appendix B B-13


recommended to you by a BOC course instructor or through BOC

No 0 0%

Don't know 0 0%

HS1B. How important was your experience in the Building Operator Certification Program in your decision to implement the heating system project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 2 100%9 0 0%10 0 0%

Don't know 0 0%

HS1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented the heating system project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 1 50%4 1 50%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

HS2. Had you implemented a similar project prior to attending the BOC training?








Respondents*

Hot air furnace 0 0%

Appendix B B-14


Wall or floorboard radiator (steam, Hot Water or electric resistance) 0 0%

Steam, hot water or electric resistance coils in ventilation system. 2 67%

Space heaters 0 0%Heat pump, air source 0 0%Heat pump, ground source 0 0%Heat pump, water loop 0 0%Electric boiler 0 0%Gas boiler 1 33%Other (please specify) 1 33%Don't know 0 0%


AC1A. The next few questions relate to the air conditioning project you implemented. Was this air conditioning improvement specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 1 33%6 0 0%7 0 0%8 2 67%9 0 0%10 0 0%

Don't know 0 0%

AC1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this air conditioning system project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 2 67%

4 0 0%

5 0 0%6 0 0%7 0 0%

Appendix B B-15


8 0 0%9 1 33%10 0 0%

Don't know 0 0%

AC2. Had you implemented a similar project prior to attending the BOC training?






AC4. What is the primary cooling system type for the system you made improvements to?


Respondents*

Chiller - air-cooled 3 100%

Chiller - water or evaporatively cooled 0 0%Evaporative cooler 0 0%Fans 1 33%Direct Expansion - air-cooled packaged or split system cooling or heat pump

1 33%

Geothermal heat pump 0 0%Window or thermal units (PTAC/PTHP) 0 0%Other (please specify) 0 0%Don't know 0 0%


E1A. The next few questions relate to the economizer project you implemented. Was this economizer project specifically recommended to you by a BOC course instructor or through BOC course materials?



E1B. How important was your experience in the Building Operator Certification Program in your decision to implement this economizer project, using a scale of


0 0 0%1 0 0%2 0 0%

Appendix B B-16


0 to 10 where 0 is not at all important and 10 is extremely important?

3 0 0%4 0 0%5 1 50%6 0 0%7 0 0%8 0 0%9 1 50%10 0 0%

Don't know 0 0%

E1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this economizer project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 1 50%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 1 50%10 0 0%

Don't know 0 0%

E2. Had you implemented a similar project prior to attending the BOC training?






WH1A. The next few questions relate to the water heating project you implemented.Was this water heating project specifically recommended to you by a BOC course instructor or through BOC course materials?



Appendix B B-17




0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

WH1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this water heating project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%

6 0 0%

7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

WH2. Had you implemented a similar project prior to attending the BOC training?






WH4. What type of water heating improvements that produced energy savings did you implement?


*

Appendix B B-18


(Do not read list. Select all that apply)

Installed timeclock to turn off circulation pump after hours 0 0%

Installed heat recovery system 0 0%Installed a more efficient hot water heater or boiler?

0 0%

Insulated pipes(s) (How thick was the insulation and how many linear feet were installed?)

0 0%

Other 0 0%Don't know 0 0%


O1A. The next few questions relate to the other project you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 1 50%9 1 50%10 0 0%

Don't know 0 0%

O1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this other project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%

1 0 0%

2 0 0%

3 0 0%

4 0 0%

5 2 100%6 0 0%7 0 0%8 0 0%9 0 0%

Appendix B B-19


10 0 0%Don't know 0 0%

O2. Had you implemented a similar project prior to attending the BOC training?






Please indicate if you have performed maintanence on the cooling system equipment differently or more frequently or both since participating in the BOC training. Maintenance on the cooling system.


Differently 0 0%More Frequently 0 0%Both 0 0%No Change 9 75%Don't know 3 25%

Please indicate if you have performed maintanence on the heating equipment differently or more frequently or both since participating in the BOC training. Maintenance on the heating equipment.



Please indicate if you have performed maintanence on motors (including belt alignment and tension) differently or more frequently or both since participating in the BOC training.Motor maintenance, including belt alignment and tension.


Differently 0 0%More Frequently 0 0%Both 0 0%No Change 8 67%

Don't know 4 33%

Please indicate if you have performed maintanence on compressed air systems differently or more frequently or both since participating in the BOC training. Maintenance on compressed air system.



Appendix B B-20


Please indicate if you have performed electrical panel maintanence differently or more frequently or both since participating in the BOC training. Electrical panel maintenance.



Please indicate if you have performed ventilation maintanence differently or more frequently or both since participating in the BOC training. Ventilation maintenance.



Please indicate if you have performed other energy savings maintanence differently or more frequently or both since participating in the BOC training. Other maintenance.



At how many facilities did you make these changes to your maintenance practices?


1 2 100%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%


CS1A. The next few questions relate to the cooling system maintenance. Was this measure specifically recommended to you


Appendix B B-21


by a BOC course instructor or through BOC course materials?


CS1B. How important was your experience in the Building Operator Certification Program in your decision to implement this cooling system maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

CS1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this cooling system maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

CS2. What type of air conditioning improvements that produced energy savings did you implement?(Do not read list. Select all that apply)


Respondents*

Changes to cooling tower service 0 0%Changes to chiller bundle cleaning 0 0%

Appendix B B-22


Changes to condenser cleaning 0 0%Changes to refrigerant charge adjustment 0 0%Other changes 0 0%Don't know 0 0%


HE1A. The next few questions relate to the heating system maintenance you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

HE1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this heating system maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

HE2. What type of heating system improvements that produced energy savings did you implement?(Do not read list. Select all that apply)


Respondents*

Heat exchanger cleaning (Please provide capacity in BTU's) 0 0%

Appendix B B-23


Blowdown frequency (Please provide boiler capacity in BTU's and number of traps) 0 0%

Steam trap 0 0%Other 0 0%Don't know 0 0%


ACM1A. The next few questions relate to the air compressor maintenance you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?



ACM1B. How important was your experience in the Building Operator Certification Program in your decision to implement this air compressor maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

ACM1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this air compressor maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

ACM2. Please tell me all the changes you have made to your air compressor equipment maintenance. (Do not read list. Select all that apply)


Respondents*

Audible leak detection 0 0%Ultra-sonic leak detection 0 0%

Appendix B B-24


Pressure optimization 0 0%End-use isolation 0 0%Filter changes 0 0%System diagnostics 0 0%Other (please specify) 0 0%Don't know 0 0%


VM1A. The next few questions relate to the ventilation maintenance project you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

VM1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this ventilation maintenance, using a 0


0 0 0%1 0 0%2 0 0%3 0 0%

Appendix B B-25


to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?

4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 1 50%

Don't know 1 50%

VM2. What type of ventilation improvements that produced energy savings did you implement?(Do not read list. Select all that apply)


Respondents*

Economizer optimization/repair 0 0%Sensor Calibration 0 0%Setpoint optimization 0 0%Balancing 0 0%Filter changes 0 0%System diagnostics 0 0%Sealed leaks / replaced door gaskets 0 0%Other (please specify) 0 0%Don't know 1 100%


OM1A. The next few questions relate to the other maintenance you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

Appendix B B-26


OM1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this other maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

Do you think that there are certain barriers that may make it difficult for potential program participants to attend or complete the BOC training? What are they? (Don’t read list. Select all that apply.)


Respondents*

Time 5 38%Cost 8 62%Not aware of it 1 8%Supervisor approval 4 31%No barriers 4 31%Don't know 2 15%


Please indicate your level of satisfaction with the following elements of the BOC training. Course schedule.


Very Dissatisfied 0 0%Dissatisfied 0 0%Neither Satisfied nor Dissatisfied 1 8%Satisfied 4 31%Very Satisfied 8 62%Don't know 0 0%

Please indicate your level of satisfaction with the following elements of the BOC training. Course instructors.



Please indicate your level of satisfaction with the following elements of the BOC training. Tuition rebate application process.


Very Dissatisfied 0 0%Dissatisfied 0 0%Neither Satisfied nor Dissatisfied 2 15%

Appendix B B-27


Satisfied 2 15%Very Satisfied 7 54%Don't know 2 15%

Please indicate your level of satisfaction with the following elements of the BOC training. Tuition rebate amount.



Please indicate your level of satisfaction with the following elements of the BOC training. Time elapsed to receive tuition rebate.



Please indicate your level of satisfaction with the following elements of the BOC training. Overall experience with the BOC Program.



Please indicate if you had already completed energy budget implementation prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed.


Completed Prior to Training 2 15%Completed Before and After Training 4 31%Only Completed After Training 0 0%Not Yet Completed 3 23%Don't know 4 31%

Please indicate if you had already recorded energy use over time prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed.



Appendix B B-28


Please indicate if you had already set energy savings goals prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed.



Please indicate if you had already achieved energy savings goals prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed.



Have you participated in any other DCEO energy efficiency programs?



How important was the BOC course in your decision to participate in these other DCEO programs? (Read list)


Very important 1 100%Somewhat important 0 0%Neutral 0 0%Somewhat unimportant 0 0%Not important at all 0 0%Don’t know/Not applicable 0 0%

What is your current job title? (Do not read list)


Operations/Facilities operations manager 2 15%Maintenance manager 0 0%HVAC supervisor or technician 0 0%Engineering manager 1 8%Facilities manager 0 0%Engineer 2 15%Maintenance manager 1 8%General contractor 0 0%Building management specialist 0 0%Other engineering position 1 8%Other manager, team leader, supervisor 0 0%Other 6 46%

How many years have you worked in this role?

(n=13)Average Years 11.5

How many building operator staff (n=12)

Appendix B B-29


is there at your current location? Average Staff 7.3

How many of these staff have completed the BOC training (either Level 1 or Level 1&2)?

(n=11)

Average BOC Completion 2.6

Does your facility currently have plans to implement any of the

following types of energy efficiency projects?


Lighting controls 2 15%Energy efficient lighting 3 23%Energy efficient motors 2 15%

VSDs 1 8%

Compressed air projects 0 0%Energy management systems 2 15%Heating system improvements 3 23%Air conditioning improvements 3 23%Economizer on air handler 1 8%Water heating efficiency improvements 0 0%Other (please explain) 0 0%None 3 23%Don't know 3 38%

Did you initiate the plans for these upcoming projects, or did someone else in your organization?


I initiated the plans 4 31%Someone else initiated the plans 6 46%Don't know 3 23%

Did you initiate the plans for the upcoming project(s) due to information you gained through the Building Operator Certification


Yes 2 50%No 2 50%

Appendix B B-30


training? Don't know 0 0%

Appendix B B-31




College/University 2 5%Elementary 1 3%Grocery 0 0%Healthcare Clinic 0 0%Heavy Industry 2 5%High School/Middle School 0 0%Hospital 3 8%Hotel/Motel 0 0%Light Industry 0 0%Lodging Hotel/Motel 0 0%Manufacturing Facility 1 3%Medical 0 0%Office - High Rise 2 5%Office - Low Rise 4 10%Office - Mid Rise| 1 3%Religious Facility 0 0%Restaurant 0 0%Retail - Department Store 1 3%Retail - Strip Mall 0 0%Retail/Service 0 0%School (K-12) 1 3%Warehouse 0 0%Other (please specify) 21 53%Don't know 1 3%

Since participating in the BOC program have you implemented any of the following types of energy efficiency projects? (Read list)

Response (n=33) Percent of Respondents*

Lighting Controls 17 43%Energy efficient lighting 20 50%Energy efficient motors 5 13%VSDs 6 15%Compressed air projects 3 8%Energy management system project 10 25%Heating system improvements 14 35%Air conditioning improvements 14 35%Economizer 8 20%Water heating efficiency improvements 2 5%Other improvements 4 10%None 8 20%Don't know 1 3%


Appendix B B-32


At how many facilities did you implement any of the previously listed projects?


1 14 45%2 5 16%3 2 6%4 2 6%5 3 10%6 0 0%7 0 0%8 1 3%9 0 0%


LC1A. The next few questions relate to the lighting controls you implemented. Were lighting controls specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 3 18%6 3 18%7 1 6%8 6 35%9 1 6%10 3 18%

Don't know 0 0%

Appendix B B-33


LC1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented these lighting controls, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 1 6%1 0 0%2 0 0%3 1 6%4 1 6%5 6 35%6 3 18%7 2 12%8 1 6%9 0 0%10 2 12%

Don't know 0 0%

LC2. Had you implemented a similar project prior to attending the BOC training?


Yes 10 59%No 7 41%Don't know 0 0%



Yes 11 65%No 4 24%Don't know 2 12%

LC4. What type of new lighting controls did you implement? (Do not read list)


Occupancy sensors 14 82%Daylighting 7 41%Don't know 2 12%


EEL1A. The next few questions relate to the energy efficient lighting you implemented. Was energy efficient lighting specifically recommended to you by a BOC course instructor or through BOC course materials?


Yes 8 40%No 12 60%Don't know 0 0%

Appendix B B-34




0 1 5%1 0 0%2 0 0%3 1 5%4 3 15%5 3 15%6 1 5%7 1 5%8 4 20%9 2 10%10 4 20%

Don't know 0 0%

EEL1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this energy efficient lighting, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 1 5%4 2 10%5 5 25%6 2 10%7 1 5%8 3 15%9 0 0%10 6 30%

Don't know 0 0%

EEL2. How many lighting projects did you complete?


1 7 35%2 2 10%3 5 25%4 1 5%5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%


Appendix B B-35


EEL3. Had you implemented a similar project prior to attending the BOC training?


Yes 15 75%No 5 25%Don't know 0 0%



Yes 15 75%No 4 20%Don't know 1 5%

EEM1A. The next few questions relate to the energy efficient motors projects you implemented. Were energy efficient motors specifically recommended to you by a BOC course instructor or through BOC course materials?



EEM1B. How important was your experience in the Building Operator Certification Program in your decision to implement these energy efficient motors, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 1 20%4 0 0%5 0 0%6 1 20%7 0 0%8 2 40%9 0 0%10 1 20%

Don't know 0 0%

Appendix B B-36


EEM1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented these energy efficient motors, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 1 20%3 0 0%4 1 20%5 0 0%6 0 0%7 0 0%8 2 40%9 0 0%10 1 20%

Don't know 0 0%

EEM2. Had you implemented a similar project prior to attending the BOC training?





Yes 2 40%No 2 40%Don't know 1 20%

EEM4. Were these motors installed for HVAC end-uses or for industrial end-uses? (select all that apply)


HVAC end-use 3 60%Industrial end-use 3 60%Don't know 0 0%


VSD1A. The next few questions relate to the VSDs you implemented. Were VSDs specifically recommended to you by a BOC course instructor or through BOC course materials?



Appendix B B-37


VSD1B. How important was your experience in the Building Operator Certification Program in your decision to implement these VSDs, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 1 17%1 0 0%2 0 0%3 0 0%4 0 0%5 2 33%6 1 17%7 0 0%8 1 17%9 0 0%10 1 17%

Don't know 0 0%

VSD1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented these VSDs, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 1 17%3 0 0%4 1 17%5 1 17%6 0 0%7 0 0%8 2 33%9 0 0%10 1 17%

Don't know 0 0%

VSD2. Had you implemented a similar project prior to attending the BOC training?






VSD4. Were the VSDs installed on existing motors part of an HVAC system?



Appendix B B-38


CA1A. The next few questions relate to the compressed air project you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 1 33%1 0 0%2 0 0%3 1 33%4 0 0%5 1 33%6 0 0%

7 0 0%

8 0 0%9 0 0%10 0 0%

Don't know 0 0%

CA1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this compressed air project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 0 0%7 0 0%8 1 33%9 1 33%10 1 33%

Don't know 0 0%

CA2. Had you implemented a similar project prior to attending the BOC training?



Appendix B B-39





EMS1A. The next few questions relate to the EMS you implemented. Were EMSs specifically recommended to you by a BOC course instructor or through BOC course materials?



EMS1B. How important was your experience in the Building Operator Certification Program in your decision to implement the EMS, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 1 10%1 0 0%2 0 0%3 0 0%4 1 10%5 0 0%6 3 30%7 1 10%8 1 10%9 0 0%10 3 30%

Don't know 0 0%

EMS1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented the EMS, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 2 20%1 0 0%2 0 0%3 0 0%4 0 0%5 2 20%6 1 10%7 1 10%8 3 30%9 0 0%10 1 10%

Don't know 0 0%

EMS2. Had you implemented a similar project prior to attending Response (n=10) Percent of

Respondents

Appendix B B-40


the BOC training?





EMS5. Did you install a new energy management system after the BOC training?



EMS6. Did you make changes to an existing energy management system after the BOC training?



HS1A. The next few questions relate to the heating system project you implemented. Were heating system projects specifically recommended to you by a BOC course instructor or through BOC course materials?


Yes 2 14%No 12 86%Don't know 0 0%

HS1B. How important was your experience in the Building Operator Certification Program in your decision to implement the heating system project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 2 14%1 0 0%2 1 7%3 0 0%4 0 0%5 4 29%6 2 14%7 2 14%8 3 21%9 0 0%10 0 0%

Don't know 0 0%

Appendix B B-41


HS1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented the heating system project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 1 7%5 2 14%6 1 7%7 2 14%8 1 7%9 1 7%10 6 43%

Don't know 0 0%

HS2. Had you implemented a similar project prior to attending the BOC training?





Yes 6 43%No 6 43%Don't know 2 14%



Hot air furnace 2 14%Wall or floorboard radiator (steam, Hot Water or electric resistance) 5 36%

Steam, hot water or electric resistance coils in ventilation system. 3 21%

Space heaters 1 7%Heat pump, air source 1 7%Heat pump, ground source 1 7%Heat pump, water loop 2 14%Electric boiler 0 0%Gas boiler 8 57%Other (please specify) 4 29%Don't know 0 0%


Appendix B B-42


AC1A. The next few questions relate to the air conditioning project you implemented. Was this air conditioning improvement specifically recommended to you by a BOC course instructor or through BOC course materials?


Yes 3 21%No 11 79%Don't know 0 0%



0 5 36%1 0 0%2 1 7%3 0 0%4 0 0%5 5 36%6 0 0%7 0 0%8 2 14%9 1 7%10 0 0%

Don't know 0 0%

AC1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this air conditioning system project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%

4 1 7%

5 2 14%6 0 0%7 0 0%8 2 14%9 1 7%10 8 57%

Don't know 0 0%

AC2. Had you implemented a similar project prior to attending the BOC training?



Appendix B B-43





AC4. What is the primary cooling system type for the system you made improvements to?


Chiller - air-cooled 5 36%

Chiller - water or evaporatively cooled 5 36%Evaporative cooler 2 14%Fans 2 14%Direct Expansion - air-cooled packaged or split system cooling or heat pump

8 57%

Geothermal heat pump 2 14%Window or thermal units (PTAC/PTHP) 1 7%Other (please specify) 3 21%Don't know 0 0%


E1A. The next few questions relate to the economizer project you implemented. Was this economizer project specifically recommended to you by a BOC course instructor or through BOC course materials?



E1B. How important was your experience in the Building Operator Certification Program in your decision to implement this economizer project, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 1 13%4 0 0%5 2 25%6 1 13%7 1 13%8 1 13%9 1 13%10 1 13%

Don't know 0 0%

Appendix B B-44


E1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this economizer project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 2 25%3 0 0%4 1 13%5 0 0%6 0 0%7 1 13%8 2 25%9 0 0%10 2 25%

Don't know 0 0%

E2. Had you implemented a similar project prior to attending the BOC training?





Yes 4 50%No 3 38%Don't know 1 13%

WH1A. The next few questions relate to the water heating project you implemented.Was this water heating project specifically recommended to you by a BOC course instructor or through BOC course materials?



Appendix B B-45




0 0 0%1 0 0%2 0 0%3 1 50%4 0 0%5 0 0%6 1 50%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

WH1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this water heating project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 1 50%5 0 0%

6 0 0%

7 0 0%8 1 50%9 0 0%10 0 0%

Don't know 0 0%

WH2. Had you implemented a similar project prior to attending the BOC training?






Appendix B B-46


WH4. What type of water heating improvements that produced energy savings did you implement? (Do not read list. Select all that apply)


Installed timeclock to turn off circulation pump after hours 0 0%

Installed heat recovery system 1 50%Installed a more efficient hot water heater or boiler?

1 50%

Insulated pipes(s) (How thick was the insulation and how many linear feet were installed?)

0 0%

Other 0 0%Don't know 0 0%


O1A. The next few questions relate to the other project you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 0 0%6 1 25%7 0 0%8 2 50%9 0 0%10 1 25%

Don't know 0 0%

Appendix B B-47


O1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this other project, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 2 50%

1 0 0%

2 1 25%

3 0 0%

4 1 25%

5 0 0%6 0 0%7 0 0%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

O2. Had you implemented a similar project prior to attending the BOC training?






Please indicate if you have performed maintanence on the cooling system equipment differently or more frequently or both since participating in the BOC training. Maintenance on the cooling system.



Please indicate if you have performed maintenance on the heating equipment differently or more frequently or both since participating in the BOC training. Maintenance on the heating equipment.



Appendix B B-48


Please indicate if you have performed maintenance on motors (including belt alignment and tension) differently or more frequently or both since participating in the BOC training. Motor maintenance, including belt alignment and tension.



Please indicate if you have performed maintenance on compressed air systems differently or more frequently or both since participating in the BOC training. Maintenance on compressed air system.



Please indicate if you have performed electrical panel maintenance differently or more frequently or both since participating in the BOC training. Electrical panel maintenance.



Please indicate if you have performed ventilation maintenance differently or more frequently or both since participating in the BOC training. Ventilation maintenance.



Please indicate if you have performed other energy savings maintenance differently or more frequently or both since participating in the BOC training. Other maintenance.



Appendix B B-49


At how many facilities did you make these changes to your maintenance practices?


1 5 26%2 2 11%3 1 5%4 2 11%5 0 0%6 1 5%7 1 5%8 0 0%9 0 0%


CS1A. The next few questions relate to the cooling system maintenance. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?



CS1B. How important was your experience in the Building Operator Certification Program in your decision to implement this cooling system maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 1 13%1 0 0%2 0 0%3 0 0%4 0 0%5 1 13%6 1 13%7 1 13%8 1 13%9 1 13%10 2 25%

Don't know 0 0%

Appendix B B-50


CS1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this cooling system maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 3 38%1 1 13%2 0 0%3 0 0%4 1 13%5 2 25%6 0 0%7 0 0%8 0 0%9 0 0%10 1 13%

Don't know 0 0%

CS2. What type of air conditioning improvements that produced energy savings did you implement? (Do not read list. Select all that apply)


Changes to cooling tower service 4 50%Changes to chiller bundle cleaning 1 13%Changes to condenser cleaning 7 88%Changes to refrigerant charge adjustment 3 38%Other changes 3 38%Don't know 0 0%


HE1A. The next few questions relate to the heating system maintenance you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?



Appendix B B-51




0 2 22%1 0 0%2 0 0%3 0 0%4 1 11%5 0 0%6 1 11%7 1 11%8 1 11%9 2 22%10 1 11%

Don't know 0 0%

HE1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this heating system maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 2 22%1 0 0%2 1 11%3 0 0%4 1 11%5 2 22%6 0 0%7 0 0%8 1 11%9 0 0%10 2 22%

Don't know 0 0%

HE2. What type of heating system improvements that produced energy savings did you implement? (Do not read list. Select all that apply)


Heat exchanger cleaning (Please provide capacity in BTU's) 5 56%

Blowdown frequency (Please provide boiler capacity in BTU's and number of traps) 2 22%

Steam trap 3 33%Other 1 11%Don't know 2 22%


ACM1A. The next few questions relate to the air compressor maintenance you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?



Appendix B B-52


ACM1B. How important was your experience in the Building Operator Certification Program in your decision to implement this air compressor maintenance, using a scale of 0 to 10 where 0 is not at all important and 10 is extremely important?


0 0 0%1 0 0%2 0 0%3 0 0%4 0 0%5 1 33%6 1 33%7 1 33%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

ACM1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this air compressor maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 0 0%1 0 0%2 0 0%3 0 0%4 1 33%5 0 0%6 0 0%7 1 33%8 0 0%9 0 0%10 1 33%

Don't know 0 0%

ACM2. Please tell me all the changes you have made to your air compressor equipment maintenance. (Do not read list. Select all that apply)


Audible leak detection 0 0%Ultra-sonic leak detection 1 33%Pressure optimization 3 100%End-use isolation 1 33%Filter changes 1 33%System diagnostics 0 0%Other (please specify) 0 0%Don't know 0 0%


Appendix B B-53


VM1A. The next few questions relate to the ventilation maintenance project you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 1 11%4 0 0%5 0 0%6 1 11%7 3 33%8 1 11%9 0 0%10 3 33%

Don't know 0 0%

VM1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this ventilation maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 3 33%1 0 0%2 0 0%3 1 11%4 2 22%5 2 22%6 0 0%7 0 0%8 0 0%9 0 0%10 1 11%

Don't know 0 0%

Appendix B B-54


VM2. What type of ventilation improvements that produced energy savings did you implement? (Do not read list. Select all that apply)


Economizer optimization/repair 4 40%Sensor Calibration 6 60%Setpoint optimization 5 50%Balancing 4 40%Filter changes 6 60%System diagnostics 5 50%Sealed leaks / replaced door gaskets 2 20%Other (please specify) 0 0%Don't know 2 20%


OM1A. The next few questions relate to the other maintenance you implemented. Was this measure specifically recommended to you by a BOC course instructor or through BOC course materials?





0 0 0%1 0 0%2 0 0%3 1 13%4 0 0%5 0 0%6 1 13%7 1 13%8 1 13%9 3 38%10 1 13%

Don't know 0 0%

Appendix B B-55


OM1C. If you had not participated in the Building Operator Certification Program, how likely is it that your organization would still have implemented this other maintenance, using a 0 to 10 scale where 0 means you definitely WOULD NOT have implemented this measure and 10 means you definitely WOULD have implemented this measure?


0 1 13%1 0 0%2 0 0%3 2 25%4 1 13%5 3 38%6 0 0%7 1 13%8 0 0%9 0 0%10 0 0%

Don't know 0 0%

Do you think that there are certain barriers that may make it difficult for potential program participants to attend or complete the BOC training? What are they? (Don’t read list. Select all that apply.)


Time 18 45%Cost 10 25%Not aware of it 19 48%Supervisor approval 9 23%No barriers 4 10%Don't know 2 5%


Please indicate your level of satisfaction with the following elements of the BOC training. Course schedule.



Please indicate your level of satisfaction with the following elements of the BOC training. Course instructors.



Appendix B B-56


Please indicate your level of satisfaction with the following elements of the BOC training. Tuition rebate application process.



Please indicate your level of satisfaction with the following elements of the BOC training. Tuition rebate amount.



Please indicate your level of satisfaction with the following elements of the BOC training. Time elapsed to receive tuition rebate.



Please indicate your level of satisfaction with the following elements of the BOC training. Overall experience with the BOC Program.



Please indicate if you had already completed energy budget implementation prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed.



Appendix B B-57


Please indicate if you had already recorded energy use over time prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed.



Please indicate if you had already set energy savings goals prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed.



Please indicate if you had already achieved energy savings goals prior to completing BOC training, before and after the training, only completed them after attending BOC training, or have not yet completed.



Have you participated in any other DCEO energy efficiency programs?


Yes 6 15%No 33 83%Don't know 1 3%

How important was the BOC course in your decision to participate in these other DCEO programs? (Read list)


Very important 4 67%Somewhat important 1 17%Neutral 0 0%Somewhat unimportant 0 0%Not important at all 1 17%Don’t know/Not applicable 0 0%

Appendix B B-58


What is your current job title? (Do not read list)


Operations/Facilities operations manager 4 10%Maintenance manager 0 0%HVAC supervisor or technician 0 0%Engineering manager 0 0%Facilities manager 1 3%Engineer 5 13%Maintenance manager 1 3%General contractor 0 0%Building management specialist 0 0%Other engineering position 0 0%Other manager, team leader, supervisor 0 0%Other 29 73%

How many years have you worked in this role?

(n=38)Average Years 10.4

How many building operator staff is there at your current location?

(n=34)Average Staff 9.8

How many of these staff have completed the BOC training (either Level 1 or Level 1&2)?

(n=35)

Average BOC Completion 1.8

Does your facility currently have plans to implement any of the

following types of energy efficiency projects?


Lighting controls 11 28%Energy efficient lighting 17 43%Energy efficient motors 5 13%VSDs 6 15%Compressed air projects 3 8%Energy management systems 8 20%Heating system improvements 12 30%Air conditioning improvements 12 30%Economizer on air handler 7 18%Water heating efficiency improvements 4 10%Other (please explain) 0 0%None 3 20%Don't know 3 28%

Appendix B B-59


Did you initiate the plans for these upcoming projects, or did someone else in your organization?


I initiated the plans 12 30%Someone else initiated the plans 10 25%Don't know 18 45%

Did you initiate the plans for the upcoming project(s) due to information you gained through the Building Operator Certification training?



Appendix B B-60

Appendix C: Supervisor Survey Instrument

1. According to our records [number] of your employees completed the Building Operator Certification Course. Specifically, our records indicate that [employee name] attended the course. Is this correct?( ) Yes( ) No (If selected, go to follow up)( ) Don’t know (If selected, go to follow up)

1A. Did you have one or more employees attend the building operator certification course between June 1 2015 and May 31 2016?( ) Yes (If selected, go to 1A.1)( ) No (If selected, go to 1A.2)( ) Don’t know (If selected, go to 1A.2)

1A.1. What are their name(s)?

1A.2. We do not have any further questions for you.

Please scroll to the end of the survey and click submit.

1. How useful would you say the Building Operator Certification course was for helping your employees perform better in the following areas?

Very Useful Somewhat Useful Not Useful

Don’t know / Not Applicable

Identifying energy efficiency improvementsMonitoring facility energy use

Improving maintenance practicesIdentifying ways to improve occupant comfort 1A. Was the course useful for helping your [employee/employees] perform better at other

aspects of their job?

1B. (if any marked not at all useful) Could you explain more about why you think the course was not useful in improving certain areas of your employee’s(s’) job performance?

2. From what you have observed, [has your employee / have your employees] used or applied any of the concepts and/or methods taught in the Building Operator Certification courses?( ) Yes( ) No( ) Don’t know

Appendix C C-1


3. Since completing the Building Operator Certification, [has your employee / have your employees] undertaken or recommended any of the following energy efficiency projects?

Undertaken Recommended Don’t KnowInstallation of lighting controlsInstallation of energy efficient lightingInstallation of variable speed drives or variable frequency drivesEnergy saving improvements to compressed air systemsEnergy management system projectsEnergy saving improvements to heating systemEnergy saving improvements to cooling systemEnergy saving economizer projectWater heating efficiency improvements

4. Since completing the Building Operator Certification, [has your employee/have your employees] undertaken or recommended any other energy saving improvements not mentioned above?( ) Yes( ) No( ) Don’t know

4A. What other energy projects [has your employee / have your employees] undertaken or recommended since attending the Building Operator Certification courses? Please specify whether these projects were undertaken or recommended?

5. [Has your employee / Have your employees] performed any new operation and maintenance actions since completing the Building Operator Certification?( ) Yes( ) No( ) Don’t know

5B. What new operations and maintenance activities [has your employee / have your employees] performed since completing the Certification?

6. Would you say that your [employee performs / employees perform] performs some past operation and maintenance more often since completing the Building Operator Certification?( ) Yes( ) No( ) Don’t know

6B. What activities [has your employee / have your employees] performed more often since completing the Certification?

Appendix C C-2


7. Would you say that your [employee performs / employees perform] some past operation and maintenance activities better since attending the course?( ) Yes( ) No( ) Don’t know

7A. What activities [has your employee / have your employees] performed better since completing the Certification?

8. Does your organization face any of the following barriers to making energy efficiency improvements? (Select all that apply) ( ) Organization/company not committed to energy efficiency improvements( ) Lack of knowledge about ways to save energy( ) Not enough staff resources to plan efficiency projects( ) Other (Please specify)( ) Don’t know

9. Since attending the Building Operator Certification courses [Has your employee / Have your employees] shared what was learned with other employees?( ) Yes (If selected, go to 9A)( ) No( ) Don’t know

9A. How [has your employee / have your employees] shared the information learned in the course with other employees? (Select all that apply)( ) On the job demonstration of concepts or methods( ) Verbal explanation of concepts or methods( ) Written explanation of concepts or methods( ) Shared course materials( ) Don’t know( ) Other _____________

10. Based on your observations, has the Building Operator Certification training course led to your [employee / employees] having increased value to your organization in terms of the following?

Yes No Don’t Know

Saving energy at your facility? ( ) ( ) ( )

Saving money? ( ) ( ) ( )

Helping to improve occupant comfort? ( ) ( ) ( )

Advising in decisions about equipment operation or replacement? ( ) ( ) ( )

Having more productive interactions with contractors? ( ) ( ) ( )

Appendix C C-3


Undertaking, recommending, or influencing any energy-efficiency projects?

11. If you were hiring a new employee, how important would the candidates having a building operator certificate be to your hiring decision?( ) Very important ( ) Important( ) Not important( ) Not at all important( ) Don’t know

12. For current employees, how important is having a building operator certificate for promotion and/or advancement?( ) Very important( ) Important( ) Not important( ) Not at all important( ) Don’t know

13. What do you consider in deciding whether or not to send your employees to the Building Operator Certification Program training course? (Select all that apply)( ) Time/staff availability( ) Training costs( ) Location of the training ( ) Instructor/sponsor for the training( ) Length of training( ) Your organization’s approval process for sending employees to training( ) Employee professional development( ) Legal requirements( ) Gain/benefits for company of certification( ) The employee’s personal interest( ) Other( ) Don’t know

14. How important was the rebate to the decision to send your [employee / employees] to the Building Operator Certification courses?( ) Very important( ) Important( ) Not important( ) Not at all important( ) Don’t know

15. Would your [employee / employees] have been sent to the Building Operator Certification course if the rebate was not available?( ) Definitely would have( ) Probably would have( ) Probably would not have

Appendix C C-4


( ) Definitely would not have( ) Don’t know

16. Do you think your [employee’s/employees’] training in the Building Operator Certification Program training course has increased the likelihood that your organization will participate in energy efficiency programs, such as equipment incentive programs?( ) Yes (If selected, go to 16A)( ) No (If selected, go to 16B)( ) Don’t know

16 A. Why has it increased the likelihood of participating in the energy efficiency programs?

16B. Why has it not increased the likelihood of participating in energy efficiency programs?

17. Do you think your (employee’s / employees’) training in the Building Operator Certification Program training course has increased the likelihood that your organization will make investments in energy efficiency?( ) Yes (If selected, go to 17A)( ) No (If selected, go to 17B)( ) Don’t know

17A. Why has it increased the likelihood that your organization will make investments in energy efficiency?

17B. Why has it not increased the likelihood that your organization will make investments in energy efficiency?

18. Would you recommend the Building Operator Certificate Program to any of your colleagues in your organization or in other organizations?( ) Yes( ) No (If selected, go to 18A)( ) Don’t know

18A. Why would you not recommend the Building Operator Certification Program to your colleagues?

19. Do you expect your organization to enroll any other staff at your facility in the Building Operator Certificate Program?( ) Yes( ) No (If selected, go to 19A)( ) Don’t know

19A. Why do you not expect your organization to enroll any more staff in the Building Operator Certification Program to your colleagues?

Appendix C C-5

executive summary - ilsagfiles.orgilsagfiles.org/sag_files/evaluation_documents/draft... · web...

Documents