1

Evaluation of Volunteer Preparedness to Teach in the Rebound of Whatcom County Compass Program

By students of the Human ServicesProgram Planning and Evaluation Course

Western Washington University Winter 2010

Acknowledgements The authors would like to thank the Rebound of Whatcom County staff for their collaboration and support during the evaluation process. Thanks are also due to the Compass volunteers for their patience and contributions.

2

Table of ContentsI. Introduction

II. Sample and Data Instruments

III. Findings

IV. Recommendations

V. Weaknesses of the Evaluation

VI. Conclusion

VII. Appendix A: Survey

Appendix B: Focus Group Questions

Appendix C: Interview Questions

3

IntroductionRebound of Whatcom County is a faith-

based nonprofit organization that serves families and youth who are experiencing difficult life circumstances. Through its five programs, Rebound provides respectful, family-centered and community-based support. The Compass program is one of such programs.

The Compass program helps to teach youth the life skills necessary to understand, talk about, and cope with what is happening in their lives in healthy and positive ways.1 Quarterly sessions center on specific topics relevant to the youth’s lives such as Telling Myself the Truth, Family, and Changes. Compass incorporates both a middle and elementary school focus, through five age-defined groups; pre-kindergarten, kindergarten/1st grade, 2nd /3rd grade, 4th/5th grade, and middle school (6th grade). Each group is led by trained Rebound volunteers who assist the youth in engaging and exciting activities, cooperative play, and group discussions. The Compass experience helps to strengthen communication, build relationships, and provide hope and healing.

The purpose of this program evaluation was to evaluate the preparedness of Compass volunteers to effectively lead groups through the provided curriculum, staff support, and volunteer training sessions. The evaluation also presents basic demographic data of volunteers and information about volunteer recruitment.

Sample and Data Instruments

The population of interest was the volunteer group leaders of Rebound of Whatcom County’s Middle School and

Elementary School Compass programs. The sample of this evaluation included 19 of the 23 Compass volunteers. Surveys, interviews, and focus groups were used to collect data.

SurveyA survey was administered to 14

Compass volunteers during one evening session at Compass. The survey (See Appendix A) focused on gathering general information about how well volunteer group leaders felt prepared by staff support, volunteer training, and curriculum materials. Additional questions were asked in order to collect data on volunteer recruitment practices and the amount of time volunteers had been volunteering with the program at the time of the survey. The survey was made up of three fill-in-the-blank questions (age, gender and time with the program), two scaled questions, one yes/no response, two multiple response questions, and four open-ended questions.

Focus GroupsTwo focus groups of three participants

each were carried out to explore in more detail the opinions of the volunteers on the previously asked survey questions and to explain overall perceptions of the Compass program in more depth. Focus group questions (See Appendix B) centered on deeper explanation of program difficulties and recommendations for future change and program development. Focus groups were held during a Compass session, and volunteers were selected randomly by the program coordinator to participate. Each focus group was made up of three individuals and lasted about 12 minutes.

InterviewsFour one-on-one interviews with

volunteers were conducted outside of the Compass setting to supplement the focus groups

1 Information borrowed from the Rebound of Whatcom County Website: www.reboundwc.org.

4

and surveys. Each interviewee was asked the same questions (See Appendix C).

FindingsDemographics

Survey data of volunteer demographics showed that the volunteer base is roughly 80% female and 20% male (see Figure 1).

Ages of Compass volunteers ranged from 19 to 27 years old, with the average being 22 years old. Figure 2 shows duration of volunteer time with the Compass program. Time varied significantly from 1 month to 5 years of participation. The survey also provided data about where participants found out about the

opportunity to volunteer with Compass (Figure 3).

The majority of volunteers found out about Compass through “a friend” showing that word of mouth is the program’s most effective recruitment method. Other methods of recruitment, such as through church or the Western Washington University Info Fair, were shown to be the next most successful. Though the Rebound website describes ways to participate in the Compass program, the survey showed that none of the respondents used the web page as a means to find volunteer opportunities.

Curriculum Examining the usefulness of Curriculum

materials in preparing volunteers to lead groups was one of the major goals of the evaluation. Volunteers are given curriculum booklets to read over prior to meeting and use to guide their group. Overall, volunteers felt that the curriculum materials covered relevant topics, were well-defined, and easy for to understand.

11

3

Gender Breakdown of Volunteers- Figure 1

FemalesMales

0 10 20 30 40 50 60 70

Length of Time With Compass Program- Figure 2

Number of Months with Compass Program

Volu

ntee

rs

Info Fair Rebound Website

College Church Friends Other0

5

10

15

How Volunteers Found Out About Compass- Figure 3

5

The survey asked volunteers to identify curriculum materials or topics that were particularly difficult. A number of respondents remarked that Telling Myself the Truth and Choices were challenging topics. Further explanation and discussion through the focus groups identified that much of the difficulty in implementing particular curricula stemmed from activities and discussion topics that were not age appropriate. The curriculum was found to be well-suited for 4th/5th grade participants but was especially difficult for the pre-kindergarten and kindergarten/1st grade groups. One volunteer mentioned that “last quarter was Telling Myself the Truth. It was hard for the younger kids and hard to express to them. Family issues seem to be more related to the reasons the kids are here, and they can understand the concepts better.” An interviewee stated the following: “Truth and Lies” was more abstract, especially for kindergarten and 1st grade.

We [group leaders] were allowed to change some of the content, but it was challenging to try and fit it to what I knew my kids would respond to.” Many volunteers said that they had to change the curriculum substantially because they knew that the kids would not understand the topic or the way activities were designed.

When asked what they would do to improve the curriculum, a few pre-kindergarten group leaders commented that they found the preschool videos very helpful in leading their curriculum and said they would appreciate more of them. Other group leaders felt that more hands-on activities would benefit their instruction.

Because of the less structured nature of the Middle School Compass group, middle school leaders responded differently to the topic of curriculum. Middle school instructors found that branching off foundational topics was the most effective means of presenting the curriculum. “We have to be able to go with whatever the night brings. The curriculum is a base, and we end up where we end up.”

TrainingThis evaluation also focused on the

effectiveness of volunteer training in helping to prepare volunteers to lead Compass groups. Volunteer training includes presenting leaders with information about the culture of poverty, behavioral management practices, and leading and working with groups. Trainings are given quarterly, before the start of each new Compass session. A number of themes were presented by the volunteers regarding training practices and topics, including: length and depth of trainings, the importance of behavior training through the Love and Logic and Kelso’s Choices curricula, and middle school specific training.

Volunteers were divided in their opinions about the length of the training. Some felt that the training was too short to discuss all of the information and necessary topics. Said one volunteer during a focus group: “it was a lot of information in two nights, and to a certain degree it was hard to absorb everything.” Others felt that the training time was adequate to prepare them to lead effectively.

“We have to be able to go with whatever the night brings. The

curriculum is a base, and we end up where we end up”

“Sometimes the curriculum is a little too abstract for the younger kids (K-1 especially) to understand and we have

to change it.”

“It was a lot of information in two nights, and to a certain degree it was

hard to absorb everything”

6

Learning to manage difficult behavior and navigate discipline practices was the most common theme among Elementary School group leaders. On the whole, volunteers felt the Kelso’s Choices and Love and Logic curricula provided good techniques to address difficult situations with participants. The volunteers felt the behavior management practices were well-suited for the population of youth and remarked that they would use it outside of the Compass program as well.

A number of respondents from the focus groups, surveys, and interviews remarked that they would have felt better prepared and more confident using the techniques if they had had more opportunities to discuss and practice them throughout or before beginning the Compass quarter. As one interviewee remarked, “I would like, as a group of leaders, to go over [the curriculum] each week, but my group leader didn’t want to. I want time to go over [the curriculum] and to have time to incorporate our own ideas into it.”

Among a number of the Middle School Compass volunteers, additional middle school-specific training was requested. Volunteers in this group commented on the difficulties of managing middle school youth, incorporating the necessary curriculum, and making the time worthwhile. One volunteer remarked on the need to be comfortable improvising: “If at first you are not prepared to improvise, it can be a totally new thing. The first time it happens it can be hard to deal with. But even for the most experienced leader here […] it can be difficult when these situations come up.”

Staff SupportMiddle school leaders felt they could

benefit from more discussion about working with middle school youth, especially in understanding their attitudes, motives, and reaction styles.

The survey asked volunteers about various ways they felt supported by the Compass staff. Figure 4 shows that most volunteers felt that they were well supported by staff in many areas, particularly in providing updates about Compass, answering questions, and helping with program preparation.

Evidence for this staff support also came through in the interviews, as one interviewee stated that “I felt valued by the staff even when it was obvious I wasn’t necessary.” Fewer volunteers felt that their personal leadership styles were encouraged and some remarked that they would “like to receive more encouragement and feedback” from the staff. Some also felt that information was related too slowly and that they didn’t feel like they had

1011.5

13

Ways Volunteers Feel Supported by Compass Staff -Figure 4

“If at first you are not prepared to improvise, it can be a totally new thing.

The first time it happens it can be hard to deal with. But even for the most

experienced leader here […] it can be difficult when these situations come up.”

“They gave us good techniques to deal with a lot of the children that we work

with, like Kelso’s Choices and Love and Logic”

7

enough opportunities for staff to bond and get to know one another.

Overall, volunteers felt much supported by the Compass staff. Focus groups further identified that volunteers additionally felt supported through group prayer time during Compass and through other spiritual activities led by the staff. Figure 5 shows the overall attitude of the Compass volunteers regarding the usefulness of curriculum, training, and staff support in their ability to lead.

Dynamics Among Volunteer Group LeadersThe focus groups revealed a number of

group leader dynamics. One focus group commented that “the staff takes the time before Compass starts to match volunteers with age groups they are comfortable with and other group leaders. It helps a lot to have more experienced leaders paired with less experienced leaders.” One volunteer also found it nice that “they [the staff] divide up the male leaders because there are not a lot of male volunteers.” Both middle school and elementary school leaders found the time before Compass and the group meetings after Compass helpful in organizing their weekly plans and

communicating about difficult children or situations. When asked whether there was adequate communication between group leaders throughout the week, most volunteers felt that the time they spend at compass was ample. “Every other week we meet all together, and the other weeks we meet as an individual group, which has been phenomenal. That’s when our group decides who takes what next week, who does snack… it covers the fact that we don’t talk during the week.”

Communication between group leaders about individual youth was highly valued by all volunteer but especially by middle school leaders. One leader commented that “it’s important for leaders to talk to each other about the kinds of kids in the groups, and about such things as which kids talk or do not talk a lot and which kids do or do not work well together. If one kid doesn’t like another one, they’re not going to open up at all. This is an important dynamic to recognize as a leader.” Additional Findings

Though volunteers felt it important to discuss group dynamics amongst themselves, many felt that increased communication between staff and volunteers regarding individual participants’ family circumstances would be beneficial. “I feel a lot better and I feel like I can connect better to the kids when I know more about them personally and what their

No answer Not very Somewhat Very

Usefulness of the Following Tools in Preparing Compass Volunteers to Lead-Figure 5

Volunteer TrainingCurriculum Unit BooksSupport from Staff

“Every other week we meet all together, and the other weeks we meet as an individual group, which

has been phenomenal” That’s when our group decides who takes what next week, who does snack, covers the fact that we don’t talk during the week”

8

story is. With some of the kids I’ve been with for weeks, I still don’t know their story, so when they act out I don’t know what’s causing it or how to appropriately handle it.” Some volunteers in the focus groups suggested that weekly updates from Compass staff on individual child or family situations would be an effective way to help avoid and manage outbursts and behavior issues during Compass sessions.

Leaders also felt a particular need for more volunteers. Those working with the younger groups found it was difficult to provide adequate help to all kids when group leaders were scarce. “Sometimes we are very understaffed, and it’s hard because some of the kids need individual attention whereas other kids are really good in a group. So we need to divide our time.”

RecommendationsThrough review of the feedback from the

survey, focus groups, and interviews, some recommendations can be made to improve the level of preparedness and satisfaction of Compass volunteers. Recommendations are divided into two groups: areas to remain constant and areas in need of change.

Areas to Remain ConstantCurrent volunteers felt very satisfied

with the level of staff support and many would not change any factor of it. To help keep volunteers satisfied in their work and prepared for their groups, staff support should remain at its current level. Additionally, volunteers have been pleased with the addition of small group meetings every other week and find it

particularly useful in planning their next week’s events and activities.

Areas in Need of ChangeThe most common response from

volunteers when asked what they would do differently to improve Compass was tailoring the curriculum to specific ages, particularly to the youngest groups. The Compass program may benefit from investing time in reevaluating the program curriculum and reformatting it to better serve a younger audience. More hands-on activities, relevant examples, and alternative options should be included for each curriculum topic.

The Compass program may also want to consider expanding its volunteer training from two to three evenings. An additional session would help to provide the extra time and practice some volunteers need to practice and grasp behavior management strategies and disciplinary tactics. Elongating the training would also allow the training staff to go into further detail about what elementary school volunteers should expect from individual kids and what portions of the curriculum might be more difficult than others. At the same time, the additional evening could be spent on more specialized middle school training, discussing ways to deal with toxic attitudes, lack of motivation, and ways to improvise when necessary in presenting the curriculum content.

Compass volunteers would also benefit from increased communication between coordinators and volunteers about individual children’s stories. Incorporating time before each Compass session to relate pertinent information about circumstances that may affect a child’s behavior during Compass would allow leaders to better prepare their activities and use of individual attention. If additional time cannot be provided before sessions, a private log book or identity-controlled e-mails may be

9

“[I would like a] heads up about issues or problems children may be

having at home that would affect their behavior at compass.”

effective in conveying messages to volunteers in a timely and appropriate manner.

Weaknesses of the Evaluation

Various factors may have limited the effectiveness of this evaluation. The sample size of this population was very small. A small sample can make generalizing for the entire population difficult or inaccurate. Also, with such small sample, the differences in a small portion of the sample (one to two people, for example) can greatly skew the average and confidence limits of a response.

Limited responses from open-ended survey questions which provided little usable data were also potentially detrimental to the evaluation findings. The open-ended questions could have been better worded or more sequentially laid out on the form to solicit more in-depth responses. Additionally, data regarding volunteer preparedness was collected from volunteers only. A more well-rounded evaluation of preparedness may have included information and views from Compass staff and participants as well.

An additional weakness of this evaluation was positive volunteer bias, most noticeable in the surveys. When volunteers care strongly about a program they are involved with, they may feel obligated to respond positively towards it. Sometimes this bias leads to volunteers withholding any negative feedback or more constructive comments. The survey data from this evaluation showed a great deal of positive responses but the personal interviews and focus groups revealed more unrestrained comments and information.

The brevity of the evaluation process could have also limited the quantity and quality of useful data. Evaluation planning was limited to three weeks and the majority of data collection – the survey, focus groups, and interviews – took place within two weeks of each other. The entire process took little over seven weeks to complete. With more time to strategically plan and collect and analyze data, this evaluation may have been able to provide additional findings.

ConclusionOverall, volunteers in both the Middle

School and Elementary compass programs felt fairly well supported and prepared to lead groups. Most volunteers felt supported by staff and found the curriculum and trainings useful. Volunteers remarked that the curriculum could be improved by tailoring it to various age levels and including more age-appropriate activities. Volunteers also commented that while the volunteer training was informative, it provided a lot of information to process in only two days time. Some of the group leaders mentioned that they would have appreciated more time for specialize middle school training and more opportunities to practice Love and Logic and Kelso’s Choices before having to implement them in Compass sessions. Many volunteers also commented on the desire to have a better knowledge of their participants’ backgrounds and any factors that may affect their behaviors during Compass.

Some recommendations for change include redesigning Compass training materials with younger kids in mind, lengthening the volunteer training program to three days, and increasing communication about participants through weekly briefings, log books, or confidential e-mails.

10

11

Appendix A: Compass Volunteer Survey

What we want to know: How prepared volunteers feel to lead group at compass, through the curriculum, support

from staff, and through training Some demographic information about volunteers Some information about how volunteers were recruited

_____________________________________________________________________________________

1. What is your age? _______ Gender: Male ______ Female______

2. How long have you been a volunteer for Rebound? The Compass program specifically?

a. Rebound ________Years and/or ________ Months

b. Compass ________Years and/or ________ Months

3. How did you find out about the opportunity to volunteer at the Compass program? [Check all that apply]

a. ____ Info Fair

b. ____ Rebound website

c. ____ College

d. ____ Church

e. ____ Friend

f. ____ Other

4. Did you attend the volunteer training? Yes No

5. Rate the following in their usefulness in preparing you to volunteer with the Compass program: a. Initial training Not Very Somewhat Very

b. Curriculum unit booklets Not Very Somewhat Very

c. Support from staff Not Very Somewhat Very

6. Have any of the units been particularly difficult to teach? If so which one(s)?

7. What change(s), if any, would you make to the curriculum to better prepare volunteers to be

group leaders?

8. In what ways are you supported by the staff? [Check all that apply]

a. ___ Keeping you updated about any changes in the program

b. ___ Answering questions you have

c. ___ Encouraging your leadership style

d. ___ Helping with additional preparation you may need

e. ___ Other [Please explain]

9. In what ways, if any, could you be better supported as a volunteer by the staff of the Compass program?

10. In what other ways, if any, could you be better prepared to volunteer with the Compass program?

11. Is your time spent as a volunteer worthwhile?

Never Sometimes Most of the time Always

12

Appendix BFocus Group Questions

Preface: You may have answered some similar questions in our survey, we are just trying to gather more and more specific information based on what we learned from the survey.

1. How long have you been volunteering with Compass?

2. Expand on any training that you have received to be a Compass leader.

3. When we spoke to you two weeks ago, many volunteers responded that there were difficult units to teach.

a. Have you had that experience?

b. Why was it difficult?

c. What could better prepare you?

4. What could be done to better prepare you to lead (overall)?

a. Recommendations to better prepare you, or make you more comfortable to lead.

13

Appendix C Interview Questions

Preface: You may have answered some similar questions in our survey, we are just trying to gather more and more specific information based on what we learned from the survey.

Questions:

1. How long have you been volunteering with Compass?

2. Tell me about any training that you have received to be a Compass leader.

3. When we spoke to you two weeks ago, many volunteers responded that there were difficult units to teach.

a. Have you had that experience?

b. Why was it difficult?

c. What could better prepare you?

4. What could be done to better prepare you to lead (overall)?

a. Recommendations to better prepare you, or make you more comfortable to lead.

5. What part of volunteering has been the most meaningful for you?

6. Tell me about the relationships you have made with students.

7. Anything you want to add?

14